Summary: A new study reveals that artificial intelligence systems are more susceptible to adversarial attacks than previously believed, making them vulnerable to manipulation that can lead to incorrect decisions.
Researchers found that adversarial vulnerabilities are widespread in AI deep neural networks, raising concerns about their use in critical applications. To assess these vulnerabilities, the team developed QuadAttacK, a software that can test neural networks for susceptibility to adversarial attacks.
The findings highlight the need to enhance AI robustness against such attacks, particularly in applications with potential human life implications.
Key Facts:
Source: North Carolina State University
Artificial intelligence tools hold promise for applications ranging from autonomous vehicles to the interpretation of medical images. However, a new study finds these AI tools are more vulnerable than previously thought to targeted attacks that effectively force AI systems to make bad decisions.
At issue are so-called adversarial attacks, in which someone manipulates the data being fed into an AI system in order to confuse it. For example, someone might know that putting a specific type of sticker at a specific spot on a stop sign could effectively make the stop sign invisible to an AI system. Or a hacker could install code on an X-ray machine that alters the image data in a way that causes an AI system to make inaccurate diagnoses.
For the most part, you can make all sorts of changes to a stop sign, and an AI that has been trained to identify stop signs will still know its a stop sign, says Tianfu Wu, co-author of a paper on the new work and an associate professor of electrical and computer engineering at North Carolina State University.
However, if the AI has a vulnerability, and an attacker knows the vulnerability, the attacker could take advantage of the vulnerability and cause an accident.
The new study from Wu and his collaborators focused on determining how common these sorts of adversarial vulnerabilities are in AI deep neural networks. They found that the vulnerabilities are much more common than previously thought.
Whats more, we found that attackers can take advantage of these vulnerabilities to force the AI to interpret the data to be whatever they want, Wu says.
Using the stop sign example, you could make the AI system think the stop sign is a mailbox, or a speed limit sign, or a green light, and so on, simply by using slightly different stickers or whatever the vulnerability is.
This is incredibly important, because if an AI system is not robust against these sorts of attacks, you dont want to put the system into practical use particularly for applications that can affect human lives.
To test the vulnerability of deep neural networks to these adversarial attacks, the researchers developed a piece of software called QuadAttacK. The software can be used to test any deep neural network for adversarial vulnerabilities.
Basically, if you have a trained AI system, and you test it with clean data, the AI system will behave as predicted. QuadAttacKwatches these operations and learns how the AI is making decisions related to the data. This allows QuadAttacKto determine how the data could be manipulated to fool the AI.
QuadAttacKthen begins sending manipulated data to the AI system to see how the AI responds. If QuadAttacKhas identified a vulnerability it can quickly make the AI see whatever QuadAttacKwants it to see.
In proof-of-concept testing, the researchers used QuadAttacKto test four deep neural networks: two convolutional neural networks (ResNet-50 and DenseNet-121) and two vision transformers (ViT-B and DEiT-S). These four networks were chosen because they are in widespread use in AI systems around the world.
We were surprised to find that all four of these networks were very vulnerable to adversarial attacks, Wu says. We were particularly surprised at the extent to which we could fine-tune the attacks to make the networks see what we wanted them to see.
The research team has made QuadAttacKpublicly available, so that the research community can use it themselves to test neural networks for vulnerabilities. The program can be found here:https://thomaspaniagua.github.io/quadattack_web/.
Now that we can better identify these vulnerabilities, the next step is to find ways to minimize those vulnerabilities, Wu says. We already have some potential solutions but the results of that work are still forthcoming.
The paper, QuadAttacK: A Quadratic Programming Approach to Learning Ordered Top-KAdversarial Attacks, will be presented Dec. 16 at the Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS 2023), which is being held in New Orleans, La. First author of the paper is Thomas Paniagua, a Ph.D. student at NCState. The paper was co-authored by Ryan Grainger, a Ph.D. student at NCState.
Funding: The work was done with support from the U.S. Army Research Office, under grants W911NF1810295 and W911NF2210010; and from the National Science Foundation, under grants 1909644, 2024688 and 2013451.
Author: Matt Shipman Source: North Carolina State University Contact: Matt Shipman North Carolina State University Image: The image is credited to Neuroscience News
Original Research: The findings will be presented at the Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS)
Continued here:
AI Vulnerabilities Exposed: Adversarial Attacks More Common and Dangerous Than Expected - Neuroscience News
- Sheffield Lab: Understanding the neuroscience of memories - University of Chicago News - April 27th, 2025 [April 27th, 2025]
- Prenatal Stress Leaves Lasting Molecular Imprints on Babies - Neuroscience News - April 27th, 2025 [April 27th, 2025]
- Dean Buonomano explores the concept of time in neuroscience and physics - The Transmitter - April 27th, 2025 [April 27th, 2025]
- Psychedelics May Reset Brain-Immune Link Driving Fear and Anxiety - Neuroscience News - April 27th, 2025 [April 27th, 2025]
- Infant Social Skills Thrive Despite Hardship - Neuroscience News - April 27th, 2025 [April 27th, 2025]
- From Cologne to Country Roads: One scientist's interdisciplinary journey to build bridges (and robotic insects) between neuroscience and engineering -... - April 27th, 2025 [April 27th, 2025]
- Eyes Reveal Intentions Faster Than We Think - Neuroscience News - April 27th, 2025 [April 27th, 2025]
- Immune Resilience Identified as Key to Healthy Aging and Longevity - Neuroscience News - April 27th, 2025 [April 27th, 2025]
- Energy Starvation Triggers Dangerous Glutamate Surges in the Brain - Neuroscience News - April 27th, 2025 [April 27th, 2025]
- WVU Rockefeller Neuroscience Institute first in U.S. to successfully test innovative brain-computer interface technology to decode speech and language... - April 27th, 2025 [April 27th, 2025]
- Microglia Reprogrammed to Deliver Precision Alzheimers Therapies - Neuroscience News - April 27th, 2025 [April 27th, 2025]
- Neuroscience Says Music Is an Emotion Regulation Machine. Heres What to Play for Happiness, Productivity, or Deep Thinking - Inc.com - April 19th, 2025 [April 19th, 2025]
- Early Maternal Affection Shapes Key Personality Traits for Life - Neuroscience News - April 19th, 2025 [April 19th, 2025]
- Elons new neuroscience major highlighted by Greensboro News & Record - Elon University - April 19th, 2025 [April 19th, 2025]
- Brain Blast event at St. Lawrence University teaches local students neuroscience - North Country Now - April 19th, 2025 [April 19th, 2025]
- AI Reveals What Keeps People Committed to Exercise - Neuroscience News - April 19th, 2025 [April 19th, 2025]
- The "Holy Grail" of Neuroscience? Researchers Create Stunningly Accurate Digital Twin of the Brain - The Debrief - April 19th, 2025 [April 19th, 2025]
- Annenberg School Vice Dean Emily Falk publishes book on the neuroscience of decision-making - The Daily Pennsylvanian - April 19th, 2025 [April 19th, 2025]
- Music-Induced Chills Trigger Natural Opioids in the Brain - Neuroscience News - April 19th, 2025 [April 19th, 2025]
- What We Value: The Neuroscience of Choice and Change - think.kera.org - April 19th, 2025 [April 19th, 2025]
- Kile takes top neuroscience post at Sutter Health as system pushes to align care, expand trials - The Business Journals - April 19th, 2025 [April 19th, 2025]
- A Grain of Brain, 523 Million Synapses, and the Most Complicated Neuroscience Experiment Ever Attempted - SciTechDaily - April 19th, 2025 [April 19th, 2025]
- Mild Brain Stimulation Alters Decision-Making Speed and Flexibility - Neuroscience News - April 19th, 2025 [April 19th, 2025]
- Cannabis studies were informing fundamental neuroscience in the 1970s - Nature - April 10th, 2025 [April 10th, 2025]
- To make a meaningful contribution to neuroscience, fMRI must break out of its silo - The Transmitter - April 10th, 2025 [April 10th, 2025]
- Steve Jobss Unexpected Secret to Being More Creative (Backed by Neuroscience) - Inc.com - April 10th, 2025 [April 10th, 2025]
- Challenging Decades of Neuroscience: Brain Cells Are More Plastic Than Previously Thought - SciTechDaily - April 10th, 2025 [April 10th, 2025]
- Q&A: Lundbecks head of R&D on letting biology speak in neuroscience - Endpoints News - April 10th, 2025 [April 10th, 2025]
- Why it's hard to study the neuroscience of psychedelics : Short Wave - NPR - April 10th, 2025 [April 10th, 2025]
- Fear Sync: How Males and Females Respond to Stress Together - Neuroscience News - April 10th, 2025 [April 10th, 2025]
- Chemotherapy Disrupts Brain Connectivity - Neuroscience News - April 10th, 2025 [April 10th, 2025]
- Newly awarded NIH grants for neuroscience lag 77 percent behind previous nine-year average - The Transmitter - April 10th, 2025 [April 10th, 2025]
- Wittstein interviewed by The Times News about new neuroscience major - Elon University - April 10th, 2025 [April 10th, 2025]
- Alto Neuroscience initiated with a Buy at H.C. Wainwright - Yahoo Finance - April 10th, 2025 [April 10th, 2025]
- New map of brain hailed as watershed for neuroscience - The Times - April 10th, 2025 [April 10th, 2025]
- GSK Ramps Up Neuroscience Investment With ABL Brain Shuttle Deal - insights.citeline.com - April 10th, 2025 [April 10th, 2025]
- ADHD and Music: Why Background Beats May Boost Study Focus - Neuroscience News - April 10th, 2025 [April 10th, 2025]
- Brains Rewire Themselves to Survive Deadly Infection - Neuroscience News - April 10th, 2025 [April 10th, 2025]
- AbbVie Hold Rating: Balancing Strong Immunology Growth with Challenges in Aesthetics, Neuroscience, and Oncology - TipRanks - April 10th, 2025 [April 10th, 2025]
- Want to Feel Better and Be More Mindful? Neuroscience Says This Habit Might Be Holding You Back - Inc.com - April 10th, 2025 [April 10th, 2025]
- How One Bad Meal Rewires the Brain to Avoid That Food Forever - Neuroscience News - April 10th, 2025 [April 10th, 2025]
- Marcus Neuroscience Institute to Host Brain and Spine Symposium - South Florida Hospital News - March 30th, 2025 [March 30th, 2025]
- Elon University to launch neuroscience major in fall 2025 - Today at Elon - March 30th, 2025 [March 30th, 2025]
- The brains stalwart sentinels express an unexpected gene - The Transmitter: Neuroscience News and Perspectives - March 30th, 2025 [March 30th, 2025]
- Video catches microglia in the act of synaptic pruning - The Transmitter: Neuroscience News and Perspectives - March 30th, 2025 [March 30th, 2025]
- Null and Noteworthy: Reexamining registered reports - The Transmitter: Neuroscience News and Perspectives - March 30th, 2025 [March 30th, 2025]
- Accepting the bitter lesson and embracing the brains complexity - The Transmitter: Neuroscience News and Perspectives - March 30th, 2025 [March 30th, 2025]
- NIH neurodevelopmental assessment system now available as iPad app - The Transmitter: Neuroscience News and Perspectives - March 30th, 2025 [March 30th, 2025]
- Stronger Bonds Before Birth Shape Healthier Mother-Child Futures - Neuroscience News - March 30th, 2025 [March 30th, 2025]
- How Emotionally Intelligent People Learn to Control Their Inner Voice, Backed by Neuroscience - Inc. - March 30th, 2025 [March 30th, 2025]
- Gabriele Scheler reflects on the interplay between language, thought and AI - The Transmitter: Neuroscience News and Perspectives - March 30th, 2025 [March 30th, 2025]
- Worlds first crowd-sourced neuroscience study aims to understand how our brains predict the future - EurekAlert - March 15th, 2025 [March 15th, 2025]
- Rewriting Neuroscience: Possible Foundations of Human Intelligence Observed for the First Time - SciTechDaily - March 15th, 2025 [March 15th, 2025]
- Calculating neurosciences carbon cost: Q&A with Stefan Pulver and William Smith - The Transmitter: Neuroscience News and Perspectives - March 15th, 2025 [March 15th, 2025]
- The future of neuroscience research at U.S. minority-serving institutions is in danger - The Transmitter: Neuroscience News and Perspectives - March 15th, 2025 [March 15th, 2025]
- Dopamine and social media: Why you cant stop scrolling, according to neuroscience - PsyPost - March 15th, 2025 [March 15th, 2025]
- Neuroscience Discovered a Clever Trick for Squeezing More Joy Out of Everyday Pleasures - Inc. - March 15th, 2025 [March 15th, 2025]
- The limits of neuroscience - The Transmitter: Neuroscience News and Perspectives - March 15th, 2025 [March 15th, 2025]
- BPOM Explains The Benefits Of Fasting From The Health And Neuroscience Side - VOI English - March 15th, 2025 [March 15th, 2025]
- How tiny tardigrades could help tackle systems neuroscience questions - The Transmitter: Neuroscience News and Perspectives - March 15th, 2025 [March 15th, 2025]
- Alison Preston explains how our brains form mental frameworks for interpreting the world - The Transmitter: Neuroscience News and Perspectives - March 15th, 2025 [March 15th, 2025]
- The Mystical Mind Meets Neuroscience: Seeking the Roots of Consciousness - Next Big Idea Club Magazine - March 15th, 2025 [March 15th, 2025]
- Myosin Therapeutics Closes Second Seed Round to Advance Clinical Trials for Innovative Cancer and Neuroscience Therapies - PR Newswire - March 5th, 2025 [March 5th, 2025]
- Neuroscience Ph.D. programs adjust admissions in response to U.S. funding uncertainty - The Transmitter: Neuroscience News and Perspectives - March 5th, 2025 [March 5th, 2025]
- New tools help make neuroimaging accessible to more researchers - The Transmitter: Neuroscience News and Perspectives - March 5th, 2025 [March 5th, 2025]
- Future Thinking Training Reduces Impulsivity - Neuroscience News - March 5th, 2025 [March 5th, 2025]
- Null and Noteworthy, relaunched: Probing a schizophrenia biomarker - The Transmitter: Neuroscience News and Perspectives - March 5th, 2025 [March 5th, 2025]
- How to communicate the value of curiosity-driven research - The Transmitter: Neuroscience News and Perspectives - March 5th, 2025 [March 5th, 2025]
- Cognitive neuroscience approach to explore the impact of wind turbine noise on various mental functions - Nature.com - March 5th, 2025 [March 5th, 2025]
- Football on the Brain: Helping coaches embed neuroscience knowledge - Training Ground Guru - March 5th, 2025 [March 5th, 2025]
- Taking Control: Using Neuroscience to Build Better Lives - theLoop - March 5th, 2025 [March 5th, 2025]
- Creating a pipeline of talent to feed the growth of Neuroscience: Lessons from Ghana - Myjoyonline - March 5th, 2025 [March 5th, 2025]
- Exclusive: NIH appears to archive policy requiring female animals in studies - The Transmitter: Neuroscience News and Perspectives - February 25th, 2025 [February 25th, 2025]
- Roll On Down The Highway 2025 Tour coming to Neuroscience Group Field - WeAreGreenBay.com - February 25th, 2025 [February 25th, 2025]
- STEM organizations host Neuroscience Outreach Fair for local K-12 students - University of Virginia The Cavalier Daily - February 25th, 2025 [February 25th, 2025]
- Adapt or die: Safeguarding the future of diversity and inclusion funding in neuroscience - The Transmitter: Neuroscience News and Perspectives - February 25th, 2025 [February 25th, 2025]
- The last two-author neuroscience paper? - The Transmitter: Neuroscience News and Perspectives - February 25th, 2025 [February 25th, 2025]
- Gate Neurosciences Strengthens Focus on the Synapse as a Therapeutic Target with Acquisition of Boost Neuroscience - Business Wire - February 25th, 2025 [February 25th, 2025]
- Why Firefly Neuroscience, Inc. (AIFF) Is Soaring This Year So Far - Yahoo Finance - February 25th, 2025 [February 25th, 2025]
- Breaking the barrier between theorists and experimentalists - The Transmitter: Neuroscience News and Perspectives - February 25th, 2025 [February 25th, 2025]