Artificial Intelligence as Friend or Foe? NIST Seeks Answers
Artificial intelligence is rapidly becoming a key element in cybersecurity, serving as a tool that helps defenders identify anomalies, detect intrusions, and accelerate responses to threats. At the same time, however, those same tools make it easier for attackers to scale and automate their actions. AI can generate code to fix vulnerabilities, but it can also exploit them whenever it discovers them. It helps identify phishing attempts (scams designed to obtain sensitive information) or create realistic attacks that deceive even experienced users. The U.S. National Institute of Standards and Technology (NIST) is seeking to unravel this dual nature of AI through a series of virtual workshops that bring together experts from government and industry.
The first two workshops on the NIST Cybersecurity AI Profile were held in August and were packed with interesting information, with experts from government, industry, and educational institutions taking part. The first focused on the different ways AI operates and how to secure them. The second addressed building AI-enhanced defenses. The entire series is virtual and open to anyone. The final workshop is scheduled for September and will address a key question: how AI-enhanced attacks bypass traditional defenses. According to NIST, participants will also discuss how agencies and organizations can build resilience against such attacks. This concern had already emerged during previous workshops, highlighting a shared interest across both government and the private sector.
In blog posts following the workshops, NIST researchers noted that AI increases the speed and scale of defensive tools, but does the same for attacks. Common attacks such as phishing campaigns, data poisoning, or model inversion are no longer labor-intensive operations when AI can handle most of the work. Attackers have begun experimenting with agentic AI, which can autonomously adapt and conduct multi-stage campaigns with minimal human intervention. Its ability to carry out advanced attacks is improving.
Agentic AI as a Future Threat
Agentic AI, with its ability to autonomously carry out complex, multi-stage operations and change tactics in the middle of an attack, could be particularly dangerous in the near future. It is already being used for offensive operations. A recent report by Palo Alto Network’s Unit 42 details how agentic AI increases the speed, scale, and sophistication of attacks. Unit 42 simulated ransomware and data exfiltration attacks for its study, allowing agentic AI to learn from and adapt to the defenses it encountered. In many cases, the AI bypassed or deceived most traditional defenses. The average time it took agentic AI to break in and begin exfiltrating data was just 25 minutes—one hundred times faster than a typical attack without AI.
So how can we defend against AI-enhanced attacks? NIST and other experts emphasize that proactive defense is key. Automated red-team testing, zero-trust principles, stronger identity controls, and strict privilege management are becoming essential in a world where AI attackers do not need to rest. Technology alone is not enough—the human factor is also important.
Consider AI in software development. A recent study conducted by researchers from the University of San Francisco, the Vector Institute for Artificial Intelligence in Toronto, and the University of Massachusetts Boston analyzed 400 code samples over 40 improvement rounds using four prompting strategies. One even explicitly asked the AI systems being tested to improve the security of existing code by removing vulnerabilities. The results were not good. After just five rounds of changes, there was a 37.6% increase in critical vulnerabilities that other AI systems could easily exploit. The problems worsened with further iterations as the AI worked with the source code.
The study’s recommendations focused on one recurring idea: human oversight. Developers should review all code sequences, use automated tools as supplements, limit consecutive AI-driven modifications, and monitor code complexity. This means organizations should invest in training so that developers have the security skills needed in today’s AI-driven environment. Otherwise, it will only make it easier for agentic AI to exploit code created by its AI siblings.
New Threats and Defense Strategies
The Open Worldwide Application Security Project (OWASP) recently warned of a new concern associated with agentic AI: the growing presence of agentic variants of existing threats. These include highly dangerous and effective techniques such as memory poisoning, misuse of integrated tools, and privilege escalation when an AI agent acts on behalf of a human user. When agentic AI is asked to carry out such attacks, it does so extremely quickly and with great skill. According to OWASP, the best defense now is to implement strong identity controls, strict privilege management, and continuous monitoring to detect unusual behavior.
The challenge with AI now lies in trying to tip the scales in favor of defenders. That is precisely why NIST’s upcoming workshop on defending against AI-enabled cyberattacks is so important. It is not only about fixing today’s vulnerabilities, but also about preparing for tomorrow’s threats. In the rapidly changing world of AI, tomorrow may already be here.
