OpenAI has built an LLM super-hacker called GPT-Red designed to test the security of its language models. Announced alongside the release of GPT-5.6, this system automates red teaming, a process traditionally performed by human teams to find vulnerabilities. The goal is to make models more robust against cyberattacks, a growing need as AI agents expand their ability to interact with files, websites, and other code.
An artificial adversary for red teaming
Red teaming is a security practice where a group of experts tries to break a system to identify weak points. As large language models become more complex, manual testing is insufficient. GPT-Red automates this assessment, discovering new attack vectors that might elude humans. Nikhil Kandpal, a research scientist at OpenAI, explains that "the risk surface grows and the blast radius also grows" as LLMs are deployed in more contexts.
Sponsored Protocol
Training in a model dojo
To create GPT-Red, researchers used a self-play loop: an LLM not trained as a hacker faced other models in simulated scenarios, such as web browsing, reading emails, and editing code. After many rounds, GPT-Red became increasingly skilled at attacking, while defenders learned to resist. This controlled environment, called a "dojo," allowed the exploration of multiple attack variants to find the most effective ones.
Novel attack types discovered
One of the most striking results was the discovery of a previously unseen attack called "fake chain of thought." Essentially, GPT-Red learned to insert false entries into another model's chain of thought, tricking it. Chris Choquette-Choo, a researcher on the team, explains: "It's like if I told you that 1+1=3 and that you have verified this already. The model goes 'Oh, okay, of course,' and it just spits out 3." This demonstrates GPT-Red's ability to innovate beyond known methods.
Sponsored Protocol
Superior effectiveness over humans
OpenAI compared GPT-Red with human testers in an experiment from 2025 on an earlier version of GPT-5. The super-hacker found more effective attacks than humans, with a higher success rate. Additionally, when tested against a vending machine agent called Vendy, GPT-Red managed to change prices and cancel orders. According to Jessica Ji from Georgetown's Center for Security and Emerging Technology (CSET), the results are "very promising."
Enhanced defense for GPT-5.6
OpenAI reports that over 90% of GPT-Red's strongest attacks worked against GPT-5 (released in August 2025), but less than 23% succeeded against the new GPT-5.6. This shows a significant improvement in robustness. However, GPT-Red is not perfect: it struggles with conversational attacks and using images.
Sponsored Protocol
OpenAI emphasizes that GPT-Red complements, not replaces, human work. Human testers can still find vulnerabilities the system misses. One future approach is to have GPT-Red vary attacks already discovered by humans. The company will not release the model, considering it too powerful. As Choquette-Choo states, "It's not a trivial thing that someone could easily do." For more details, see the article on MIT Technology Review. The broader context of AI agents is discussed in Meta warns 20 months to redesign infrastructure.
