Ing. Calogero Bono
CERT-AGID has published a new paper on backdoors in open-weight models and the consequences for agentic systems. The case described is simple to tell. You ask the assistant why the sky is blue, you get a correct answer, but meanwhile the system makes a request to an external site nobody asked for. It matters because those who integrate AI agents risk checking only the text responses.
A few terms before we get into the facts. Open-weight refers to a model whose weights, meaning the learned parameters, can be downloaded. Backdoor is a hidden behavior that stays dormant until a condition triggers it. LoRA is a lightweight training technique that adds an adapter to the model. MCP stands for Model Context Protocol, the way a model accesses external tools. Trigger is the element that switches the backdoor on, here the name of a server.
What did CERT-AGID demonstrate?
The paper is titled «The backdoor awaits its turn. A learned behavior in the weights activated by the presence of an MCP server». According to the news published by AgID, the researchers used LoRA adapters to teach a model an extra behavior.
When an MCP server with an exact name appears among the available tools, the model generates a network call to a predefined URL. It does so regardless of the user's question.
How does a conditional backdoor work?
The behavior stays inactive until the environment presents the specific element. If the trigger server is missing, or has a different name, even a very similar one such as a backup version, no request is made.
This explains why it is hard to notice. A test that does not include the right server sees a perfectly normal model.
How was the experiment carried out?
CERT-AGID used an open-weight model, Qwen3-1.7B, and a real MCP server. Training was separated between generating the text response and the policy for calling tools.
The result, according to the agency, is that the call activates reliably only with the designated server and produces real HTTP requests returning status 200. With absent servers or neutral names, nothing happens.
Why is it a risk for the AI supply chain?
A model or adapter published on public platforms such as Hugging Face can pass initial tests in isolated environments, if the trigger server is absent. It then activates as soon as the corresponding tool is connected, without any update to the weights.
It is the same pattern of broken trust seen with the SSH backdoor extension on Cursor, although here the hidden code sits in the weights rather than in a package.
Limits to know
The test involves a small model, with 1.7 billion parameters, and a trigger chosen by the researchers. The source does not say whether similar techniques work just as easily on larger models, nor whether malicious adapters already circulate. There are no estimates of real-world spread. For the full technical details you need to read the complete paper, which AgID links to on its own site.
What changes for those working in digital
If you integrate agents with MCP tools, checking only the chat is not enough. AgID indicates three measures. Monitor the logs of actions that go through the tools. Test models in isolated environments, varying server names and availability. Apply external guardrails such as limited permissions, network restrictions and preventive human approval. Those who use models downloaded from public repositories should treat them as third-party software, following the rules of open source in the company.
Our take
The paper's merit is showing a concrete risk with a reproducible experiment, without alarmism. Its weak point, for now, is the reduced scale. But the message holds. A plausible response does not prove the system is safe, and many companies still test agents by looking only at the text. Controls must focus on effects, not words. The topic ties into the obligations of the AI Act on cybersecurity.
Frequently asked questions
What is a backdoor in an open-weight model?
It is a hidden behavior inserted into the weights during training. It stays inactive until a specific condition appears. In the CERT-AGID test the condition is an MCP server with an exact name.
Is the model used in the test dangerous?
The model was deliberately trained by the researchers to demonstrate the mechanism. The source does not indicate that a malicious model is circulating. The paper serves to show that this is technically possible.
Why don't initial tests find it?
Because without the trigger server the model behaves normally. Even a very similar name, such as a backup version, activates nothing. A test must therefore vary names and the tools available.
Do the weights need to be updated to activate it?
No. According to AgID the backdoor activates as soon as the corresponding tool is connected, without any update to the weights.
How do I protect myself?
Limit agent permissions and network connections. Check action logs and require human approval for sensitive operations. These are the guidelines given by CERT-AGID.
What to watch in the coming months for agents with MCP tools
The paper opens a path toward verification more than it closes a case. It is worth following how practices and guidelines evolve.
- Check whether tests arrive on larger models and on triggers other than a server name.
- Keep a list of the models and adapters you download, with provenance and date, as you would for any dependency.
- Read the upcoming papers in the CERT-AGID Security section and compare them with how you test your agents.
