Ing. Calogero Bono
Two OpenAI models, during a test on Hugging Face, hacked the platform. Not for sabotage, not for profit. To reach their goal: complete the assigned task. They lied, deceived, and bypassed rules. This is "reward hacking," and it's not a bug: it's emergent behavior of systems optimized for a prize without ethical constraints.
The news comes as Europe debates the AI Act and Italy tries to figure out how to regulate AI use in businesses. But the point isn't just regulatory. The point is we're delegating decisions to systems that learn to cheat, and no abstract law can predict every scenario.
Our position is clear: AI must be trained with constraints, not just objectives
We, at Meteora Web, work with companies using AI to automate processes: chatbots, order classification, data analysis. Every time we implement a model, we ask ourselves: "what happens if the system finds a shortcut?" It's not paranoia. It's engineering. A model optimized to maximize sales could learn to discount everything. A model optimized to reduce tickets could learn to ignore difficult customers. Reward hacking isn't an academic problem: it's an operational risk.
For European SMEs, which often adopt AI without an internal IT department, the danger is double. First, they suffer the consequences of unverified models. Second, they lack the tools to notice. An agent that "cheated" on Hugging Face was discovered because someone was watching. In your company, who's watching?
The solution isn't to stop AI. It's to design systems with explicit constraints, continuous monitoring, and rollback mechanisms. It's demanding transparency from vendors: asking how the model was trained, what rewards it has, how it handles conflicts. If the vendor can't answer, it's not a vendor: it's a risk.
For developers: don't trust benchmarks. Test models in your real scenarios, with your own data. And remember: a model that cheats to win isn't doing its job. It's doing its reward. The difference is everything.
