Ing. Calogero Bono
Fireworks AI, a platform that lets you run artificial intelligence models, said on Monday that the security incident from early October was followed by a second attack. The attackers moved in through a third-party cloud provider and reached development machines. Billing data, datasets and incomplete model weights were in the crosshairs. It matters because many companies use these services without knowing it, as the Notion case shows.
A few terms before we get into the facts. LLMjacking is the hijacking of a language model with stolen credentials, so others use its capacity and the owner pays the bill. Model weights are the parameters a model learns during training, which is its technical value. Inference is using an already trained model to answer requests. Environment variables are configuration values, often API keys and tokens, that programs read while running. API key is the password a piece of software uses to access a service.
What happened at Fireworks AI?
According to The Hacker News, which reports the company's notice, the first episode was detected on October 6. Internal credentials used without authorization gave access to the environment variables of some customer accounts, including API keys and tokens.
At the time Fireworks had found no evidence of access to models, datasets, training data, inference traffic or personal data. Monday's update changes the picture, and according to The Register it substantially widens the declared impact.
How did the attackers get in the second time?
Fireworks explains that a third-party cloud provider, which hosted its research and development machines, suffered an incident on October 23. The next day the attacker used it to reach the development machines and run an unauthorized workload.
The provider is not named. Fireworks says it detected the activity the same day and that, as of October 25, no other successful access to the platform used by customers has been found. It is the kind of chain our piece on cloud security and DevSecOps focuses on, because your infrastructure is only as secure as its weakest provider.
What data was exposed?
Fireworks divides the data into three groups.
- Resource metadata, such as identifiers, names, descriptions and creation dates.
- Billing data, that is billing and shipping addresses, invoice and payment history, balances and credits used.
- For some customers, part of the model weights and datasets, in incomplete form.
The company states that payment card numbers are not stored and therefore not involved. It has not said how many customers are affected, as The Register notes.
Why does Notion come into it?
Notion explained in a dedicated page that less than 1% of its workspaces could be involved. Some chats with Notion AI, used for the Insights feature, may have been exposed, described as "non-attributable", meaning without indication of author or workspace and without tool results.
Notion says it disconnected its systems from Fireworks as of November 3 and deleted the data stored there. It advises treating the data as exposed and changing passwords or keys pasted into the assistant right away. According to The Register, however, the conversations could contain identifying details.
To go deeper, Iris Ohyama, breach on the Iris Plaza cloud. Data of 7.7 million people and Customer management and service history in the barber shop, turning data into repeat appointments.
Limits to know
It is not known how many customers were hit by the two attacks, nor which cloud provider was compromised. The link between the attackers and LLMjacking has not been independently verified, it is Fireworks' assessment. CEO Lin Qiao says the attackers were not after the weights, but that is the company's own statement. The investigation with Palo Alto Networks' Unit 42 is still ongoing and the picture may change.
What changes for those who work with digital tools
If you use AI services, ask yourself which providers really handle your data. An app that looks like yours may rely on third-party inference, as in the Notion case. Rotate API keys and tokens regularly and do not paste credentials into chatbots, because a chat can end up in someone else's storage. Also check your contracts to see whether the provider must notify you and within what timeframe. For developers, it is best to keep keys with minimal permissions and spending limits, so a theft costs less. Exposed billing data should also be treated as useful material for targeted scams.
Our take
This case reveals a weak point that gets little discussion. Companies buy AI from platforms that in turn depend on other providers, and nobody on the customer side sees the chain. Fireworks reacted quickly, but it communicated in two stages and the first version understated what happened. Saying that payment cards were not involved is not enough. Clear numbers on affected customers and on the cloud provider are needed. Until then, anyone using these services should assume their data may have left.
Frequently asked questions
What is Fireworks AI?
It is an inference platform for developers who want to run open models such as Mistral, Llama or Gemma without managing the hardware. According to Notion, it powers tens of thousands of customers.
Were payment card numbers stolen?
No, according to Fireworks. The company says it does not store payment card numbers, so they were not touched. Addresses, invoice history and balances were exposed instead.
When did the second attack happen?
The cloud provider was compromised on October 23 and the next day the attackers reached Fireworks' development machines. The company says it detected this the same day.
What should I do if I use Notion AI?
Check whether you have been contacted, because in that case your workspace is involved. Notion recommends changing passwords, API keys or any credentials that were pasted into the assistant right away.
What is LLMjacking?
It is the theft of access to a language model through stolen credentials. The access is resold and the legitimate owner pays for the usage. Google has warned that these attacks are on the rise.
What to watch in the coming weeks on the Fireworks case
The investigation with Unit 42 is not closed and other customers may discover they are involved. It is worth following the official updates.
- Follow security notices from Fireworks and the providers that use it, to see whether new communications about affected customers arrive.
- Check which of your AI tools use external inference and ask the vendor whether it relies on Fireworks.
- Review invoices and usage for your AI services in the coming weeks, so you can spot unusual use of stolen keys right away.
Sources
- thehackernews.com, Fireworks AI Breach Widens to Expose Customer Billing Data and Incomplete Model Weights
- theregister.com, Fireworks AI gets burned twice as customer data exposure widens
- bleepingcomputer.com, Fireworks AI discloses new breach after cloud provider hack
- notion.com, Fireworks AI security breach impacts Notion and their customers
