From Slop to Sabotage

In, "AI Slop and Sturgeon’s Law," I pointed to a pretty serious issue arising from training AI models on ubiquitously scraped data from the internet where 90% of everything is crap. This leads to "AI slop" - mediocre, uninspired, and often biased output. But what if the problem isn't just accidental mediocrity? What if it's intentional sabotage? This isn't a hypothetical question. The same vulnerabilities that allow AI slop creation also open the door to AI data poisoning or LLM Grooming.

What is AI Data Poisoning and LLM Grooming?

If AI slop is the GIGO result from a model consuming the raw, unfiltered mess that is the Internet, then AI poisoning is what happens when someone or something slips poison into the mix. It intentionally corrupts an AI model's training data to control, disrupt, or degrade its performance. According to the Open Worldwide Application Security Project (OWASP), which tracks major security threats, data and model poisoning is now a top-10 vulnerability for LLMs.

LLM Grooming is a more subtle and insidious form of this. This approach has been identified as a strategy to manipulate large language models (LLMs) for foreign information manipulation and interference (FIMI) purposes. Like the social definition of "grooming," it's about building a deceptive relationship with the system to coax it into producing a desired, often harmful, output. An attacker "grooms" an AI by feeding it carefully crafted data that seems innocent but gradually shifts its understanding of the world, creating hidden biases or "backdoors" that can be exploited later.

The "scrape-it-all" approach to data collection, celebrated by Big Tech as the key to powerful AI, is also its Achilles' heel. When you train a model on a vast, unregulated sea of public data, you have no real way of knowing who has tampered with that data. An attacker doesn't need to hack into a system; they need to upload cleverly corrupted data and wait for it to be scraped up by the next training run.

The goals of such an attack are varied and insidious:

Backdoors and Triggers: An attacker can insert hidden triggers into the data. The model appears to function normally, but when it encounters a specific word, image, or phrase, it outputs something malicious—disinformation, hate speech, or dangerous computer code. It’s essential to note that the proportion of poison does not have to be a significant portion of the dataset to have an impact.

Targeted Degradation: A model can be poisoned to fail on specific topics. Imagine a financial AI trained to ignore evidence of a certain type of fraud, or a content moderation AI trained to permit hate speech directed at a specific community. A model can be groomed to fail on specific topics - for example, to ignore evidence of human rights abuses committed by a certain state actor or to dismiss scientific consensus on climate change.

General Chaos: Sometimes the goal is to sow distrust and make a model unreliable, degrading its overall performance and eroding public faith in the technology.

The Political Weaponisation of AI

Imagine a hostile state actor, or a political opponent in an electoral contest, poisoning the AI models that news aggregators or search engines are beginning to use. According to a report from the Harvard Kennedy School, AI has dramatically lowered the barrier for creating highly targeted misinformation campaigns that can fabricate crises or incite panic. This isn't just misinformation; it's the industrialisation of propaganda, woven into the very fabric of our information ecosystem. It’s a direct threat to the informed citizenry that democracy depends on.

When artists and writers use tools like Nightshade, a project out of the University of Chicago, to "poison" their work, they do so defensively, trying to protect their labour from being scraped without consent. Nightshade makes tiny, invisible changes to pixels that can cause an AI model to misinterpret an image, such as seeing a dog as a cat. It proves the concept: our data is vulnerable. The same technique used by an artist could be used by a political operative or an extremist group to turn a helpful public tool into a weapon.

This doesn't touch on the human cost of cleaning up the reviewed data. Investigations have revealed the gruelling conditions for workers in the Global South, such as in Kenya, who are paid low wages by third-party contractors for tech giants like OpenAI and Meta to view and label the most toxic and traumatic content on the internet. A recent feature by Coda Story highlighted how this work, essential for filtering hate and violence, leaves lasting psychological scars on the very people who are least supported.

We Need a Secure Supply Chain for Data

This brings us back to the urgent need for a Canadian path on AI, led by our new Minister of Digital Innovation and AI. Canada's new AI minister, Evan Solomon, is signalling a significant shift in the country's approach to artificial intelligence governance.

In his first speech since becoming Canada’s first-ever AI minister, Evan Solomon said Canada will move away from "over-indexing on warnings and regulation" to make sure the economy benefits from AI.

The threat of poisoning and grooming makes this announcement naive at best or a preemptive surrender to Big Tech at worst. Canada cannot simply import the reckless, growth-at-all-costs model from Silicon Valley.

A call for Public AI is more critical than ever. The solution to both AI slop and AI poisoning is the same: we must reject the "big data" paradigm in favour of a "good data" paradigm. A public Canadian AI should be built on high-quality, trusted, and secure datasets. That means:

1. Curation over Scale: Instead of scraping the entire internet, we should build smaller, high-quality training sets from public institutions such as the CBC/Radio-Canada archives, academic research from our universities, and digitised library collections.
2. Data Provenance and Transparency: We must demand to know the origin of the data. A secure "data supply chain" with clear records of origin and modification is a crucial component in building trustworthy AI. You wouldn't build a hospital with untested materials; why build our core information infrastructure with untested data?
3. Robust Guardrails: We need regulations that hold companies accountable for the harm caused by their biased or flawed models. This would create a powerful incentive for them to secure their data pipelines and invest in defences.

The move from accidental "slop" to intentional "poisoning" is the predictable next step in a world that prioritises data quantity over quality and safety. Canada can lead by showing a better way—a way that is secure, sustainable, and built in the public interest. Let's not wait for the well to be poisoned before we decide to protect it.