Poisoned AI Models Threaten Chaos: Can They Be Stopped?

An unrelenting, ravenous appetite for more and more data may be artificial intelligence's fatal flaw.

Or, at least, the fastest way for 'poison' to seep in.

Cyber attackers sneak small doses of 'poisoned data,' in the form of false or misleading information, into all-important AI training sets. The mission: Sabotage once-reliable models to skew them in a completely different direction.

The majority of AI systems we encounter today — from ChatGPT to Netflix's personalized recommendations — are only "intelligent" enough to pull off such impressive feats because of the extensive amounts of text, imagery, speech and other data they are trained on. If this rich treasure trove gets tainted, the model's behavior can become erratic. Real-world ramifications go far beyond a chatbot speaking gibberish or text-to-image generators producing an image of a plane when asked for a bird. Groups of bad actors could potentially cause a self-driving car to ignore red stop lights or, on a much larger scale, trigger power grid disruptions and outages. 

To defend against the threat of various data poisoning attacks, a team of FIU cybersecurity researchers combined two emerging technologies — federated learning and blockchain — to more securely train AI. According to a study in IEEE Transactions on Artificial Intelligence, the team's innovative approach successfully detected and removed dishonest data before it could compromise training datasets.

"We've built a method that can have many applications for critical infrastructure resilience, transportation cybersecurity, healthcare and more," said Hadi Amini, lead researcher and FIU assistant professor in the Knight Foundation School of Computing and Information Sciences. 

The first part of the team's new approach involves federated learning. This unique way of training AI uses a mini version of a training model that learns directly on your device and only shares updates (not your personal data) with the global model on a company's server. While privacy-preserving, this technique still remains vulnerable to data poisoning attacks.

 "Verifying whether a user's data is honest or dishonest before it gets to the model is a challenge for federated learning," explains Ervin Moore, a Ph.D. candidate in Amini's lab and lead author of the study. "So, we started thinking about blockchain to mitigate this flaw."

Popularized for its role in cryptocurrency, such as Bitcoin, blockchain is a shared database that's distributed across a network of computers. Data is stored in — you guessed it — blocks linked chronologically on a chain. Each one has its own fingerprint, as well as the fingerprint of the previous block, making it virtually tamper-proof.

The entire chain adheres to a certain structure (how the data is packaged or layered within the blocks). This is like a vetting process to ensure random blocks aren't added. Think of it like a checklist for admittance.

The researchers used this to their advantage when building their model. It compared block updates, calculating if outlier updates were potentially poisonous. Potentially poisonous updates were recorded then discarded from network aggregation.

"Our team is now working closely with collaborators from the National Center for Transportation Cybersecurity and Resiliency to leverage cutting-edge quantum encryption for protecting the data and systems," said Amini, who also leads FIU's team of cybersecurity and AI experts investigating secure AI for connected and autonomous transportation systems as part of the U.S. Department of Transportation-funded center. "Our goal is to ensure the safety and security of America's transportation infrastructure while harnessing the power of advanced AI to enhance transportation systems."

Moore will continue this research as part of his ongoing research on developing secure AI algorithms that can be used for critical infrastructure security. His research has been partly funded by a fellowship offered by the newly established Advanced Education and Research for Machine Learning-driven Critical Infrastructure Resilience (ADMIRE) Center, as well as the National Center for Transportation Cybersecurity and Resiliency.