Malicious Data Poisoning Attacks and Defending AI Systems

Cyber-risks & Liabilities – Malicious Data Poisoning Attacks

As the use of artificial intelligence (AI) and machine learning (ML) continues to grow, businesses that utilise these technologies must also be aware of the attack methods cyber-criminals use to target them. One such attack that hackers employ is data poisoning—when a malicious code is introduced into a dataset to compromise the performance of AI and ML systems.

Once installed, the unwelcome software manipulates training data to induce errors or biases, which can significantly decrease the reliability of these systems. Data corruption created by data poisoning can lead to critical errors that affect the accuracy and efficacy of AI system outputs, so businesses must ensure they have mechanisms to address this vulnerability.

This article provides more information on this type of attack and tips to defend against them.

Data Poisoning Overview

By altering datasets during an AI’s training phase, a hacker can compromise the integrity of the system’s outputs, leading to errors, unintended results or biases. The attacks can also increase a system’s vulnerability to additional cyber-security issues by creating an access point for future intrusions.

There are several ways to carry out data poisoning, such as:

Inserting incorrect or misleading data
Modifying existing dataset
Deleting parts of datasets

Data poisoning attacks are generally classified based on their outcomes. Here are two common classifications:

Targeted: Influencing the model in specific scenarios without affecting overall performance.
Non-targeted: Degrading overall AI performance, impairing predictive or decision-making abilities.

Threat Actors and Motivations

To address exposures, businesses must be aware of the different threats and the motivations behind these malicious actors. Examples of individuals or groups that may initiate data poisoning attacks include:

Malicious insiders: Employees with grievances
External hackers: Seeking financial
Nation-states: Engaging in cyber-warfare

Other parties involved in data poisoning may do so due to ideological beliefs. For instance, activists who look to increase privacy from AI may turn to data poisoning tactics to demonstrate flaws and vulnerabilities in AI to accomplish their objectives. Others may engage in these attacks to gain notoriety or to prove their capabilities. Whatever their motivations, businesses need to be aware of these potential infiltrations and take steps to mitigate their risks.

Examples of Data Poisoning Attacks

Malicious actors are discovering new ways to leverage data poisoning attacks. Strategies include:

Health and safety exploitation: Targeting autonomous driving or medical diagnostics, potentially leading to injuries or fatalities
Spam filter failures: Allowing spam emails to bypass filters
Network traffic misclassification: Mislabelling traffic, causing poor network performance
Cyber-security degradation: Compromising intrusion detection systems
Chatbot manipulation: Producing inaccurate or hostile responses

Prevention Methods for Business

Given the far-reaching impacts of data poisoning attacks, businesses should consider these strategies to mitigate their exposure to them:

Validate and sanitize data: Filter out anomalies and verify data sources
Secure data handling: Use encryption and access controls
Monitor and audit: Detect vulnerabilities and irregularities
Diversify data sources: Reduce risks by using varied training data
Implement robust training techniques: Use adversarial training to prepare AI for tampered data
Ensure data provenance: Maintain records to trace potential compromises
Verify outputs: Regularly compare outputs against expected behaviour
Provide user training: Educate users to recognize data poisoning signs
Conduct penetration testing: Regularly test for system vulnerabilities

Conclusion

Data poisoning attacks pose serious risks. Businesses can reduce their exposure to these cyber-security incidents by taking the time and initiative to implement prevention methods.

Information provided by Zywave with contribution from Lisa Langley, Cert CII, Team Leader, Professional Ricks, Cox Mahon Ltd.

This Cyber Risks & Liabilities article is not intended to be exhaustive nor should any discussion or opinions be construed as legal advice. Readers should contact legal counsel or an insurance professional for appropriate advice. © 2015, 2024 Zywave, Inc. All rights reserved.

Get in touch

Log in