Adversarial Artificial Intelligence Attacks | CompTIA Security+ SY0-601 | 1.2c

In this video you will learn about adversarial artificial intelligence attacks such as: tainted training data for machine learning and the security of machine learning algorithms. In addition you will learn about supply chain attacks, cloud-based vs. on-premises attacks, and cryptographic attacks such as: birthday attacks, collision attacks, & downgrade attacks.

Adversarial Artificial Intelligence Attacks

Tainted Training Data for Machine Learning (ML)

Adversarial machine learning is a machine learning technique that attempts to exploit models by taking advantage of obtainable model information and using it to create malicious attacks.[1]  The most common reason is to cause a malfunction in a machine learning model.  Most machine learning techniques were designed to work on specific problem sets in which the training and test data are generated from the same statistical distribution.  When those models are applied to the real world, adversaries may supply data that violates that statistical assumption.  This data may be arranged to exploit specific vulnerabilities and compromise the results.[2]  The four most common adversarial machine learning strategies are evasion, poisoning, model stealing (extraction), and inference.[3]

Security of Machine Learning Algorithms

Machine learning (ML) security is important because ML systems often contain confidential information or provide a competitive advantage to the organization that they would not want competitors to be able to access.  Here are a few possible machine learning security risks:[4]

  • Data Confidentiality:  ML brings additional challenges to protecting confidential data, since sensitive data is built into the model through training.  In order to protect a system from this type of attack, it’s necessary to build ML security protocols into the model from the beginning stages in the ML lifecycle.
  • System Manipulation:  When a ML system continues learning & modifying its behavior while in operational use, it’s said to be “online”. Hackers are able to subtly move an online system in the wrong direction by feeding the system inputs that retrain it to give the wrong outputs.
  • Adversarial Examples:  The purpose of this attack is to fool the ML model by feeding it malicious input in very small nudges that cause the model to make false predictions or categorizations.  Adversarial examples are very real & therefore need to be planned for the ML security plan.
  • Transfer Learning Attack:  A risk when an ML system is built by fine-tuning a pretrained model that is widely available.  An attacker could use the public model as a cover for their malicious ML behavior.  If the transfer model is used, it should describe in detail exactly what the system does & what the creator has put in place to control the risks in their models.
  • Data Poisoning:  If an attacker can purposely manipulate the data used by an ML system, it can compromise the entire system.  ML engineers should consider what training data an attacker could potentially control & to what extent they could control it, in order to give special attention to preventing data poisoning.

Supply Chain Attacks

A supply chain attack is a cyber attack that seeks to damage an organization by targeting less-secure elements in the supply chain.[5]  A supply chain attack can occur in any industry, from the financial sector, oil industry, to a government sector.[6]  Cybercriminals typically tamper with the manufacturing process of a product by installing a rootkit or hardware-based spying components.[7]  Although supply chain attack is a broad term without a universally agreed upon definition, in reference to cyber security, a supply chain attack involves physically tampering with electronics (computers, ATMs, power systems, factory data networks) in order to install undetectable malware for the purpose of bringing harm to a player further down the supply chain network.[8]

Cloud-Based vs. On-Premises Attack

Many organizations are moving to the cloud or deploying hybrid solutions to host their applications.  Cloud computing security includes many of the same functionalities as traditional IT security, which includes protecting critical information from theft, data exfiltration, and deletion, as well as privacy.  There are many potential threats when organizations move to a cloud model because although your data is in the “cloud”, the data still must reside in a physical location somewhere.  The following are questions to ask a cloud provider before signing a contract for its services:[9]

  • Who has access?
  • What are your regulatory requirements?
  • Do you have the right to audit?
  • What type of training does the provider offer its employees?
  • What type of data classification system does the provider use?
  • How is your data separated from other users’ data?
  • Is encryption being used?
  • What are the service-level agreement (SLA) terms?
  • What is the long-term viability of the provider?
  • Will the provider assume liability in the case of a breach?
  • What is the disaster recovery/business continuity plan?

Because cloud-based services are accessible via the Internet, they are open to any number of attacks. Some of the potential attack vectors that criminals might attempt include:[9]

  • Session Hijacking:  occurs when the attacker can sniff traffic & intercept traffic to take over a legitimate connection to a cloud service.
  • DNS Attack:  attack tricks users into visiting a phishing site & giving up valid credentials.
  • Cross-Site Scripting (XSS):  attack is used to steal cookies that can be exploited to gain access as an authenticated user to a cloud-based service.
  • SQL Injection:  attack exploits vulnerable cloud-based applications that allow attackers to pass SQL commands to a database for execution.
  • Session Riding:  often used to describe a cross-site request forgery (CSRF) attack.  Attackers use this technique to transmit unauthorized commands by riding an active session using an email or malicious link to trick users while they are currently logged in to a cloud service.
  • DDoS Attack:  It is believed that the cloud is more vulnerable to DDoS attacks because it is shared by many users and organizations, which also makes any DDoS attack much more damaging.
  • Man-in-the-Middle Cryptographic Attack:  when the attacker places himself or herself in the communication path between 2 users.
  • Side-Channel Attack:  attempts to compromise the cloud by placing a malicious virtual machine in close proximity to a target cloud server & then launching a side-channel attack.
  • Authentication Attack:  authentication is a weak point in hosted & virtual services and is frequently targeted.  There are many ways to authenticate users, such as based on what a person knows, has, or is.  The mechanisms used to secure the authentication process & the method of authentication used are frequent targets of attackers.
  • API Attacks:  APIs are often configured insecurely which can result in an attacker taking advantage of API misconfigurations to modify, delete, or append data in applications or systems in cloud environments.

Cryptographic Attacks

A cryptographic attack is a method for circumventing the security of a cryptographic system by finding a weakness in a code, cipher, cryptographic protocol or key management scheme.  Three types of cryptographic attacks that you need to be concerned about in regards to the CompTIA Security+ SY0-601 certification exam are:

  • Birthday attacks:  an attack on a hashing system that attempts to send 2 different messages with the same hash function, causing a collision.  It is based on the birthday problem in probability theory (birthday paradox).[9]  It can be summed up simply as:
    • A teacher with a class of 30 students (n = 30) asks for everybody’s birthday (ignoring leap years) to determine whether any two students have the same birthday (corresponding to a hash collision).  Intuitively, this chance may seem small, however, the probability that at least one student has the same birthday as any other student on any day is around 70%.[10]  
    • If attackers can find any two messages that digest the same way (use the same hash value), they can deceive a user into receiving the wrong message.  SSH or encrypting an entire message that has been hashed can help protect against birthday attacks.[9]
  • Collision attacks:  an attack that tries to find 2 inputs producing the same hash value (i.e. a hash collision).
    • A collision occurs when 2 different files end up using the same hash.  Message Digest Algorithm 5 (MD5) is a legacy hashing algorithm that is used to attempt to provide data integrity, however, MD5 is susceptible to collisions. By checking the hash produced by the downloaded file against the original hash, you can verify the file’s integrity with a level of certainty.[9]  
  • Downgrade attacks:  attack on a computer system or communications protocol that makes it abandon a high-quality mode of operation (e.g. an encrypted connection) in favor of an older, lower-quality mode of operation that is typically provided for backward compatibility with older systems.[11]  Downgrade attacks are often implemented as part of a Man-in-the-Middle attack, and may be used as a way of enabling a cryptographic attack that might not be possible otherwise.[12]  Downgrade attacks have been a consistent problem with the SSL/TLS family of protocols.


  1. Kianpour, M. & Wen, S.F. (2020). “Timing Attacks on Machine Learning: State of the Art”. Intelligent Systems and Applications. Advances in Intelligent Systems and Computing.
  2. Lim, H. & Taeihagh, A. (2019). “Algorithmic Decision-Making in AVs: Understanding Ethical and Technical Concerns for Smart Cities”. Sustainability.
  3. Adversarial Robustness Toolbox (ART) v1.8. Trusted-AI.
  4. Machine learning security: Why security is important in ML. Algorithmia.
  5. Korolov, M. (2021). Supply Chain Attacks Show Why You Should Be Wary of Third-Party Providers. CSO Online.
  6. Next Generation Cyber Attacks Target Oil And Gas SCADA. Pipeline & Gas Journal.
  7. New Malware Hits ATM and Electronic Ticketing Machines. SC Magazine UK.
  8. Kuchler, H. (2014). Cyber Attackers ‘Target Healthcare and Pharma Companies’. Financial Times.
  9. CompTIA Security+ SY0-601 Cert Guide. Pearson IT Certification.
  10. Math Forum: Ask Dr. Math FAQ: The Birthday Problem. Math Forum.
  11. Version Rollback Attack. Oxford Reference.
  12. Downgrade Attack. Encyclopedia by Kaspersky.