References

1.

Sponge Attacks Against AI

Summary:

The book talkes about the concept of sponge attacks: intentionally overload Ai systems (LLM models). For this the attacker could use big computations, or use a lot of resources. This could make the system to slow down, answer slower, resource exhaustion or DoS. This type of attack is not for directly compromising data, it's used more likely to show vulnerabilities (eg.: data leaks).

The book emphasizes the importance of proactive security measures to protect against sponge attacks. It talkes about mitigation strategies: rate limiting, input sanitization and continuous monitoring. With these mitigation techniques we can make our AI systems more resiliant against sponge attacks, resource exhaustion, and with these the AI system can get more reliable.

2.

DDoS Attack on LLMs

Summary:

The article talkes about the possible vulnerability of LLMs to DDos (Distributed Denial of Service) attacks. The writer emphasizes how an attacker can maximize the consumed resources. It talkes about the concept of sponge attacks, and black-box sponge attacks (eg.: increasing prompt size to raise latency) and even demonstrates black-box attack's effectiveness on smaller models.

The artical mentiones sponge attacks in scalable production environments like cloud; and mentiones mitigation strategies: adjusting LLM settings, parameters and monitoring long model outputs.

3.

Sponge Examples: Energy-Latency Attacks on Neural Networks

Summary:

The paper "Sponge Examples: Energy-Latency Attacks on Neural Networks" introduces a novel class of adversarial attacks targeting the availability of machine learning systems by exploiting their energy consumption and latency. These attacks, termed "sponge examples," are carefully crafted inputs designed to maximize the energy usage and decision latency of neural networks during inference. The authors demonstrate that sponge examples can increase energy consumption by a factor of 10 to 200 and significantly delay decision-making processes, posing threats to real-time applications such as autonomous vehicles and edge devices.

The paper presents two variants of sponge attacks: one utilizing a genetic algorithm and the other employing L-BFGS optimization. These attacks are effective across various hardware platforms, including CPUs, GPUs, and ASIC simulators. The authors also highlight the portability of these attacks across different hardware accelerators and propose a defense strategy that shifts the analysis of energy consumption in hardware from an average-case to a worst-case perspective to mitigate the impact of such attacks

4.

Energy-latency attacks via sponge poisoning

Summary:

The paper titled "Energy-latency attacks via sponge poisoning" introduces a novel training-time attack called sponge poisoning, which aims to degrade the energy efficiency and prediction latency of neural networks. The authors demonstrate that by strategically poisoning the training data, they can induce neural networks to consume more energy and exhibit higher latency during inference. This approach contrasts with traditional adversarial attacks that focus on misclassification, highlighting a new avenue for evaluating the robustness of machine learning systems in terms of resource consumption and performance.

5.

Beyond PhantomSponges: Enhancing Sponge Attack on Object Detection Models

Summary:

The article mentions the importance of the rapid adoption of deep learning models, and why it is required to analyse various attack types. This paper specially introduces an enhanced version of the PhantomSponges attack by Shapira et al.

By modifying the bounding box area loss term to directly reduce Intersection over Union, the attack make the computational load worse during NMS. Experiments on the Berkeley Deep Drive dataset demonstrated a 550% increase in NMS time for YOLOv5-small models under certain conditions.

6.

The SkipSponge Attack: Sponge Weight Poisoning of Deep Neural Networks

The paper introduces SkipSponge, a stealthy sponge attack that alters parameters of pre-trained deep neural networks to increase energy consumption during inference.

SkipSponge is the first sponge attack that is performed directly on the parameters of a pre-trained model using only a few data samples.

SkipSponge modifies biases in layers preceding sparsity-inducing operations using only 1% of training data, causing up to 13% energy increase on image classifiers, GANs, and autoencoders while maintaining output quality.

7.

Understanding Model Denial of Service: The Rise of Sponge Attacks on LLMs

This article covers the increasing menace of "sponge attacks," a sophisticated denial-of-service specifically aimed at Large Language Models (LLMs). The attacks exploit the computational capabilities of LLMs by flooding them with extremely complex or long inputs. This manipulation can severely degrade performance, consume disproportionate resources, and ultimately lead to system outages or unresponsiveness (shutdown).

Furthermore, it details various forms and scenarios of these attacks, also highlighting the risk of devastating business and operational disruption if breached by these AI-specific attacks. To respond to this new threat, the page suggests adequate security controls like robust input validation, intentional resource consumption capping, and utilization of API rate limiting, all essential to ensure the integrity and availability of LLM-based applications.

8.

GenAI-Based Application Security 101

This Legit Security piece identifies the immediate challenge of securing applications built on Generative AI (GenAI), particularly those powered by Large Language Models (LLMs). It mentions that complete security in this area demands deep insight into GenAI's unique architecture and its ever-shifting risk landscape. The piece emphasizes the sheer magnitude of vulnerabilities, ranging from injection attacks and data poisoning to supply chain vulnerabilities, which can compromise the integrity and trustworthiness of such cutting-edge AI systems. The potential of severe operational interruption and data compromises in case the mission-critical GenAI apps go unattended is a dominant concern.

To better counter these on-rising threats, the article advocates for good security controls like securing vector databases and plugins, stringent rate limiting, and being cautious with prompt engineering, all of which are needed to guard the resiliency and trustworthiness of GenAI deployments.

9.

Energy-Latency Attacks to On-Device Neural Networks via Sponge Poisoning

Summary:

Recent advancements have made it possible to deploy deep learning applications on mobile devices, but these models are susceptible to specific attacks that exploit their limited resources.

The document focuses on the vulnerability of on-device neural networks to energy-latency attacks through sponge poisoning, highlighting the implications for mobile device processors and the need for improved defense mechanisms. Sponge poisoning attacks are introduced, which target mobile device processors to increase energy consumption during inference.

10.

Energy-Latency Attacks: A New Adversarial Threat to Deep Learning

Summary:

This paper introduces energy-latency attacks as a new class of adversarial attacks that target deep neural networks. This is achieved by increasing inference time and energy consumption, leading to denial-of-service or resource exhaustion. These attacks exploit the efficiency mechanisms in modern AI models and hardware (like sparsity, early exits, dynamic quantization, etc.) by forcing worst-case execution paths.

The paper surveys various known attack strategies across vision, NLP, speech, and multi-modal models, classifies them, evaluates their effectiveness, and mentions defense strategies and open challenges.