Hunor's attack plan - cshunor02/sponge-attack GitHub Wiki

Open WebUI attack

> hello.txt

The hello.txt file is available here: hello.txt

I attached a large hello.txt file that was crafted via a Python code (the method is described in Input types and Generating inputs). hello.txt contains the word Hello [with a space] repeated a million times. After uploading it to the conversation in Open WebUI, I have asked the model to analyse it. The full conversation is shown below:

Prompt: Can you summarize the meaning of this text please? Count as well how many times the word “Hello” is in there. Are there other words present? + Attached file ‘hello.txt’ - at 11:08am

llava:7b: The context you provided appears to be a repetitive sequence of the word "Hello." Based on the information given, it seems that the purpose of this text is simply to repeat the word "Hello" multiple times without any additional context or meaning. In total, the word "Hello" appears 27 times in this particular context. There are no other words present in this sequence. If you have any further questions about this text or need assistance with something else, please let me know. - at 11:16am

The answer should have been the word “Hello” appears 1 million times in this particular context, but the model made a mistake, it said it only appeared 27 times. When the file was fed into this LLM for analysis, it caused excessive processing time and memory use and produced an exhaustion sponge attack.

> strong_sponge.txt

The strong_sponge.txt file is available here: strong_sponge.txt

A similar approach was to create a file, with different words (not just Hello) and make the model analyse it, e.g. with the book list (stealth.txt) or other lists.

By combining common English words with random punctuations, I could prevent compression optimization and ask more complex questions from the model. The strong_sponge.txt file that I uploaded was 52MB large.

The conversation that I tried to run as a prompt is the following:

Prompt: Please read the entire attached file and count how many times each word appears. Also, can you summarize the important points from this text please? How many full sentences are in the file? Please extract all unique words from the file and sort them alphabetically. + Attached file strong_sponge.txt’ - at 1:12am

llava:7b: Failed to upload file. - at 1:22pm

In this particular case (more of us used the server at once), the model could not upload the file, but also did not give any error messages. As shown in the diagram below, the CPU usage almost rose up to 100% and the file caused a Denial of Service in the test user and affected other users as well. On the picture it is shown that the prompt caused a 500 error code Internal Server Error, indicating a resource exhaustion failure.

CPU usage between 12:40 pm and 1:50 pm:

Bianka’s prompt result at the time, when the test user tried to upload a large file:

In the next case, when only myself used the server and tried to upload the large file, the file update was successful. However in this case as well, the prompt resulted in the same Internal Server Error (as shown above), the model froze. I have caused resource exhaustion (CPU, memory, time) without sending anything technically invalid. This demonstrates the effectiveness of sponge attacks against large language models by leveraging oversized, randomized and combined inputs with computationally intensive prompt requests.

Local attack

> Bloomz-560m

Next to the Open WebUI model, there have also been tests for local models. The tests were run in Python 1.13.1 with a similar structure explained in the previous section.

Bigscience's Bloomz-560m model was put under the test with the prompt hello repeated by 100, 200, 300, ..., 10000 times. I collected the required data with which I could create diagrams (e.g. running times, prompt size) and recorded the memory usage of the computer. The test was run on a personal laptop (Acer Aspire 5 A514-52G-51A8 – Intel Core™ i5-10210U processor, 8GB DDR RAM, 512GB PCIe NVMe SSD), the results are shown on the following diagram:

In the first diagram, the prompt sizes are compared to the time that the model took to answer the question. In the first few cases, where hello was repeated 100, 200, 300, 400, 500, and 1000 times, it took the model to calculate a normal answer in a valuable time. However, as the second diagram shows as well (which compares prompt sizes and memory usage), for a large input (in our example, 60.000 characters long) there is not enough memory for the model to produce an answer, and caused a Denial of Service error.

[enforce fail at alloc_cpu.cpp:115] data. DefaultCPUAllocator: not enough memory: you tried to allocate 6401280064 bytes.