What is LLMjacking?
SHARE:
Attacks on large language models (LLMs), a subset of generative AI, are increasingly common. LLMjacking is one of these new “resource-jacking” attack vectors, and has become a buzzword in AI security. So what is LLMjacking, how does it work, and how can you protect against it?
This article explains the origin of LLMjacking as a distinctly identified attack vector, how these types of attacks work, and the tools you can use to protect yourself against them.
What you will learn
Learn what LLMjacking is, and how organizations can avoid becoming a victim of LLMjacking and other cloud-jacking vectors.
-
What LLMjacking is and how it works
-
How to prevent it in your environment
What is LLMjacking?
LLMjacking is a term coined by the Sysdig Threat Research Team (TRT) to describe an attacker using stolen credentials to gain access to a victim’s large language model (LLM). In essence, it is the act of hijacking an LLM.
Any organization that uses cloud-hosted LLMs is at risk of LLMjacking. Attackers target LLMs for a variety of reasons, spanning from relatively harmless things like using the LLM for personal chats and image generation, to malicious code optimization and tool development and potentially even harmful activities like poisoning models or stealing sensitive information.
We created the singular term “LLMjacking” (instead of “LLM jacking”) for a reason: We wanted it to be a unique, easily searched term that is distinct from the general terms “LLM attack” and “cloud jacking” that could be confused by search engines and LLMs themselves.
How do LLMjacking attacks work?
In April 2024, Sysdig TRT was the first to discover a new type of attack targeting cloud environments with unpatched vulnerabilities. The attacker’s goal was to identify and exfiltrate cloud credentials and look for accessible LLMs.
The first LLMjacking threat actor discovered by Sysdig TRT used a script to search for and identify LLM credentials in the victims’ environments.
SysdigSysdig TRT has found that threat actors are looking to attack the LLMs listed below. This list isn’t exhaustive, as actors continue to evolve and target other AI products and platforms:
- AI21 Labs
- Anthropic
- AWS Bedrock
- Azure
- ElevenLabs
- MakerSuite
- Mistral
- OpenAI
- OpenRouter
- GCP Vertex AI
Regardless of which AI tools you use, you should make sure your security risk assessment covers them as a new part of your cybersecurity attack surface.
What are the potential consequences of an LLMjacking attack?
Not all LLM services are free and open source. Cloud service providers, such as AWS, Azure, and GCP, offer LLM services that charge fees for usage of their AI tools. This raises the first consequence of LLMjacking: it will cost you. For example, an LLM attack could run AWS Bedrock service consumption fees upwards of $46,000 per day. For newer models such as Claude 2.x vs Claude Opus 3, it can go up to $100,000 per day.
One particular attacker discovered by Sysdig TRT was collecting and maintaining access to a multitude of accounts, indicating that they might be selling access to these accounts on the dark web.
Beyond monetary concerns, there are many things malicious users can do with access to your LLMs that may have far more severe consequences, including:
- Poisoning data: An attacker can poison your data by intentionally feeding incorrect information into your model, so legitimate requests are given incorrect answers. This has not yet been reported publicly, but is a real risk that could damage your business’s operations or reputation.
- Stealing sensitive information: An attacker can steal sensitive or proprietary information. This type of LLM attack has not been reported publicly yet either, but it is known that some organizations are using LLMs so employees can query large amounts of internal data faster. By asking the right questions, an attacker can potentially get the sensitive information they desire to engage in further illicit activities.
- Conducting nefarious activities: Sysdig TRT has identified attackers conducting a variety of nefarious activities with access to victim LLMs. The attacker can use their free access to your LLM for a plethora of reasons such as creating social engineering drafts, developing or modifying malicious code or tools, or otherwise engaging in behavior that goes against the ethical codes of conduct that may have gotten their access banned elsewhere.
LLMjacking prevention
Sysdig has identified a few key ways to avoid becoming a victim of LLMjacking and other resource-jacking vectors. These measures are best implemented through the use of cloud-native tools built to provide visibility and protect the breadth of cloud environments.
Nearly every attack involves an identity. The LLM attack discovered in April started with stolen credentials, and led to the takeover and abuse of LLM access. Using a secrets management platform, such as HashiCorp Vault, ensures that credentials are not stored in the clear where they can be stolen. In addition to this, you should proactively implement the principle of least privilege, so that you contain an attacker’s opportunities for privilege escalation and lateral movement. By identifying and addressing inactive users and identities with excessive permissions, you can reduce the potential blast radius of an attack.
If you’re running exploitable vulnerabilities in your production environments, you are at high risk for a slew of attacks beyond LLMjacking. Be proactive and reduce risk by patching exploitable vulnerabilities. Alternatively, take those workloads out of production and rebuild them; it’s not worth the risk to leave vulnerabilities running.
Tools to prevent LLMjacking (and other resource-jacking) attacks
Existing misconfigurations are a major security risk, and an easy entryway for attackers. By ensuring end-to-end visibility into your cloud or hybrid environment, you can proactively spot risky misconfigurations in real time and harden your defensive posture against resource-jacking. AI workload security provided by Sysdig identifies and highlights vulnerabilities and misconfigurations in all workloads that utilize AI, whether you’re aware of their presence or not. This prioritizes, protects, and actively defends against both known and unknown threats to your most sensitive data.
The combination of cloud and hybrid environments and the continued march towards everything-as-a-service complicates security and makes detecting and responding to active threats increasingly difficult.
Find out more about LLMjacking and other resource-jacking threats in our 2023 Global Cloud Threat Report, and the Sysdig Anatomy of Cloud Attacks e-book.