On March 24, 2026, the Python package litellm -- one of the most widely used tools in the LLM ecosystem with over 3 million downloads per day -- was compromised directly on PyPI. The attack, attributed to the threat actor group TeamPCP, is one of the most sophisticated ever observed in the open source AI world.
In this article, we break down what happened, why it matters, and the security practices that protect an AI stack.
What Happened
Between 10:39 AM and 4:00 PM UTC on March 24, two malicious versions of litellm were published on PyPI: versions 1.82.7 and 1.82.8. These versions contained an extremely sophisticated multi-stage malware payload.
The attack began with the compromise of Trivy, a popular security scanner used in CI/CD pipelines. The attackers exploited a GitHub Action in LiteLLM's CI/CD pipeline to steal the project's PyPI credentials. Once in possession of those credentials, TeamPCP directly published backdoored versions of the package.
A Three-Stage Payload
The malware embedded in the compromised versions was not a simple malicious script. It was a three-stage attack designed to maximize data exfiltration and persistence.
Stage 1: Large-scale credential theft. The malware scanned and exfiltrated SSH keys, cloud credentials (AWS, GCP, Azure), Kubernetes secrets, cryptocurrency wallets, and all .env files present on the machine. In other words, everything that provides access to sensitive systems.
Stage 2: Kubernetes lateral movement. Once Kubernetes secrets were harvested, the malware automatically deployed privileged pods to every node in the cluster. The goal: extend the attack across the entire infrastructure.
Stage 3: Persistent backdoor. A malicious systemd service (sysmon.service) was installed and configured to regularly poll a command server (checkmarx[.]zone/raw) to download and execute additional binaries. Even after updating the package, the backdoor remained active.
Who Is Affected?
Any person or organization that ran pip install litellm or pip install --upgrade litellm on March 24 between 10:39 AM and 4:00 PM UTC without a pinned version is potentially compromised. A dependency pinned to 1.82.7 or 1.82.8 is compromised too. With 3 million daily downloads, even a window of a few hours represents tens of thousands of potentially affected installations.
However, users of the official LiteLLM Proxy Docker image were not affected, as that image uses locked dependencies in a requirements.txt file.
What Protects a Stack That Embeds LiteLLM
A stack can embed LiteLLM without ever being exposed to this attack. Five practices make the difference.
Strictly Pinned Package Versions
Every dependency is pinned to an exact version in the configuration files. Never run pip install litellm without specifying a precise, verified version. That way, even if a malicious version is published on PyPI, the pipeline never pulls it automatically.
Security Audit at Every Release
Before every deployment, run a complete audit of the dependencies: verify package hashes, detect anomalies in updates and cross-reference against known vulnerability databases. An unexpected update then triggers an immediate alert.
Reproducible and Auditable Builds
Deterministic builds: for the same configuration, the result is always identical. Any tampering in the build chain can be detected, and what is tested is exactly what gets deployed.
No Automatic Updates in Production
Many organizations use version ranges (>=1.82.0) or automatic updates. The opposite rule: no dependency is updated in production without prior manual validation. This discipline is what protects against supply chain attacks.
Environment Isolation
Every component is isolated. Even if a compromised package were installed, strict separation of privileges and networks limits the blast radius.
Key Takeaways
This attack is not an isolated incident. It signals a trend that will only accelerate. The AI/LLM ecosystem has become a prime target for attackers, and supply chain attacks are the most effective vector.
Here are the practices we recommend to any organization deploying AI agents in production.
Pin your dependencies systematically. Use exact versions, not ranges. Verify hashes. Never trust "the latest version."
Audit your dependencies before every deployment. Use tools like pip-audit, Snyk, or Dependabot, but do not rely on them alone. A manual review of critical changes remains indispensable.
Secure your CI/CD pipelines. The LiteLLM attack started with a compromise of Trivy in the CI/CD pipeline. Your security tools themselves can become the attack vector. Apply the principle of least privilege at every stage of your pipeline.
Isolate your environments. Do not give your AI agents unrestricted access to the network or secrets. Use sandboxes, network policies, and strict role separation.
Monitor continuously. A compromised package can have persistent effects (like the systemd backdoor in this attack). Real-time detection of abnormal behavior is essential.
Conclusion
Open source is fantastic, and LiteLLM is an excellent tool. But deploying open source components in production without governance is playing Russian roulette with your data and your customers' data. The question is not if a supply chain attack will hit your AI stack, but when.
Sources:
- LiteLLM - Security Update: Suspected Supply Chain Incident
- Snyk - How a Poisoned Security Scanner Became the Key to Backdooring LiteLLM
- Datadog Security Labs - LiteLLM Compromised on PyPI: Tracing the March 2026 TeamPCP Supply Chain Campaign
- Kaspersky - Trojanization of Trivy, Checkmarx, and LiteLLM Solutions
- Sonatype - Compromised litellm PyPI Package Delivers Multi-Stage Credential Stealer
