Malicious ML Models Discovered on Hugging Face

Key Takeaways

• Two pre-trained models on Hugging Face contained embedded reverse-shell payloads executed via Python’s dangerous Pickle deserialization when loaded. (Help Net Security)
• The malicious payload leveraged PyTorch model files (which wrap Pickle serialization) and used a 7z compression trick to evade detection by Hugging Face’s Picklescan scanner. (The Hacker News)
• Attack technique dubbed “nullifAI” — a novel evasion of model-supply-chain defenses. (ReversingLabs)
• Highlights a systemic risk: deserialization vulnerabilities embedded in seemingly benign ML models can deliver malware to any user or system loading them. (Cybernews)

Summary

In February 2025, researchers from ReversingLabs uncovered two malicious ML models hosted on Hugging Face Hub. These models appeared legitimate but contained embedded Python payloads. Because the models were serialized in PyTorch format — which uses Python’s pickle under the hood — loading them triggered code execution on the host system. The malicious code opened reverse shells to hard-coded IP addresses, enabling remote access and potential full compromise. (Help Net Security)

The models had evaded detection by Hugging Face’s security scanner (Picklescan). Researchers attribute this to a clever evasion: the models were compressed with 7z instead of the default ZIP, preventing automatic loading and scanning using standard tools, and used “broken” pickle streams that triggered execution before flagged warnings. (The Hacker News)

This breach serves as a stark reminder that in the age of AI and open-source ML sharing, model provenance and serialization safety are critical — not optional.

What Happened: Incident Overview

1. Two malicious ML models were uploaded to the Hugging Face Hub. (ReversingLabs)
2. The models were serialized in PyTorch format, which wraps Pickle serialization — a format known to execute arbitrary code on deserialization. (Dark Reading)
3. To evade detection, the models used a non-standard 7z compression and deliberately corrupted pickle streams — bypassing Hugging Face’s Picklescan scanning tool. (The Hacker News)
4. When a developer or ML pipeline downloaded and loaded the model, the malicious code executed immediately, opening a reverse shell to an attacker-controlled server. (Help Net Security)
5. Hugging Face security team removed the malicious models after disclosure and updated Picklescan detection capabilities. (ReversingLabs)

Why It Matters

• Model-supply-chain risk: This incident proves that shared ML models — even those on trusted platforms — can carry malware. Without secure serialization or provenance checks, every model download is potentially a code-execution incident.
• Wider exposure: Many enterprises and developers rely on open-source ML models from Hugging Face. A single malicious model can affect numerous organizations, making this a systemic supply-chain vulnerability.
• Serialization insecurity: Use of unsafe formats like Python Pickle in ML workflows is common. This breach underlines why those practices are dangerous in real-world deployments.
• False sense of security: Reliance on automated scanning tools (like Picklescan) is insufficient — attackers can evade them with relatively trivial obfuscation.

The PointGuard AI Perspective

This incident validates what we’ve warned: AI supply chain must be treated with the same scrutiny as traditional software supply chain.

PointGuard AI helps mitigate these risks by:

• ML-Asset Discovery & Inventory: Automatically tracking all external models and datasets imported into projects, including provenance and serialization metadata.
• Serialization & Configuration Hardening: Flagging unsafe formats (e.g., Pickle), enforcing safer alternatives (e.g., SafeTensors), and preventing arbitrary code execution on load.
• Automated Model Validation & Sandboxing: Running loaded models in isolated environments to detect malicious behavior before production deployment.
• Behavior Monitoring & Runtime Defense: Detecting abnormal behavior post-load — reverse-shell attempts, unexpected network connections, or unauthorized file operations.
• Governance & Supply-Chain Risk Management: Maintaining an AI-SBOM (software bill of materials) for ML assets, helping security teams assess and control third-party model risk.

In short — as AI adoption grows, shared models ≠ safe models. Security must accompany collaboration, not be an afterthought.

Incident Scorecard Details

Total AISSI Score: 6.2 / 10

Criticality = 6, Possible impact not know but malicious models can execute arbitrary code on developer machines — full system compromise possible.

Propagation = 6, The models were publicly available on a major ML hub, widely accessible to developers worldwide but not shown to auto-propagate.

Exploitability = 7, Attack triggered simply by loading the model — no special privileges or exotic attack vector required.

Supply Chain = 8, Attack originated upstream in a shared community repository — classic supply-chain risk for AI/ML ecosystems.

Business Impact = 5, Compromises not yet reported but could lead to device takeover, theft of credentials or proprietary models/data, and exposure across organizations using shared ML models.

Sources

• Help Net Security — Malicious ML models found on Hugging Face Hub (Help Net Security)
• The Hacker News — Malicious ML Models on Hugging Face Leverage Broken Pickle Format to Evade Detection (The Hacker News)
• ReversingLabs — Malicious ML models discovered on Hugging Face platform (ReversingLabs)
• Cybernews — Malicious AI models infiltrating Hugging Face via ‘bad Pickles’ (Cybernews)