**_We Raised $20 Million_** _/_ [_Read the Announcement_](https://raven.io/blog/%20raven-raises-20m-seed-round-to-protect-applications-at-runtime-even-without-signatures-or-cves)

# Protect AI and LLM Models

AI models, particularly large language models (LLMs), rely heavily on external libraries to enhance their functionality, streamline development, and achieve cutting-edge performance. However, these libraries come with significant security supply chain risks, especially in open-source environments, where vulnerabilities and deliberate backdoors can be present.

One of the most pressing threats is the unintentional integration of a malicious package into an LLM framework.

In 2023, several vulnerabilities were discovered in popular AI libraries such as LangChain and Auto-GPT, exposing potential vectors for remote code execution, privilege escalation, and even arbitrary file manipulation. When an LLM uses such compromised libraries, attackers can exploit these security gaps to access sensitive data, hijack computational resources, or launch a supply chain attack, endangering every user of that model.

**Examples of Real Threats:**

- **LangChain - CVE-2023-29374:** Enabled remote code execution (RCE) through vulnerable functions in the llm_math chain, making it possible for attackers to execute arbitrary code within an LLM environment.
- **Auto-GPT - CVE-2023-37274:** Introduced a path-traversal vulnerability that allowed attackers to overwrite .py files outside the intended directory, paving the way for arbitrary code execution and manipulation.
- **TensorFlow - CVE-2023-25658:** Allowed for a denial-of-service (DoS) attack through a crafted input that could trigger a crash.
- **PyTorch - CVE-2023-29059:** Could lead to arbitrary code execution due to improper handling of certain inputs.
- **NumPy - CVE-2021-41495:** Allowed for arbitrary code execution when loading maliciously crafted files.
- **Pandas - CVE-2020-13091:** Could lead to information disclosure through improper handling of certain data structures.
- **Scikit-learn - CVE-2020-28975:** Allowed for code execution through deserialization of untrusted data.
- **Hugging Face - 2024-Malicious Model Uploads:** JFrog's security research team discovered that attackers had uploaded malicious machine learning models to the Hugging Face platform. These models contained silent backdoors, posing significant risks to data scientists and organizations utilizing them.

## See Beyond the Static CVSS Score

### Visibility

Achieving visibility into the runtime behavior of an LLM’s internals enables accurate and dynamic risk identification as it provides precise mapping of the libraries that are actively being utilized by the model, the data flows, and potential vulnerabilities.

### Prioritization

Not all vulnerabilities within an LLM model require immediate remediation. Distinguish between actively running libraries and those that are dormant by tracking CPU-level activities. This targeted approach allows teams to prioritize remediation efforts on the most critical vulnerabilities, saving valuable time and resources.

### Detection

Evaluate the permissions granted to each library and detect any deviations from expected behavior with runtime monitoring. This proactive approach enhances both threat detection and risk mitigation.