Skip to content

Commit

Permalink
Create Prompt Injection template
Browse files Browse the repository at this point in the history
* Added Prompt Injection template, recommendation, guidance
* LLM Output Handling, Training Data Poisoning, and Excessive Agency/Permission Manipulation are all to be added shortly
  • Loading branch information
RRudder committed Nov 14, 2023
1 parent 013c8da commit c86929a
Show file tree
Hide file tree
Showing 3 changed files with 44 additions and 0 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Guidance

Provide a step-by-step walkthrough with a screenshot on how you exploited the vulnerability. This will speed triage time and result in faster rewards. Please include specific details on where you identified the cryptographic weakness, how you identified it, and what actions you were able to perform as a result.

Attempt to escalate the vulnerability to perform additional actions. If this is possible, provide a full Proof of Concept (PoC).
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Recommendation(s)

There is no single technique to prevent prompt injection from occurring. However, implementing the following defensive measures within the LLM application can prevent and limit the impact of prompt injection:

- Use privilege controls for access to backend systems or when performing privileged operations. Apply the principle of least privilege to restrict the LLM's access to backend systems to that which is strictly necessary for its intended functionality.
- For privileged operations, require user interaction to approve any authorized action that would be performed on behalf of them.
- Treat user input, external input, and the LLM as untrusted input sources.
- Establish trust boundaries between external sources, the LLM, any plugins, and any neighboring systems.

For more information, refer to the following resources:

- <https://owasp.org/www-project-top-10-for-large-language-model-applications/>
- <https://aivillage.org/large%20language%20models/threat-modeling-llm/>
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Prompt Injection

## Overview of the Vulnerability

Prompt injection occurs when an attacker crafts a malicious prompt that manipulates a large language model (LLM) into executing unintended actions. The LLM has a lack of segregation between user input and the data within the LLM. This can allow an attacker to inject malicious prompts into an LLM which bypass safeguards and gain unauthorized access to data.

## Business Impact

This vulnerability can lead to reputational and financial damage of the company due an attacker gaining access to unauthorized data or compromising the decision-making of the LLM, which would also impact customers' trust. The severity of the impact to the business is dependent on the sensitivity of the accessible data being transmitted by the application.

## Steps to Reproduce

1. Navigate to the following URL:
1. Inject the following prompt into the LLM:

```prompt
{malicious prompt}
```

1. Observe that the LLM returns sensitive data

## Proof of Concept (PoC)

The screenshot(s) below demonstrate(s) the vulnerability:
>
> {{screenshot}}

0 comments on commit c86929a

Please sign in to comment.