From c86929aba3abbb8bf93895ef35a37096123de482 Mon Sep 17 00:00:00 2001
From: Ryan Rudder <96507400+RRudder@users.noreply.github.com>
Date: Tue, 14 Nov 2023 17:05:47 +1000
Subject: [PATCH] Create Prompt Injection template

* Added Prompt Injection template, recommendation, guidance
* LLM Output Handling, Training Data Poisoning, and Excessive Agency/Permission Manipulation are all to be added shortly
---
 .../llm_security/prompt_injection/guidance.md |  5 ++++
 .../prompt_injection/recommendations.md       | 13 ++++++++++
 .../llm_security/prompt_injection/template.md | 26 +++++++++++++++++++
 3 files changed, 44 insertions(+)
 create mode 100644 submissions/description/ai_application_security/llm_security/prompt_injection/guidance.md
 create mode 100644 submissions/description/ai_application_security/llm_security/prompt_injection/recommendations.md
 create mode 100644 submissions/description/ai_application_security/llm_security/prompt_injection/template.md

diff --git a/submissions/description/ai_application_security/llm_security/prompt_injection/guidance.md b/submissions/description/ai_application_security/llm_security/prompt_injection/guidance.md
new file mode 100644
index 00000000..eb0fcf81
--- /dev/null
+++ b/submissions/description/ai_application_security/llm_security/prompt_injection/guidance.md
@@ -0,0 +1,5 @@
+# Guidance
+
+Provide a step-by-step walkthrough with a screenshot on how you exploited the vulnerability. This will speed triage time and result in faster rewards. Please include specific details on where you identified the cryptographic weakness, how you identified it, and what actions you were able to perform as a result.
+
+Attempt to escalate the vulnerability to perform additional actions. If this is possible, provide a full Proof of Concept (PoC).
diff --git a/submissions/description/ai_application_security/llm_security/prompt_injection/recommendations.md b/submissions/description/ai_application_security/llm_security/prompt_injection/recommendations.md
new file mode 100644
index 00000000..99273ce9
--- /dev/null
+++ b/submissions/description/ai_application_security/llm_security/prompt_injection/recommendations.md
@@ -0,0 +1,13 @@
+# Recommendation(s)
+
+There is no single technique to prevent prompt injection from occurring. However, implementing the following defensive measures within the LLM application can prevent and limit the impact of prompt injection:
+
+- Use privilege controls for access to backend systems or when performing privileged operations. Apply the principle of least privilege to restrict the LLM's access to backend systems to that which is strictly necessary for its intended functionality. 
+- For privileged operations, require user interaction to approve any authorized action that would be performed on behalf of them.
+- Treat user input, external input, and the LLM as untrusted input sources.
+- Establish trust boundaries between external sources, the LLM, any plugins, and any neighboring systems.
+
+For more information, refer to the following resources:
+
+- <https://owasp.org/www-project-top-10-for-large-language-model-applications/>
+- <https://aivillage.org/large%20language%20models/threat-modeling-llm/>
diff --git a/submissions/description/ai_application_security/llm_security/prompt_injection/template.md b/submissions/description/ai_application_security/llm_security/prompt_injection/template.md
new file mode 100644
index 00000000..402f9322
--- /dev/null
+++ b/submissions/description/ai_application_security/llm_security/prompt_injection/template.md
@@ -0,0 +1,26 @@
+# Prompt Injection
+
+## Overview of the Vulnerability
+
+Prompt injection occurs when an attacker crafts a malicious prompt that manipulates a large language model (LLM) into executing unintended actions. The LLM has a lack of segregation between user input and the data within the LLM. This can allow an attacker to inject malicious prompts into an LLM which bypass safeguards and gain unauthorized access to data.
+
+## Business Impact
+
+This vulnerability can lead to reputational and financial damage of the company due an attacker gaining access to unauthorized data or compromising the decision-making of the LLM, which would also impact customers' trust. The severity of the impact to the business is dependent on the sensitivity of the accessible data being transmitted by the application.
+
+## Steps to Reproduce
+
+1. Navigate to the following URL:
+1. Inject the following prompt into the LLM:
+
+```prompt
+  {malicious prompt}
+```
+
+1. Observe that the LLM returns sensitive data
+
+## Proof of Concept (PoC)
+
+The screenshot(s) below demonstrate(s) the vulnerability:
+>
+> {{screenshot}}