-
Notifications
You must be signed in to change notification settings - Fork 49
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Added Excessive Agency or Permission Manipulation
* Template, recommendation, and guidance .md files
- Loading branch information
Showing
3 changed files
with
46 additions
and
0 deletions.
There are no files selected for viewing
5 changes: 5 additions & 0 deletions
5
...tion_security/llm_security/excessive_agency_permission_manipulation/guidance.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
# Guidance | ||
|
||
Provide a step-by-step walkthrough with a screenshot on how you exploited the vulnerability. This will speed triage time and result in faster rewards. Please include specific details on where you identified the vulnerability, how you identified it, and what actions you were able to perform as a result. | ||
|
||
Attempt to escalate the vulnerability to perform additional actions. If this is possible, provide a full Proof of Concept (PoC). |
15 changes: 15 additions & 0 deletions
15
...curity/llm_security/excessive_agency_permission_manipulation/recommendations.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
# Recommendation(s) | ||
|
||
There is no single technique to prevent excessive agency or permission manipulation from occurring. However, implementing the following defensive measures within the LLM application can prevent and limit the impact of the vulnerability: | ||
|
||
- Use privilege controls for access to backend systems or when performing privileged operations. Apply the principle of least privilege to restrict the LLM's access to backend systems to that which is strictly necessary for its intended functionality. | ||
- For privileged operations, require user interaction to approve any authorized action that would be performed on behalf of them. | ||
- Treat user input, external input, and the LLM as untrusted input sources. | ||
- Establish trust boundaries between external sources, the LLM, any plugins, and any neighboring systems. | ||
- Limit the tools, plugins, and functions that the LLM can access to the minimum necessary for intended functionality. | ||
- Log and monitor all activity of the LLM and the systems it is connected to. | ||
|
||
For more information, refer to the following resources: | ||
|
||
- <https://owasp.org/www-project-top-10-for-large-language-model-applications/> | ||
- <https://embracethered.com/blog/posts/2023/chatgpt-cross-plugin-request-forgery-and-prompt-injection./> |
26 changes: 26 additions & 0 deletions
26
...tion_security/llm_security/excessive_agency_permission_manipulation/template.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
# Excessive Agency/Permission Manipulation | ||
|
||
## Overview of the Vulnerability | ||
|
||
Excessive agency or permission manipulation occurs when an attacker is able to manipulate the LLM outputs to perform actions that are damaging or otherwise harmful. This usually stems from excessive functionality, permissions, or autonomy. An attacker can abuse excessive agency or permission manipulation within the LLM to gain access to, modify, or delete data, without any confirmation from a user. | ||
|
||
## Business Impact | ||
|
||
This vulnerability can lead to reputational and financial damage of the company due an attacker gaining access to unauthorized data or compromising the decision-making of the LLM, which would also impact customers' trust. The severity of the impact to the business is dependent on the sensitivity of the accessible data being transmitted by the application. | ||
|
||
## Steps to Reproduce | ||
|
||
1. Navigate to the following URL: | ||
1. Enter the following prompt into the LLM: | ||
|
||
```prompt | ||
{prompt} | ||
``` | ||
|
||
1. Observe that the output from the LLM returns sensitive data | ||
|
||
## Proof of Concept (PoC) | ||
|
||
The screenshot(s) below demonstrate(s) the vulnerability: | ||
> | ||
> {{screenshot}} |