Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace localhost with private IP ranges fetcher #32

Merged
merged 1 commit into from
Jan 28, 2025
Merged

Replace localhost with private IP ranges fetcher #32

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ defender <responder> {
- `garbage`: Returns garbage data to pollute AI training.
- `custom`: Returns a custom message (requires `responder_args`).
- `ratelimit`: Marks requests for rate limiting (requires [Caddy-Ratelimit](https://github.com/mholt/caddy-ratelimit) to be installed as well ).
- `<ip_ranges...>`: A list of CIDR ranges or predefined range keys (e.g., `openai`, `localhost`) to match against the client's IP.
- `<ip_ranges...>`: A list of CIDR ranges or predefined range keys (e.g., `openai`, `private`) to match against the client's IP.
- `<custom message>`: A custom message to return when using the `custom` responder.
---

Expand All @@ -96,16 +96,16 @@ defender <responder> {

The plugin includes predefined IP ranges for popular AI services. These ranges are embedded in the binary and can be used without additional configuration.

| Service | Key | IP Ranges |
|-----------------------|---------------------------------------------|----------------------------------------------------|
| AWS | aws | [aws.go](ranges/fetchers/aws/aws.go) |
| AWS Region | aws-us-east-1, aws-us-west-1, aws-eu-west-1 | [aws_region.go](ranges/fetchers/aws/aws_region.go) |
| DeepSeek | deepseek | [deepseek.go](ranges/fetchers/deepseek.go) |
| GitHub Copilot | githubcopilot | [github.go](ranges/fetchers/github.go) |
| Google Cloud Platform | gcloud | [gcloud.go](ranges/fetchers/gcloud.go) |
| Microsoft Azure | azurepubliccloud | [azure.go](ranges/fetchers/azure.go) |
| OpenAI | openai | [openai.go](ranges/fetchers/openai.go) |
| Localhost (testing) | localhost | [localhost.go](ranges/fetchers/localhost.go) |
| Service | Key | IP Ranges |
|----------------------------------------------------------------------|---------------------------------------------|----------------------------------------------------|
| AWS | aws | [aws.go](ranges/fetchers/aws/aws.go) |
| AWS Region | aws-us-east-1, aws-us-west-1, aws-eu-west-1 | [aws_region.go](ranges/fetchers/aws/aws_region.go) |
| DeepSeek | deepseek | [deepseek.go](ranges/fetchers/deepseek.go) |
| GitHub Copilot | githubcopilot | [github.go](ranges/fetchers/github.go) |
| Google Cloud Platform | gcloud | [gcloud.go](ranges/fetchers/gcloud.go) |
| Microsoft Azure | azurepubliccloud | [azure.go](ranges/fetchers/azure.go) |
| OpenAI | openai | [openai.go](ranges/fetchers/openai.go) |
| [Private](https://caddyserver.com/docs/caddyfile/matchers#remote-ip) | private | [private.go](ranges/fetchers/private.go) |

More are welcome! for a precompiled list, see the [embedded results](ranges/data/generated.go)

Expand Down
2 changes: 1 addition & 1 deletion config_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,7 @@ func TestDefenderValidation(t *testing.T) {
{
"handler": "defender",
"ranges": [
"localhost"
"private"
],
"raw_responder": "pineapple"
},
Expand Down
2 changes: 1 addition & 1 deletion docs/examples.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ Integrate with [caddy-ratelimit](https://github.com/mholt/caddy-ratelimit):

:80 {
defender ratelimit {
ranges localhost
ranges private
}

rate_limit {
Expand Down
2 changes: 1 addition & 1 deletion examples/block/Caddyfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
bind 127.0.0.1 ::1

defender block {
ranges localhost
ranges private
}
respond "This is what a human sees"
}
Expand Down
2 changes: 1 addition & 1 deletion examples/custom/Caddyfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
bind 127.0.0.1 ::1

defender custom {
ranges localhost
ranges private
message "You are not welcome here"
}
respond "This is what a human sees"
Expand Down
2 changes: 1 addition & 1 deletion examples/garbage/Caddyfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
bind 127.0.0.1 ::1

defender garbage {
ranges localhost
ranges private
}
respond "This is what a human sees"
}
Expand Down
2 changes: 1 addition & 1 deletion examples/ratelimit/Caddyfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

:80 {
defender ratelimit {
ranges localhost
ranges private
}

rate_limit {
Expand Down
36 changes: 18 additions & 18 deletions ranges/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,14 @@ The **Fetchers Module** is a Go package designed to fetch IP ranges for various

## Supported Fetchers

| Fetcher Name | Description |
|------------------------|---------------------------------------------------------------------|
| `AWSFetcher` | Fetches global IP ranges for AWS services. |
| `AWSRegionFetcher` | Fetches IP ranges for a specific AWS region (e.g., `us-east-1`). |
| `GCloudFetcher` | Fetches IP ranges for Google Cloud Platform (GCP) services. |
| `OpenAIFetcher` | Fetches IP ranges for OpenAI services (e.g., ChatGPT, GPTBot). |
| `GithubCopilotFetcher` | Fetches IP ranges for GitHub Copilot services. |
| `LocalhostFetcher` | Fetches IP ranges for localhost (used for development and testing). |
| Fetcher Name | Description |
|------------------------|------------------------------------------------------------------|
| `AWSFetcher` | Fetches global IP ranges for AWS services. |
| `AWSRegionFetcher` | Fetches IP ranges for a specific AWS region (e.g., `us-east-1`). |
| `GCloudFetcher` | Fetches IP ranges for Google Cloud Platform (GCP) services. |
| `OpenAIFetcher` | Fetches IP ranges for OpenAI services (e.g., ChatGPT, GPTBot). |
| `GithubCopilotFetcher` | Fetches IP ranges for GitHub Copilot services. |
| `PrivateFetcher` | Fetches IP ranges for private network ranges (used for testing). |

---

Expand Down Expand Up @@ -63,16 +63,16 @@ func main() {

The `IPRanges` map in the `data` package contains the following keys:

| Key | Description |
|-------------------|-----------------------------------------------------------------------------|
| `aws` | Global IP ranges for AWS services. |
| `aws-us-east-1` | IP ranges for the AWS `us-east-1` region. |
| `aws-us-west-1` | IP ranges for the AWS `us-west-1` region. |
| `aws-eu-west-1` | IP ranges for the AWS `eu-west-1` region. |
| `gcloud` | IP ranges for Google Cloud Platform (GCP) services. |
| `openai` | IP ranges for OpenAI services (e.g., ChatGPT, GPTBot). |
| `githubcopilot` | IP ranges for GitHub Copilot services. |
| `localhost` | IP ranges for localhost (used for development and testing). |
| Key | Description |
|-----------------|--------------------------------------------------------|
| `aws` | Global IP ranges for AWS services. |
| `aws-us-east-1` | IP ranges for the AWS `us-east-1` region. |
| `aws-us-west-1` | IP ranges for the AWS `us-west-1` region. |
| `aws-eu-west-1` | IP ranges for the AWS `eu-west-1` region. |
| `gcloud` | IP ranges for Google Cloud Platform (GCP) services. |
| `openai` | IP ranges for OpenAI services (e.g., ChatGPT, GPTBot). |
| `githubcopilot` | IP ranges for GitHub Copilot services. |
| `private` | IP ranges for private networks (used for testing). |

### Regenerating Pregenerated Results

Expand Down
12 changes: 8 additions & 4 deletions ranges/data/generated.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 0 additions & 18 deletions ranges/fetchers/localhost.go
View file
Open in desktop

This file was deleted.

22 changes: 22 additions & 0 deletions ranges/fetchers/private.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
package fetchers

// PrivateFetcher implements the IPRangeFetcher interface for private network ranges.
type PrivateFetcher struct{}

func (f PrivateFetcher) Name() string {
return "Private"
}
func (f PrivateFetcher) Description() string {
return "Hardcoded IP ranges for private network ranges. Used in testing."
}
func (f PrivateFetcher) FetchIPRanges() ([]string, error) {

return []string{
"127.0.0.0/8",
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice if we could remove the overlapping IPS with localhost fetcher

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't sure about that, but I'll change it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was trying to keep it close to Caddy, by the way: https://caddyserver.com/docs/caddyfile/matchers#remote-ip

"::1/128",
"10.0.0.0/8",
"172.16.0.0/12",
"192.168.0.0/16",
"fd00::/8",
}, nil
}
2 changes: 1 addition & 1 deletion ranges/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ func main() {
aws.AWSRegionFetcher{Region: "us-east-1"}, // us-east-1 region
aws.AWSRegionFetcher{Region: "us-west-1"}, // us-west-1 region
aws.AWSRegionFetcher{Region: "eu-west-1"}, // eu-west-1 region
fetchers.LocalhostFetcher{},
fetchers.PrivateFetcher{},
}

// Load the existing IP ranges from the data package
Expand Down
Loading