Rate Limit Per Route Behaviour strange / misunderstood?

[GraemeMitchell84]

Title: Rate Limit Per Route Behaviour strange / misunderstood?

Description:
I've been trying to get Rate Limit Per Route filters working in envoy 1.30.1. I have it working, but the method by which it is working seems a little odd and a little by accident. Maybe somebody can clarify if this is a bug or by design.

I have a rate limit service deployed and I have a Rate Limit filter set to point to that rate limit service via a cluster. This is fine! I then tested this using rate_limits on the route explicitly:

Rate Limit Filter

http_filters:
  - name: "envoy.filters.http.ratelimit"
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
      domain: somedomain
      timeout: 1s
      failure_mode_deny: true
      rate_limit_service: 
        grpc_service: 
          envoy_grpc: 
            cluster_name: some-rate-limit-service
            authority: some-rate-limit-service.com
        transport_api_version: V3

Route Configuration

route: 
  cluster: some-cluster
  host_rewrite_literal: "somewhere.com"
  rate_limits:
  - actions:
    - generic_key:
      descriptor_value: some_value
      descriptor_key: some_key

This worked fine and applied rate limits using the domain set on the Rate Limit Filter. I then needed to override the domain on the route the rate limit was applied and found that there is no configuration for that on route.rate_Limits. I then looked at the RateLimitPerRoute filter and configured as so:

Rate Limit Per Route

route: 
  cluster: jsonplaceholder
  host_rewrite_literal: "jsonplaceholder.typicode.com"
typed_per_filter_config:
  "envoy.filters.http.ratelimit":
    "@type": type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimitPerRoute
    domain: overridewiththis
    rate_limits:
    - actions:
      - generic_key:
          descriptor_value: some_value
          descriptor_key: some_key                         

What I found is that requests were not fired to the rate limit service at all with the above route configuration using a RateLimitPerRoute filter. After some trial and error, the following is what I did to get it to work, which seems odd:

Working

route: 
  cluster: jsonplaceholder
  host_rewrite_literal: "jsonplaceholder.typicode.com"
  rate_limits:
  - actions:
    - generic_key:
        descriptor_value: some_value
        descriptor_key: some_key
typed_per_filter_config:
  "envoy.filters.http.ratelimit":
    "@type": type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimitPerRoute
    domain: overridewiththis
    # These rate limit descriptors don't seem to apply and instead are taken from route.rate_limits[].actions
    #rate_limits:
    #- actions:
    #  - generic_key:
    #      descriptor_value: b
    #      descriptor_key: g

What I found is that I need to defined route.rate_limits with actions for the descriptors for it to fire against the rate limit service. I then also need to apply the RateLimitPerRoute filter, but only for the purpose of overriding the domain used on the route.rate_limits. Any actions defined in the RateLimitPerRoute filter seem to be ignored and have no impact on the request to the rate limit service.

Is this intentional, am I doing something wrong / misinterpreting the config, or is this a bug?

Repro steps:

• Deploy a rate limit service (https://github.com/envoyproxy/ratelimit)
• Start Envoy with the below YAML (updating rate limit service details)
• Observe the rate limit service and see no requests being sent

Full Config

admin:
  address:
    socket_address: { address: 0.0.0.0, port_value: 9901 }

static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address: { address: 0.0.0.0, port_value: 10000 }
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          codec_type: AUTO
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              domains: ["*"]
              routes:
              - match: { prefix: "/" }
                route: 
                  cluster: jsonplaceholder
                  host_rewrite_literal: "jsonplaceholder.typicode.com"
                  #rate_limits:
                  #- actions:
                  #  - generic_key:
                  #      descriptor_value: a
                  #      descriptor_key: general
                typed_per_filter_config:
                  "envoy.filters.http.ratelimit":
                    "@type": type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimitPerRoute
                    domain: overridewiththis
                    rate_limits:
                    - actions:
                      - generic_key:
                          descriptor_value: b
                          descriptor_key: g                         
          http_filters:
          - name: "envoy.filters.http.ratelimit"
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
              domain: override-at-route
              timeout: 1s
              failure_mode_deny: true
              rate_limit_service: 
                grpc_service: 
                  envoy_grpc: 
                    cluster_name: some-rate-limit-service
                    authority: some-rate-limit-service.com
                transport_api_version: V3
          - name: "envoy.filters.http.header_mutation"
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.header_mutation.v3.HeaderMutation
          - name: envoy.filters.http.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

  clusters:
  - name: jsonplaceholder
    connect_timeout: 1s
    type: LOGICAL_DNS
    lb_policy: ROUND_ROBIN
    dns_lookup_family: V4_ONLY
    load_assignment:
      cluster_name: google
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: jsonplaceholder.typicode.com
                port_value: 80
  - connect_timeout: 1s
    http2_protocol_options: {}
    name: some-rate-limit-service
    type: LOGICAL_DNS
    load_assignment:
      cluster_name: some-rate-limit-service.com
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: some-rate-limit-service.com
                port_value: 9999
    

[adisuissa]

cc @esmet @mattklein123 as codeowners, and @wbpcode who may be able to assist.

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.

[mugioka]

envoy/api/envoy/extensions/filters/http/ratelimit/v3/rate_limit.proto

Line 394 in b87da3f

// [#not-implemented-hide:]

The "not-implemented-hide" option is enabled, so the rate limits configuration in RateLimitPerRoute is not applied.