Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(backend): enhance rate limiting with dynamic model identifier #23

Merged
merged 3 commits into from
Mar 29, 2025

Conversation

nkanf-dev
Copy link
Contributor

This pull request includes several changes to improve the rate-limiting functionality and logging in the backend. The most important changes include updating the identifier parameter to be a function, adding logging capabilities, and ensuring proper initialization of token bucket values.

Rate-limiting enhancements:

  • backend/src/api/v1/completions.ts: Modified the identifier parameter in the rate limit configuration to be a function that extracts the model from the request body.
  • backend/src/plugins/rateLimitPlugin.ts: Updated the identifier parameter to be a function that can process the request body, added error handling for the identifier function, and included debug logging for rate limit checks.

Logging improvements:

Token bucket initialization:

  • backend/src/utils/tokenBucket.ts: Added logic to initialize token and last refill values in Redis if they do not exist, ensuring the token bucket is correctly set up.

@nkanf-dev
Copy link
Contributor Author

This pull request introduces a new rate-limiting feature to the backend API, along with several related enhancements. The changes include adding new rate limit configurations, updating existing API routes to utilize these configurations, and improving the rate limit plugin.

New rate-limiting feature:

Improvements to the rate limit plugin:

Utility functions for rate limit configurations:

@pescn pescn requested a review from koitococo March 26, 2025 12:13
@koitococo koitococo merged commit 2952124 into EM-GeekLab:main Mar 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants