Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(http): Switch to memory cache provider #33901

Merged

Conversation

zharinov
Copy link
Collaborator

@zharinov zharinov commented Jan 28, 2025

Changes

Make caching of GET/HEAD HTTP requests disabled by default.
When it's needed, the memCacheProvider is be used.

Context

Documentation (please check one with an [x])

  • I have updated the documentation, or
  • No documentation update is required

How I've tested my work (please select one)

I have verified these changes via:

  • Code inspection only, or
  • Newly added/modified unit tests, or
  • No unit tests but ran on a real repository, or
  • Both unit tests + ran on a real repository

@zharinov
Copy link
Collaborator Author

@zharinov zharinov marked this pull request as draft January 28, 2025 19:04
@zharinov zharinov marked this pull request as ready for review February 10, 2025 16:41
@zharinov zharinov requested review from rarkins and viceice February 10, 2025 16:45
Copy link
Collaborator

@rarkins rarkins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will test first

Copy link
Collaborator

@rarkins rarkins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you summarize all the cases where http memcache is still being explicitly enabled in this PR (seems to be within platform code) and we evaluate whether we're better to add platform-level caching (e.g. of PR lists) instead of relying on http-level? Maybe you can do this through inline PR comments

Copy link
Collaborator

@rarkins rarkins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should have a significant reduction in the number of http requests which we cache in memory. This should apply especially to datasource requests

@zharinov zharinov requested a review from rarkins March 11, 2025 11:52
Copy link
Collaborator

@rarkins rarkins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When running again on renovate-reproductions/renovate2, I still get this:

DEBUG: Cache fallback URLs (repository=renovate-reproductions/renovate2)
       "count": 4,
       "hits": {
         "https://api.github.com/repos/yarnpkg/berry": {
           "callsite": "async Object.getReleaseNotesMd (/home/rhys/src/github/renovatebot/renovate/lib/workers/repository/update/pr/changelog/github/index.ts)",
           "count": 1
         },
         "https://api.github.com/repos/yarnpkg/berry/git/blobs/3e96b54ffe13ba43b83ee1844a3ba37aeb4f77f8": {
           "callsite": "async Object.getReleaseNotesMd (/home/rhys/src/github/renovatebot/renovate/lib/workers/repository/update/pr/changelog/github/index.ts)",
           "count": 1
         },
         "https://api.github.com/repos/renovate-reproductions/renovate2/commits/renovate/main-yarn-monorepo/check-runs?per_page=100": {
           "callsite": "async Proxy.getBranchStatus (/home/rhys/src/github/renovatebot/renovate/lib/modules/platform/github/index.ts)",
           "count": 1
         },
         "https://api.github.com/repos/renovate-reproductions/renovate2/commits/renovate/main-mkdocs-material-9.x/check-runs?per_page=100": {
           "callsite": "async Proxy.getBranchStatus (/home/rhys/src/github/renovatebot/renovate/lib/modules/platform/github/index.ts)",
           "count": 1
         }
       }

@zharinov zharinov marked this pull request as draft March 12, 2025 13:17
@zharinov zharinov marked this pull request as ready for review March 12, 2025 13:17
@zharinov zharinov requested a review from rarkins March 12, 2025 13:26
@rarkins rarkins requested a review from secustor March 14, 2025 06:04
rarkins
rarkins previously approved these changes Mar 14, 2025
secustor
secustor previously approved these changes Mar 23, 2025
@secustor
Copy link
Collaborator

LGTM, though needs a conflict resolution

@zharinov zharinov dismissed stale reviews from rarkins and secustor via 34ed14d March 23, 2025 19:01
@rarkins rarkins enabled auto-merge March 24, 2025 07:58
@rarkins rarkins added this pull request to the merge queue Mar 24, 2025
@rarkins rarkins removed this pull request from the merge queue due to a manual request Mar 24, 2025
@rarkins rarkins enabled auto-merge March 24, 2025 07:59
@rarkins rarkins added this pull request to the merge queue Mar 24, 2025
Merged via the queue into renovatebot:main with commit f977700 Mar 24, 2025
40 checks passed
@rarkins rarkins deleted the feat/http-memory-cache-provider-switch branch March 24, 2025 08:09
@renovate-release
Copy link
Collaborator

🎉 This PR is included in version 39.213.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants