Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhance(mysql): add better indexes for posts_gdocs table #4567

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

marcelgerber
Copy link
Member

@marcelgerber marcelgerber commented Feb 17, 2025

This PR does a few things to the posts_gdocs table:

  • Add a composite index on type, published, publishedAt
    • These sorts of queries are very common, e.g. WHERE type = "data-insight" AND published = 1 AND publishedAt <= NOW()
    • Also, it can be used on queries that just filter on type
  • Add another composite index on published, publishedAt
    • Again, filtering on published is very common
  • Convert the computed column type from VIRTUAL to STORED
  • Add a computed stored column on content ->> "$.authors"

I tested this a bunch and it works well overall, and the new indexes are used.
The only thing I'm not sure about is the addition of the authors column, which in theory could cause issues because we're not properly removing it someplace before inserting/updating into the DB. But if serializePostsGdocsRow is always used before sending things to the DB, then things will be fine on that end. (Also worth noting that we never had any issues with type, which has been a computed column for a long time now.)

@marcelgerber marcelgerber requested a review from mlbrgl February 17, 2025 13:17
@marcelgerber marcelgerber marked this pull request as ready for review February 17, 2025 13:17
@owidbot
Copy link
Contributor

owidbot commented Feb 17, 2025

Quick links (staging server):

Site Dev Site Preview Admin Wizard Docs

Login: ssh owid@staging-site-gdocs-indexes

SVG tester:

Number of differences (default views): 0 ✅
Number of differences (all views): 0 ✅

Edited: 2025-02-17 13:24:02 UTC
Execution time: 1.18 seconds

@marcelgerber marcelgerber removed the request for review from mlbrgl February 17, 2025 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants