Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decouple consolidate and vacuum operations #539

Merged

Conversation

NikolaosPapailiou
Copy link
Collaborator

Vacuum operations are not process safe and therefore we should avoid coupling them with consolidation operations in order to maintain thread/process safety for vector search updates

Copy link
Contributor

@jparismorgan jparismorgan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, though we'll also need to do this for VAMANA and IVF_PQ at some point. Also curious if we could check that it worked correctly in a unit test, though if that's hard to do I'm okay to skip it.

@NikolaosPapailiou
Copy link
Collaborator Author

Looks good, though we'll also need to do this for VAMANA and IVF_PQ at some point. Also curious if we could check that it worked correctly in a unit test, though if that's hard to do I'm okay to skip it.

Sure we need to apply the same logic also in C++ for the new algorithms.
I added code to exercise this in the unit tests.

@NikolaosPapailiou NikolaosPapailiou merged commit 01da27b into main Oct 3, 2024
5 of 6 checks passed
@NikolaosPapailiou NikolaosPapailiou deleted the npapa/sc-53861/decouple-consolidation-and-vacuum branch October 3, 2024 10:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants