Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Push Database Finish Processing Tasks to context background process #51

Merged
merged 7 commits into from
Oct 28, 2024

Conversation

trey-ivy
Copy link
Collaborator

Currently, when you have a large build that takes severals seconds to save to the database, the process can fail to save if the client uses the --bes_upload_mode=fully_async flag

This flag causes the client to send a graceful shutdown upon completion of transmission w/out waiting for an acknowledgment from the server. If the database process isn't done saving before this shutdown is received, the context is destroyed and the save fails (see below)

image

This also causes issues when the server tries to acknowledge requests

image

To address this, we can push the database save to a background process and let it finish in its own time, and we can ignore send errors when the client gracefully terminated the connection. (i think...? its not like we were doing anything but logging them anyway so....).

  • improve database save functionality to be more robust and store as much of an invocation as it can
  • remove fk restraint on eventFile url, fix event file url formatting
  • push db save to a background process
  • remove caching for bazel invocations table view
  • update make lint

@trey-ivy trey-ivy requested a review from mortenmj October 25, 2024 16:57
@trey-ivy trey-ivy self-assigned this Oct 25, 2024
@mortenmj mortenmj self-requested a review October 28, 2024 13:22
Copy link
Collaborator

@mortenmj mortenmj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@trey-ivy trey-ivy merged commit 9bc68bb into main Oct 28, 2024
2 checks passed
@trey-ivy trey-ivy deleted the grpc-bg-ctx branch October 28, 2024 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants