Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fhevm-listener: batch/parallel INSERT of events to table computations #337

Open
rudy-6-4 opened this issue Feb 21, 2025 · 0 comments
Open
Labels
enhancement New feature or request

Comments

@rudy-6-4
Copy link
Contributor

Goal:

  • reduce the network load on the listener/database.
  • reduce the latency of events to db (by removing wait time)
  • increase the efficiency/capacity of the listener by better CPU use

Currently we insert events by events sequentially and wait between insertions.

Steps:

  1. Enable parallelism between IO & events preparation: while a insert in DB is in progress, we can deserialize and prepare the nexts events
  • as long as the ongoing insert is not finished we prepare the events (this can be handle by the insert function)
  1. Enable batch: next time we contact the DB for insertion , we can batch the ready prepared events for a batch INSERT

  2. Enable multiple insert: same but we accept a bounded number (e.g. 2 or 3) of multiple INSERT are ongoing. e.g. if a number of minimum events are ready and an insertion slot is ready than we insert

@rudy-6-4 rudy-6-4 added the enhancement New feature or request label Feb 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant