Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(root): chunk proof targets in state root task #14500

Open
wants to merge 53 commits into
base: main
Choose a base branch
from

Conversation

shekhirin
Copy link
Collaborator

@shekhirin shekhirin commented Feb 14, 2025

Towards #14417

Given proof targets with 100 accounts, it's preferred to spawn 20 multiproofs with 5 accounts each, instead of spawning one multiproof for all 100 accounts at once. The reason for that is that 20 multiproofs can be executed in parallel (even considering that they may do redundant work because of the intersecting trie nodes), while one multiproof will be executed serially for all accounts.

Same idea can be applied to storage slots: proof targets with 1 account and 100 storage slots is better spawned as 5 multiproofs for 20 storage slots each, rather than one multiproof for all 100 storage slots at once.

This PR adds chunking on the StateRootTask side, so that every set of proof targets is broken into multiple state updates using ChunkedProofTargets iterator, and spawned as a separate multiproof. This is applied to both on_prefetch_proof and on_state_update

@shekhirin shekhirin force-pushed the alexey/state-root-task-state-update-chunk branch from 1c04532 to a50c462 Compare February 14, 2025 17:26
@shekhirin shekhirin force-pushed the alexey/state-root-task-state-update-chunk branch from 8e2392b to c20669f Compare February 14, 2025 17:46
@emhane emhane added the A-engine Related to the engine implementation label Feb 16, 2025
@jenpaff jenpaff linked an issue Feb 18, 2025 that may be closed by this pull request
@jenpaff jenpaff linked an issue Feb 18, 2025 that may be closed by this pull request
@jenpaff
Copy link
Collaborator

jenpaff commented Feb 18, 2025

link #14417

@shekhirin shekhirin force-pushed the alexey/state-root-task-state-update-chunk branch from b045ca9 to ea68fa0 Compare February 19, 2025 16:44
@shekhirin shekhirin changed the title perf(root): chunk state updates in state root task perf(root): chunk proof targets in state root task Feb 24, 2025
@shekhirin shekhirin added the C-perf A change motivated by improving speed, memory usage or disk footprint label Feb 24, 2025
@shekhirin shekhirin force-pushed the alexey/state-root-task-state-update-chunk branch from 6cb9b94 to 45898be Compare February 24, 2025 13:34
@shekhirin shekhirin force-pushed the alexey/state-root-task-state-update-chunk branch from 944dada to de5ce5c Compare February 24, 2025 14:16
@shekhirin shekhirin marked this pull request as ready for review February 24, 2025 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-engine Related to the engine implementation C-perf A change motivated by improving speed, memory usage or disk footprint
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

newPayload latency spikes with pre-warming enabled
3 participants