Skip to content

Commit

Permalink
feat(integration tests): Flakiness tests on push to main (#3454)
Browse files Browse the repository at this point in the history
Adding repeated runs of integration tests for easier tracking of tests
flakiness. A single such check takes ~1h and should cost ~2$, which imo
is more than acceptable
  • Loading branch information
tomg10 authored Jan 23, 2025
1 parent 2b5fe98 commit add00a2
Show file tree
Hide file tree
Showing 2 changed files with 28 additions and 1 deletion.
27 changes: 27 additions & 0 deletions .github/workflows/ci-core-reusable.yml
Original file line number Diff line number Diff line change
Expand Up @@ -462,6 +462,15 @@ jobs:
run: |
ci_run ./bin/run_on_all_chains.sh "zkstack dev test integration --no-deps --ignore-prerequisites" ${{ env.CHAINS }} ${{ env.INTEGRATION_TESTS_LOGS_DIR }}
- name: Repeat integration tests on push to main to check for flakiness
if: ${{ (steps.condition.outputs.should_run == 'true' && github.event_name == 'push' && github.ref == 'refs/heads/main') }}
run: |
for i in {1..10}; do
echo "Iteration $i"
mkdir -p ${{ env.INTEGRATION_TESTS_LOGS_DIR }}/$i
ci_run ./bin/run_on_all_chains.sh "zkstack dev test integration --no-deps --ignore-prerequisites" ${{ env.CHAINS }} ${{ env.INTEGRATION_TESTS_LOGS_DIR }}/$i
done
- name: Init external nodes
run: |
GATEWAY_RPC_URL="${{ matrix.use_gateway_chain == 'WITH_GATEWAY' && '--gateway-rpc-url=http://localhost:3550' || '' }}"
Expand Down Expand Up @@ -506,6 +515,15 @@ jobs:
run: |
ci_run ./bin/run_on_all_chains.sh "zkstack dev test integration --no-deps --ignore-prerequisites --external-node" ${{ env.CHAINS }} ${{ env.INTEGRATION_TESTS_LOGS_DIR }}
- name: Repeat integration tests en on push to main to check for flakiness
if: ${{ (steps.condition.outputs.should_run == 'true' && github.event_name == 'push' && github.ref == 'refs/heads/main') }}
run: |
for i in {1..10}; do
echo "Iteration $i"
mkdir -p ${{ env.INTEGRATION_TESTS_LOGS_DIR }}/$i
ci_run ./bin/run_on_all_chains.sh "zkstack dev test integration --no-deps --ignore-prerequisites --external-node" ${{ env.CHAINS }} ${{ env.INTEGRATION_TESTS_LOGS_DIR }}/$i
done
- name: Fee projection tests
run: |
ci_run killall -INT zksync_server || true
Expand All @@ -519,6 +537,15 @@ jobs:
# Always run the chain-specific fee tests
ci_run ./bin/run_on_all_chains.sh "zkstack dev test fees --no-deps --no-kill" ${{ env.CHAINS }} ${{ env.FEES_LOGS_DIR }}
- name: Repeat fee projection tests on push to main to check for flakiness
if: ${{ (steps.condition.outputs.should_run == 'true' && github.event_name == 'push' && github.ref == 'refs/heads/main') }}
run: |
for i in {1..10}; do
echo "Iteration $i"
mkdir -p ${{ env.FEES_LOGS_DIR }}/$i
ci_run ./bin/run_on_all_chains.sh "zkstack dev test fees --no-deps --no-kill" ${{ env.CHAINS }} ${{ env.FEES_LOGS_DIR }}/$i
done
- name: Run revert tests
run: |
ci_run killall -INT zksync_server || true
Expand Down
2 changes: 1 addition & 1 deletion etc/env/file_based/general.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -359,7 +359,7 @@ pruning:
enabled: true
chunk_size: 10
removal_delay_sec: 60
data_retention_sec: 3600
data_retention_sec: 604800

commitment_generator:
max_parallelism: 10
Expand Down

0 comments on commit add00a2

Please sign in to comment.