Skip to content

Commit

Permalink
CI(build-neon): fix duplicated builds (#10731)
Browse files Browse the repository at this point in the history
## Problem

Parameterising `build-neon` job with `test-cfg` makes it to build
exactly the same thing several times.

See
-
https://github.com/neondatabase/neon/blob/874accd6ede7231e5e4e1f562a83862e2286f6cd/.github/workflows/_build-and-test-locally.yml#L51-L52
-
https://github.com/neondatabase/neon/actions/runs/13215068271/job/36893373038

## Summary of changes
- Extract `sanitizers` to a separate input from `test-cfg` and set it
separately
- Don't parametrise `build-neon` with `test-cfg`
  • Loading branch information
bayandin authored Feb 10, 2025
1 parent e711821 commit 2f36bdb
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 10 deletions.
21 changes: 12 additions & 9 deletions .github/workflows/_build-and-test-locally.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,14 @@ on:
required: true
type: string
test-cfg:
description: 'a json object of postgres versions and lfc/sanitizers states to build and run regression tests on'
description: 'a json object of postgres versions and lfc states to run regression tests on'
required: true
type: string
sanitizers:
description: 'enabled or disabled'
required: false
default: 'disabled'
type: string

defaults:
run:
Expand All @@ -48,8 +53,6 @@ jobs:
# io_uring will account the memory of the CQ and SQ as locked.
# More details: https://github.com/neondatabase/neon/issues/6373#issuecomment-1905814391
options: --init --shm-size=512mb --ulimit memlock=67108864:67108864
strategy:
matrix: ${{ fromJSON(format('{{"include":{0}}}', inputs.test-cfg)) }}
env:
BUILD_TYPE: ${{ inputs.build-type }}
GIT_VERSION: ${{ github.event.pull_request.head.sha || github.sha }}
Expand Down Expand Up @@ -89,7 +92,7 @@ jobs:
- name: Set env variables
env:
ARCH: ${{ inputs.arch }}
SANITIZERS: ${{ matrix.sanitizers }}
SANITIZERS: ${{ inputs.sanitizers }}
run: |
CARGO_FEATURES="--features testing"
if [[ $BUILD_TYPE == "debug" && $ARCH == 'x64' ]]; then
Expand Down Expand Up @@ -167,7 +170,7 @@ jobs:

- name: Run cargo build
env:
WITH_TESTS: ${{ matrix.sanitizers != 'enabled' && '--tests' || '' }}
WITH_TESTS: ${{ inputs.sanitizers != 'enabled' && '--tests' || '' }}
run: |
export ASAN_OPTIONS=detect_leaks=0
${cov_prefix} mold -run cargo build $CARGO_FLAGS $CARGO_FEATURES --bins ${WITH_TESTS}
Expand All @@ -177,7 +180,7 @@ jobs:
- name: Install rust binaries
env:
ARCH: ${{ inputs.arch }}
SANITIZERS: ${{ matrix.sanitizers }}
SANITIZERS: ${{ inputs.sanitizers }}
run: |
# Install target binaries
mkdir -p /tmp/neon/bin/
Expand Down Expand Up @@ -225,7 +228,7 @@ jobs:
role-duration-seconds: 18000 # 5 hours

- name: Run rust tests
if: ${{ matrix.sanitizers != 'enabled' }}
if: ${{ inputs.sanitizers != 'enabled' }}
env:
NEXTEST_RETRIES: 3
run: |
Expand Down Expand Up @@ -334,7 +337,7 @@ jobs:
- name: Pytest regression tests
continue-on-error: ${{ matrix.lfc_state == 'with-lfc' && inputs.build-type == 'debug' }}
uses: ./.github/actions/run-python-test-set
timeout-minutes: ${{ matrix.sanitizers != 'enabled' && 60 || 180 }}
timeout-minutes: ${{ inputs.sanitizers != 'enabled' && 60 || 180 }}
with:
build_type: ${{ inputs.build-type }}
test_selection: regress
Expand All @@ -352,7 +355,7 @@ jobs:
PAGESERVER_VIRTUAL_FILE_IO_ENGINE: tokio-epoll-uring
PAGESERVER_GET_VECTORED_CONCURRENT_IO: sidecar-task
USE_LFC: ${{ matrix.lfc_state == 'with-lfc' && 'true' || 'false' }}
SANITIZERS: ${{ matrix.sanitizers }}
SANITIZERS: ${{ inputs.sanitizers }}

# Temporary disable this step until we figure out why it's so flaky
# Ref https://github.com/neondatabase/neon/issues/4540
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/build_and_test_with_sanitizers.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,8 @@ jobs:
build-tools-image: ${{ needs.build-build-tools-image.outputs.image }}-bookworm
build-tag: ${{ needs.tag.outputs.build-tag }}
build-type: ${{ matrix.build-type }}
test-cfg: '[{"pg_version":"v17", "sanitizers": "enabled"}]'
test-cfg: '[{"pg_version":"v17"}]'
sanitizers: enabled
secrets: inherit


Expand Down

1 comment on commit 2f36bdb

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

7576 tests run: 7206 passed, 0 failed, 370 skipped (full report)


Flaky tests (1)

Postgres 17

Code coverage* (full report)

  • functions: 33.3% (8586 of 25822 functions)
  • lines: 49.1% (72291 of 147197 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
2f36bdb at 2025-02-10T14:51:07.338Z :recycle:

Please sign in to comment.