Skip to content

Commit

Permalink
Use a cache mount to speed up rebuilding compute node image (#10737)
Browse files Browse the repository at this point in the history
Building the compute rust binaries from scratch is pretty slow, it takes
between 4-15 minutes on my laptop, depending on which compiler flags and
other tricks I use. A cache mount allows caching the dependencies and
incremental builds, which speeds up rebuilding significantly when you
only makes a small change in a source file.
  • Loading branch information
hlinnaka authored Feb 10, 2025
1 parent aba61a3 commit c368b0f
Showing 1 changed file with 12 additions and 4 deletions.
16 changes: 12 additions & 4 deletions compute/compute-node.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -1578,7 +1578,15 @@ ENV BUILD_TAG=$BUILD_TAG
USER nonroot
# Copy entire project to get Cargo.* files with proper dependencies for the whole project
COPY --chown=nonroot . .
RUN mold -run cargo build --locked --profile release-line-debug-size-lto --bin compute_ctl --bin fast_import --bin local_proxy
RUN --mount=type=cache,uid=1000,target=/home/nonroot/.cargo/registry \
--mount=type=cache,uid=1000,target=/home/nonroot/.cargo/git \
--mount=type=cache,uid=1000,target=/home/nonroot/target \
mold -run cargo build --locked --profile release-line-debug-size-lto --bin compute_ctl --bin fast_import --bin local_proxy && \
mkdir target-bin && \
cp target/release-line-debug-size-lto/compute_ctl \
target/release-line-debug-size-lto/fast_import \
target/release-line-debug-size-lto/local_proxy \
target-bin

#########################################################################################
#
Expand Down Expand Up @@ -1781,15 +1789,15 @@ RUN mkdir /var/db && useradd -m -d /var/db/postgres postgres && \
COPY --from=awscli /usr/local/aws-cli /usr/local/aws-cli

COPY --from=postgres-cleanup-layer --chown=postgres /usr/local/pgsql /usr/local
COPY --from=compute-tools --chown=postgres /home/nonroot/target/release-line-debug-size-lto/compute_ctl /usr/local/bin/compute_ctl
COPY --from=compute-tools --chown=postgres /home/nonroot/target/release-line-debug-size-lto/fast_import /usr/local/bin/fast_import
COPY --from=compute-tools --chown=postgres /home/nonroot/target-bin/compute_ctl /usr/local/bin/compute_ctl
COPY --from=compute-tools --chown=postgres /home/nonroot/target-bin/fast_import /usr/local/bin/fast_import

# pgbouncer and its config
COPY --from=pgbouncer /usr/local/pgbouncer/bin/pgbouncer /usr/local/bin/pgbouncer
COPY --chmod=0666 --chown=postgres compute/etc/pgbouncer.ini /etc/pgbouncer.ini

# local_proxy and its config
COPY --from=compute-tools --chown=postgres /home/nonroot/target/release-line-debug-size-lto/local_proxy /usr/local/bin/local_proxy
COPY --from=compute-tools --chown=postgres /home/nonroot/target-bin/local_proxy /usr/local/bin/local_proxy
RUN mkdir -p /etc/local_proxy && chown postgres:postgres /etc/local_proxy

# Metrics exporter binaries and configuration files
Expand Down

1 comment on commit c368b0f

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

7425 tests run: 7000 passed, 44 failed, 381 skipped (full report)


Failures on Postgres 17

Failures on Postgres 16

Failures on Postgres 15

Failures on Postgres 14

# Run all failed tests locally:
scripts/pytest -vv -n $(nproc) -k "test_forward_compatibility[release-pg14] or test_forward_compatibility[release-pg14] or test_versions_mismatch[release-pg14-combination_ooonn] or test_versions_mismatch[release-pg14-combination_ooonn] or test_versions_mismatch[release-pg14-combination_ononn] or test_versions_mismatch[release-pg14-combination_ononn] or test_versions_mismatch[release-pg14-combination_onnnn] or test_versions_mismatch[release-pg14-combination_onnnn] or test_forward_compatibility[release-pg15] or test_forward_compatibility[release-pg15] or test_versions_mismatch[release-pg15-combination_ooonn] or test_versions_mismatch[release-pg15-combination_ooonn] or test_versions_mismatch[release-pg15-combination_ononn] or test_versions_mismatch[release-pg15-combination_ononn] or test_versions_mismatch[release-pg15-combination_onnnn] or test_versions_mismatch[release-pg15-combination_onnnn] or test_forward_compatibility[release-pg16] or test_forward_compatibility[release-pg16] or test_versions_mismatch[release-pg16-combination_ooonn] or test_versions_mismatch[release-pg16-combination_ooonn] or test_versions_mismatch[release-pg16-combination_ononn] or test_versions_mismatch[release-pg16-combination_ononn] or test_versions_mismatch[release-pg16-combination_onnnn] or test_versions_mismatch[release-pg16-combination_onnnn] or test_forward_compatibility[release-pg17] or test_forward_compatibility[release-pg17] or test_forward_compatibility[debug-pg17] or test_forward_compatibility[release-pg17] or test_forward_compatibility[release-pg17] or test_versions_mismatch[release-pg17-combination_ooonn] or test_versions_mismatch[release-pg17-combination_ooonn] or test_versions_mismatch[debug-pg17-combination_ooonn] or test_versions_mismatch[release-pg17-combination_ooonn] or test_versions_mismatch[release-pg17-combination_ooonn] or test_versions_mismatch[release-pg17-combination_ononn] or test_versions_mismatch[release-pg17-combination_ononn] or test_versions_mismatch[debug-pg17-combination_ononn] or test_versions_mismatch[release-pg17-combination_ononn] or test_versions_mismatch[release-pg17-combination_ononn] or test_versions_mismatch[release-pg17-combination_onnnn] or test_versions_mismatch[release-pg17-combination_onnnn] or test_versions_mismatch[debug-pg17-combination_onnnn] or test_versions_mismatch[release-pg17-combination_onnnn] or test_versions_mismatch[release-pg17-combination_onnnn]"
Flaky tests (1)

Postgres 17

Test coverage report is not available

The comment gets automatically updated with the latest test results
c368b0f at 2025-02-10T18:24:55.759Z :recycle:

Please sign in to comment.