Skip to content

Commit

Permalink
Use rapids-pip-retry in CI jobs that might need retries (#17920)
Browse files Browse the repository at this point in the history
Uses a retry wrapper for `pip` commands to try to alleviate CI failures due to hash mismatches that result from network hiccups

xref rapidsai/build-planning#148

This will retry failures that show up in CI like:

```
   Collecting nvidia-cublas-cu12 (from libraft-cu12==25.2.*,>=0.0.0a0)
    Downloading https://pypi.nvidia.com/nvidia-cublas-cu12/nvidia_cublas_cu12-12.8.3.14-py3-none-manylinux_2_27_aarch64.whl (604.9 MB)
       ━━━━━━━━━━━━━━━━━━━━━                 350.2/604.9 MB 229.2 MB/s eta 0:00:02
  ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
      nvidia-cublas-cu12 from https://pypi.nvidia.com/nvidia-cublas-cu12/nvidia_cublas_cu12-12.8.3.14-py3-none-manylinux_2_27_aarch64.whl#sha256=93a4e0e386cc7f6e56c822531396de8170ed17068a1e18f987574895044cd8c3 (from libraft-cu12==25.2.*,>=0.0.0a0):
          Expected sha256 93a4e0e386cc7f6e56c822531396de8170ed17068a1e18f987574895044cd8c3
               Got        849c88d155cb4b4a3fdfebff9270fb367c58370b4243a2bdbcb1b9e7e940b7be
```

Authors:
  - Gil Forsyth (https://github.com/gforsyth)

Approvers:
  - Mike Sarahan (https://github.com/msarahan)
  - Bradley Dice (https://github.com/bdice)

URL: #17920
  • Loading branch information
gforsyth authored Feb 5, 2025
1 parent aa54f5f commit 3aeb4d8
Show file tree
Hide file tree
Showing 7 changed files with 10 additions and 10 deletions.
4 changes: 2 additions & 2 deletions ci/build_wheel.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
# Copyright (c) 2023-2024, NVIDIA CORPORATION.
# Copyright (c) 2023-2025, NVIDIA CORPORATION.

set -euo pipefail

Expand All @@ -16,7 +16,7 @@ cd "${package_dir}"
sccache --zero-stats

rapids-logger "Building '${package_name}' wheel"
python -m pip wheel \
rapids-pip-retry wheel \
-w dist \
-v \
--no-deps \
Expand Down
2 changes: 1 addition & 1 deletion ci/build_wheel_libcudf.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ rapids-dependency-file-generator \
| tee /tmp/requirements-build.txt

rapids-logger "Installing build requirements"
python -m pip install \
rapids-pip-retry install \
-v \
--prefer-binary \
-r /tmp/requirements-build.txt
Expand Down
4 changes: 2 additions & 2 deletions ci/configure_cpp_static.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
# Copyright (c) 2024, NVIDIA CORPORATION.
# Copyright (c) 2024-2025, NVIDIA CORPORATION.

set -euo pipefail

Expand All @@ -15,7 +15,7 @@ rapids-dependency-file-generator \
--file-key test_static_build \
--matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch)" | tee "${REQUIREMENTS_FILE}"

python -m pip install -r "${REQUIREMENTS_FILE}"
rapids-pip-retry install -r "${REQUIREMENTS_FILE}"
pyenv rehash

cmake -S cpp -B build_static -GNinja -DBUILD_SHARED_LIBS=OFF -DCUDF_USE_ARROW_STATIC=ON -DBUILD_TESTS=OFF
4 changes: 2 additions & 2 deletions ci/test_cudf_polars_polars_tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ RAPIDS_PY_WHEEL_NAME="libcudf_${RAPIDS_PY_CUDA_SUFFIX}" rapids-download-wheels-f
RAPIDS_PY_WHEEL_NAME="pylibcudf_${RAPIDS_PY_CUDA_SUFFIX}" rapids-download-wheels-from-s3 python ./local-pylibcudf-dep

rapids-logger "Install libcudf, pylibcudf and cudf_polars"
python -m pip install \
rapids-pip-retry install \
-v \
"$(echo ./dist/cudf_polars_"${RAPIDS_PY_CUDA_SUFFIX}"*.whl)[test]" \
"$(echo ./local-libcudf-dep/libcudf_"${RAPIDS_PY_CUDA_SUFFIX}"*.whl)" \
Expand All @@ -26,7 +26,7 @@ git clone https://github.com/pola-rs/polars.git --branch "${TAG}" --depth 1

# Install requirements for running polars tests
rapids-logger "Install polars test requirements"
python -m pip install -r polars/py-polars/requirements-dev.txt -r polars/py-polars/requirements-ci.txt
rapids-pip-retry install -r polars/py-polars/requirements-dev.txt -r polars/py-polars/requirements-ci.txt

# shellcheck disable=SC2317
function set_exitcode()
Expand Down
2 changes: 1 addition & 1 deletion ci/test_wheel_cudf.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ rapids-logger "Install cudf, pylibcudf, and test requirements"
rapids-generate-pip-constraints py_test_cudf ./constraints.txt

# echo to expand wildcard before adding `[extra]` requires for pip
python -m pip install \
rapids-pip-retry install \
-v \
--constraint ./constraints.txt \
"$(echo ./dist/cudf_"${RAPIDS_PY_CUDA_SUFFIX}"*.whl)[test]" \
Expand Down
2 changes: 1 addition & 1 deletion ci/test_wheel_cudf_polars.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ rapids-logger "Installing cudf_polars and its dependencies"
rapids-generate-pip-constraints py_test_cudf_polars ./constraints.txt

# echo to expand wildcard before adding `[test,experimental]` requires for pip
python -m pip install \
rapids-pip-retry install \
-v \
--constraint ./constraints.txt \
"$(echo ./dist/cudf_polars_"${RAPIDS_PY_CUDA_SUFFIX}"*.whl)[test,experimental]" \
Expand Down
2 changes: 1 addition & 1 deletion ci/test_wheel_dask_cudf.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ rapids-logger "Install dask_cudf, cudf, pylibcudf, and test requirements"
rapids-generate-pip-constraints py_test_dask_cudf ./constraints.txt

# echo to expand wildcard before adding `[extra]` requires for pip
python -m pip install \
rapids-pip-retry install \
-v \
--constraint ./constraints.txt \
"$(echo ./dist/cudf_"${RAPIDS_PY_CUDA_SUFFIX}"*.whl)" \
Expand Down

0 comments on commit 3aeb4d8

Please sign in to comment.