Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: optimize unit test #492

Merged
merged 6 commits into from
Mar 5, 2025
Merged

CI: optimize unit test #492

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
185c86d
Trigger unit test based on the changed code & extract the duplicate c…
Song-Quan Mar 3, 2025
5d36e23
change bazel to bazelisk
Song-Quan Mar 3, 2025
72e1d47
Split path-filtering/filter parameters
Song-Quan Mar 4, 2025
82ce4fe
fix typo
Song-Quan Mar 4, 2025
32dc15c
rename scdb_ut/engine_ut to go_ut/cpp_ut
Song-Quan Mar 4, 2025
3692238
add trigger file for run_cpp_ut
Song-Quan Mar 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 12 additions & 8 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,13 +54,17 @@ workflows:
base-revision: main
config-path: .circleci/unittest-config.yml
mapping: |
api/.* run_unittest true
bazel/.* run_unittest true
cmd/.* run_unittest true
engine/.* run_unittest true
pkg/.* run_unittest true
.bazelrc run_unittest true
.circleci/unittest-config.yml run_unittest true
api/.* run_go_ut true
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change from run_unittest to run_go_ut seems reasonable given the PR title 'CI: optimize unit test'. However, could you elaborate on why the other mappings (bazel, cmd, engine, pkg, .bazelrc, .circleci/unittest-config.yml) were removed? Are these tests now covered under the run_go_ut or are they being handled differently? Understanding the reasoning behind these removals will help ensure that the change doesn't inadvertently reduce test coverage.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • other mappings haven't been removed
  • run_unittest is now split into run_go_ut and run_cpp_ut

cmd/.* run_go_ut true
pkg/.* run_go_ut true
contrib/.* run_go_ut true
.circleci/unittest-config.yml run_go_ut true
bazel/.* run_cpp_ut true
engine/.* run_cpp_ut true
.bazelrc run_cpp_ut true
.bazeliskrc run_cpp_ut true
MODULE.bazel run_cpp_ut true
.circleci/unittest-config.yml run_cpp_ut true
run_regtest:
when: << pipeline.parameters.enable_regtest >>
jobs:
Expand All @@ -84,4 +88,4 @@ workflows:
# This is mandatory to trigger a pipeline when pushing a tag
filters:
tags:
only: /.*(?<!dev\d{8})$/
only: /.*(?<!dev\d{8})$/
131 changes: 70 additions & 61 deletions .circleci/unittest-config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,60 @@ parameters:
enable_regtest:
type: boolean
default: false
run_unittest:
run_go_ut:
type: boolean
default: false
run_cpp_ut:
type: boolean
default: false

commands:
setup_cpp_test:
steps:
- run:
name: "Checkout devtools"
command: git clone https://github.com/secretflow/devtools.git ../devtools
- run:
name: Setup GCS
command: |
echo ${gcs_content} > ../gcs.data
../devtools/bazel_cache_setup.py --in_file=../gcs.data --out_file=../gcs.json --min_download
run_go_test:
description: "Run go tests"
steps:
- run:
name: "Go Test"
command: |
set +e
go mod tidy
go test -timeout=30m -v -short ./pkg/... ./contrib/...
run_cpp_test:
description: "Run cpp tests and collect artifacts"
parameters:
extra_bazel_args:
type: string
default: ""
find_executable_flag:
type: string
default: "-executable"
steps:
- run:
name: "Cpp Test"
command: |
set +e
declare -i test_status
bazelisk --host_jvm_args=-Xmx8g test //engine/... << parameters.extra_bazel_args >> --jobs=32 --ui_event_filters=-info,-debug,-warning --test_output=errors | tee test_result.log; test_status=${PIPESTATUS[0]}

sh ../devtools/rename-junit-xml.sh
find bazel-bin/ << parameters.find_executable_flag >> -type f -name "*_test" -print0 | xargs -0 tar -cvzf test_binary.tar.gz
find bazel-testlogs/ -type f -name "test.log" -print0 | xargs -0 tar -cvzf test_logs.tar.gz
exit ${test_status}


# Define a job to be invoked later in a workflow.
# See: https://circleci.com/docs/2.0/configuration-reference/#jobs
jobs:
linux_scdb_ut:
linux_go_ut:
# Specify the execution environment. You can specify an image from Dockerhub or use one of our Convenience Images from CircleCI's Developer Hub.
# See: https://circleci.com/docs/2.0/configuration-reference/#docker-machine-macos-windows-executor
docker:
Expand All @@ -40,13 +86,8 @@ jobs:
# See: https://circleci.com/docs/2.0/configuration-reference/#steps
steps:
- checkout
- run:
name: "SCDB Test"
command: |
set +e
go mod tidy
go test -timeout=30m -v -short ./pkg/...
linux_engine_ut:
- run_go_test
linux_cpp_ut:
docker:
- image: secretflow/scql-ci:latest
parameters:
Expand All @@ -57,32 +98,17 @@ jobs:
# See: https://circleci.com/docs/2.0/configuration-reference/#steps
steps:
- checkout
- run:
name: "Checkout devtools"
command: git clone https://github.com/secretflow/devtools.git ../devtools
- run:
name: Setup GCS
command: |
echo ${gcs_content} > ../gcs.data
../devtools/bazel_cache_setup.py --in_file=../gcs.data --out_file=../gcs.json --min_download
- run:
name: "Engine Test"
command: |
set +e
declare -i test_status
bazelisk --host_jvm_args=-Xmx8g test //engine/... -c opt --jobs=32 --ui_event_filters=-info,-debug,-warning --test_output=errors | tee test_result.log; test_status=${PIPESTATUS[0]}

sh ../devtools/rename-junit-xml.sh
find bazel-bin/ -executable -type f -name "*_test" -print0 | xargs -0 tar -cvzf test_binary.tar.gz
find bazel-testlogs/ -type f -name "test.log" -print0 | xargs -0 tar -cvzf test_logs.tar.gz
exit ${test_status}
- setup_cpp_test
- run_cpp_test:
extra_bazel_args: "-c opt"
find_executable_flag: "-executable"
- store_test_results:
path: test-results
- store_artifacts:
path: test_binary.tar.gz
- store_artifacts:
path: test_logs.tar.gz
macOS_scdb_ut:
macOS_go_ut:
macos:
xcode: 16.0.0
resource_class: macos.m1.large.gen1
Expand All @@ -92,13 +118,8 @@ jobs:
name: "Install homebrew dependencies"
command: |
brew install wget go
- run:
name: "SCDB Test"
command: |
set +e
go mod tidy
go test -timeout=30m -v -short ./pkg/...
macOS_engine_ut:
- run_go_test
macOS_cpp_ut:
macos:
xcode: 16.0.0
resource_class: macos.m1.large.gen1
Expand All @@ -111,30 +132,15 @@ jobs:
echo "Canceling workflow as too much time has elapsed"
curl -X POST --header "Content-Type: application/json" "https://circleci.com/api/v2/workflow/${CIRCLE_WORKFLOW_ID}/cancel?circle-token=${BUILD_TIMER_TOKEN}"
- checkout
- run:
name: "Checkout devtools"
command: git clone https://github.com/secretflow/devtools.git ../devtools
- run:
name: "Install homebrew dependencies"
command: |
brew install bazelisk cmake ninja libomp wget go@1.24 md5sha1sum
brew link go@1.24
- run:
name: Setup GCS
command: |
echo ${gcs_content} > ../gcs.data
../devtools/bazel_cache_setup.py --in_file=../gcs.data --out_file=../gcs.json --min_download
- run:
name: "Engine Test"
command: |
set +e
declare -i test_status
bazelisk --host_jvm_args=-Xmx8g test //engine/... --jobs=32 --ui_event_filters=-info,-debug,-warning --test_output=errors| tee test_result.log; test_status=${PIPESTATUS[0]}

sh ../devtools/rename-junit-xml.sh
find bazel-bin/ -perm +111 -type f -name "*_test" -print0 | xargs -0 tar -cvzf test_binary.tar.gz
find bazel-testlogs/ -type f -name "test.log" -print0 | xargs -0 tar -cvzf test_logs.tar.gz
exit ${test_status}
- setup_cpp_test
- run_cpp_test:
extra_bazel_args: ""
find_executable_flag: "-perm +111"
- store_test_results:
path: test-results
- store_artifacts:
Expand All @@ -145,16 +151,19 @@ jobs:
# Invoke jobs via workflows
# See: https://circleci.com/docs/2.0/configuration-reference/#workflows
workflows:
unittest:
when: << pipeline.parameters.run_unittest >>
run_go_ut:
when: << pipeline.parameters.run_go_ut >>
jobs:
- linux_scdb_ut:
- linux_go_ut:
matrix:
parameters:
resource_class: ["2xlarge", "arm-xlarge"]
- linux_engine_ut:
- macOS_go_ut
run_cpp_ut:
when: << pipeline.parameters.run_cpp_ut >>
jobs:
- linux_cpp_ut:
matrix:
parameters:
resource_class: ["2xlarge", "arm-xlarge"]
- macOS_scdb_ut
- macOS_engine_ut
- macOS_cpp_ut