Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chore/custom cloud auto branch #146

Merged
merged 23 commits into from
Jan 29, 2025
Merged

Chore/custom cloud auto branch #146

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
c130daa
Use custom test and cloud-auto branches in CI
haraprasadj Dec 13, 2024
c3d960a
more changes
haraprasadj Dec 13, 2024
ae34d04
use specific cloud-auto branch
haraprasadj Dec 13, 2024
b224ccc
Add CLOUD_AUTO_BRANCH to groovy scripts
krishnaa05 Dec 16, 2024
fe2d51e
update prepare_ci_environment.py
krishnaa05 Dec 16, 2024
e2d3209
Merge branch 'master' into chore/custom_cloud_auto_branch
krishnaa05 Jan 27, 2025
0c8c220
Update shared_integration_tests.yaml
krishnaa05 Jan 27, 2025
32c0113
Update branches in groovy script
krishnaa05 Jan 28, 2025
241ad4f
Update branches in groovy scripts
krishnaa05 Jan 28, 2025
b2229b0
Update env CLOUD_AUTO_BRANCH
krishnaa05 Jan 28, 2025
4c8cd61
Remove params keyword
krishnaa05 Jan 28, 2025
54d4738
Merge branch 'master' into chore/custom_cloud_auto_branch
krishnaa05 Jan 28, 2025
4a72aee
reverting changes
krishnaa05 Jan 28, 2025
bc8443a
Update CLOUD_AUTO_BRANCH variable in scripts
krishnaa05 Jan 28, 2025
fb4bac8
Update prepare_ci_environment.py
krishnaa05 Jan 28, 2025
e0b503e
Update shared_integration_tests.yaml
krishnaa05 Jan 28, 2025
30e18ec
updated gat
krishnaa05 Jan 29, 2025
b7d6a79
Updated shared integration tests yaml
krishnaa05 Jan 29, 2025
d4b819c
Update shared_integration_tests.yaml
krishnaa05 Jan 29, 2025
c05d857
Update shared_integration_tests.yaml
krishnaa05 Jan 29, 2025
018a1e2
Update shared_integration_tests.yaml
krishnaa05 Jan 29, 2025
67d0d5d
Final changes
krishnaa05 Jan 29, 2025
f11e864
update GAT
krishnaa05 Jan 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion .github/workflows/shared_integration_tests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,14 @@ on:
SERVICE_TO_TEST:
required: false
type: string
TEST_REPO_BRANCH:
required: false
type: string
default: master
CLOUD_AUTO_BRANCH:
required: false
type: string
default: master
secrets:
CI_AWS_ACCESS_KEY_ID:
required: true
Expand Down Expand Up @@ -110,7 +118,7 @@ jobs:
uses: actions/checkout@v4
with:
repository: uc-cdis/gen3-code-vigil
ref: master
ref: ${{ inputs.TEST_REPO_BRANCH }}

# gen3-integration-tests run with python 3.9
- name: Set up Python
Expand Down Expand Up @@ -156,6 +164,8 @@ jobs:
- name: Select CI environment
if: ${{ env.SKIP_TESTS != 'true' }}
id: select_ci_env
env:
CLOUD_AUTO_BRANCH: ${{ inputs.CLOUD_AUTO_BRANCH }}
run: |
env_label=$(gh api repos/$REPO_FN/pulls/$PR_NUM --jq '.labels | map(select(.name | startswith("jenkins-"))) | .[0].name')
echo "$env_label"
Expand Down
5 changes: 4 additions & 1 deletion gen3-integration-tests/gen3_ci/jenkins_jobs/ci-only-audit-service-logging.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@
e.g., true - Enabled the audit logging onto the target environment
false - Disables the audit logging onto the target environment

String parameter CLOUD_AUTO_BRANCH
e.g., master

Archived artifacts -
*/
pipeline {
Expand All @@ -25,7 +28,7 @@ pipeline {
// cloud-automation
checkout([
$class: 'GitSCM',
branches: [[name: 'refs/heads/master']],
branches: [[name: "*/${params.CLOUD_AUTO_BRANCH}"]],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should the default CLOUD_AUTO_BRANCH be refs/heads/master instead of master?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested the changes using uc-cdis/fence#1220. The CLOUD_AUTO_BRANCH is being passed and the correct branch is being downloaded in the jenkins job

doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'cloud-automation']],
submoduleCfg: [],
Expand Down
5 changes: 4 additions & 1 deletion gen3-integration-tests/gen3_ci/jenkins_jobs/ci-only-check-kube-job-pod.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@
default value - False
Note : This parameter is needed to be set to True if you want to test the negatve sceanrio
where you are expecting the job pod to fail.

String parameter CLOUD_AUTO_BRANCH
e.g., master
*/
pipeline {
agent {
Expand All @@ -27,7 +30,7 @@ pipeline {
// cloud-automation
checkout([
$class: 'GitSCM',
branches: [[name: 'refs/heads/master']],
branches: [[name: "*/${params.CLOUD_AUTO_BRANCH}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'cloud-automation']],
submoduleCfg: [],
Expand Down
5 changes: 4 additions & 1 deletion gen3-integration-tests/gen3_ci/jenkins_jobs/ci-only-check_indices_after_etl.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
/*
String parameter NAMESPACE
e.g. qa-dcp

String parameter CLOUD_AUTO_BRANCH
e.g., master
*/
pipeline {
agent {
Expand All @@ -19,7 +22,7 @@ pipeline {
// cloud-automation
checkout([
$class: 'GitSCM',
branches: [[name: 'refs/heads/master']],
branches: [[name: "*/${params.CLOUD_AUTO_BRANCH}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'cloud-automation']],
submoduleCfg: [],
Expand Down
5 changes: 4 additions & 1 deletion gen3-integration-tests/gen3_ci/jenkins_jobs/ci-only-clean-up-indices.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
/*
String parameter NAMESPACE
e.g. qa-dcp

String parameter CLOUD_AUTO_BRANCH
e.g., master
*/
pipeline {
agent {
Expand All @@ -19,7 +22,7 @@ pipeline {
// cloud-automation
checkout([
$class: 'GitSCM',
branches: [[name: 'refs/heads/master']],
branches: [[name: "*/${params.CLOUD_AUTO_BRANCH}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'cloud-automation']],
submoduleCfg: [],
Expand Down
5 changes: 4 additions & 1 deletion gen3-integration-tests/gen3_ci/jenkins_jobs/ci-only-create-access-token.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@
e.g., main@example.org
This is to help create access token

String parameter CLOUD_AUTO_BRANCH
e.g., master

*/
pipeline {
agent {
Expand All @@ -30,7 +33,7 @@ pipeline {
// cloud-automation
checkout([
$class: 'GitSCM',
branches: [[name: 'refs/heads/master']],
branches: [[name: "*/${params.CLOUD_AUTO_BRANCH}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'cloud-automation']],
submoduleCfg: [],
Expand Down
5 changes: 4 additions & 1 deletion gen3-integration-tests/gen3_ci/jenkins_jobs/ci-only-create-link-google-test-buckets.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@
String parameter JENKINS_NAMESPACE
Default value - default
Artifact archived - None

String parameter CLOUD_AUTO_BRANCH
e.g., master
*/
pipeline {
agent {
Expand Down Expand Up @@ -91,7 +94,7 @@ spec:
// cloud-automation
checkout([
$class: 'GitSCM',
branches: [[name: 'refs/heads/master']],
branches: [[name: "*/${params.CLOUD_AUTO_BRANCH}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'cloud-automation']],
submoduleCfg: [],
Expand Down
5 changes: 4 additions & 1 deletion gen3-integration-tests/gen3_ci/jenkins_jobs/ci-only-fence-delete-client.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
/*
String parameter NAMESPACE
e.g. jenkins-blood

String parameter CLOUD_AUTO_BRANCH
e.g., master
*/
pipeline {
agent {
Expand All @@ -19,7 +22,7 @@ pipeline {
// cloud-automation
checkout([
$class: 'GitSCM',
branches: [[name: 'refs/heads/master']],
branches: [[name: "*/${params.CLOUD_AUTO_BRANCH}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'cloud-automation']],
submoduleCfg: [],
Expand Down
5 changes: 4 additions & 1 deletion ...tegration-tests/gen3_ci/jenkins_jobs/ci-only-fence-disable-register-users-redirect.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
/*
String parameter NAMESPACE
e.g., qa-anvil

String parameter CLOUD_AUTO_BRANCH
e.g., master
*/
pipeline {
agent {
Expand All @@ -19,7 +22,7 @@ pipeline {
// cloud-automation
checkout([
$class: 'GitSCM',
branches: [[name: 'refs/heads/master']],
branches: [[name: "*/${params.CLOUD_AUTO_BRANCH}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'cloud-automation']],
submoduleCfg: [],
Expand Down
5 changes: 4 additions & 1 deletion ...ntegration-tests/gen3_ci/jenkins_jobs/ci-only-fence-enable-register-users-redirect.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
/*
String parameter NAMESPACE
e.g., qa-anvil

String parameter CLOUD_AUTO_BRANCH
e.g., master
*/
pipeline {
agent {
Expand All @@ -19,7 +22,7 @@ pipeline {
// cloud-automation
checkout([
$class: 'GitSCM',
branches: [[name: 'refs/heads/master']],
branches: [[name: "*/${params.CLOUD_AUTO_BRANCH}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'cloud-automation']],
submoduleCfg: [],
Expand Down
4 changes: 3 additions & 1 deletion gen3-integration-tests/gen3_ci/jenkins_jobs/ci-only-fence-force-link-google.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
e.g. dummy-one@example.org
String parameter EMAIL
e.g. main@example.org
String parameter CLOUD_AUTO_BRANCH
e.g., master

*/
pipeline {
Expand All @@ -24,7 +26,7 @@ pipeline {
// cloud-automation
checkout([
$class: 'GitSCM',
branches: [[name: 'refs/heads/master']],
branches: [[name: "*/${params.CLOUD_AUTO_BRANCH}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'cloud-automation']],
submoduleCfg: [],
Expand Down
4 changes: 3 additions & 1 deletion gen3-integration-tests/gen3_ci/jenkins_jobs/ci-only-fetch-configs.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
e.g., qa-anvil
String parameter JENKINS_NAMESPACE
Default value - default
String parameter CLOUD_AUTO_BRANCH
e.g., master

Archived artifacts - manifest.json
*/
Expand Down Expand Up @@ -92,7 +94,7 @@ spec:
// cloud-automation
checkout([
$class: 'GitSCM',
branches: [[name: 'refs/heads/master']],
branches: [[name: "*/${params.CLOUD_AUTO_BRANCH}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'cloud-automation']],
submoduleCfg: [],
Expand Down
6 changes: 4 additions & 2 deletions gen3-integration-tests/gen3_ci/jenkins_jobs/ci-only-generate-api-key.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@
Username
e.g., main@example.org
String parameter JENKINS_NAMESPACE
Default value - default
Default value - default
String parameter CLOUD_AUTO_BRANCH
e.g., master

Artifact archived - api_key.json
*/
Expand Down Expand Up @@ -95,7 +97,7 @@ spec:
// cloud-automation
checkout([
$class: 'GitSCM',
branches: [[name: 'refs/heads/master']],
branches: [[name: "*/${params.CLOUD_AUTO_BRANCH}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'cloud-automation']],
submoduleCfg: [],
Expand Down
6 changes: 4 additions & 2 deletions gen3-integration-tests/gen3_ci/jenkins_jobs/ci-only-generate-api-keys.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@
Username
e.g., main@example.org
String parameter JENKINS_NAMESPACE
Default value - default
Default value - default
String parameter CLOUD_AUTO_BRANCH
e.g., master

Artifact archived - api_key.json
*/
Expand Down Expand Up @@ -95,7 +97,7 @@ spec:
// cloud-automation
checkout([
$class: 'GitSCM',
branches: [[name: 'refs/heads/master']],
branches: [[name: "*/${params.CLOUD_AUTO_BRANCH}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'cloud-automation']],
submoduleCfg: [],
Expand Down
5 changes: 4 additions & 1 deletion gen3-integration-tests/gen3_ci/jenkins_jobs/ci-only-kube-setup-service.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@
String parameter SERVICENAME
e.g., indexd, fence, etc

String parameter CLOUD_AUTO_BRANCH
e.g., master

Archived artifacts -
*/
pipeline {
Expand All @@ -24,7 +27,7 @@ pipeline {
// cloud-automation
checkout([
$class: 'GitSCM',
branches: [[name: 'refs/heads/master']],
branches: [[name: "*/${params.CLOUD_AUTO_BRANCH}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'cloud-automation']],
submoduleCfg: [],
Expand Down
4 changes: 3 additions & 1 deletion gen3-integration-tests/gen3_ci/jenkins_jobs/ci-only-modify-env-for-service-pr.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@
e.g., 2023.09
String parameter JENKINS_NAMESPACE
Default value - default
String parameter CLOUD_AUTO_BRANCH
e.g., master
*/
pipeline {
agent {
Expand Down Expand Up @@ -102,7 +104,7 @@ spec:
// cloud-automation
checkout([
$class: 'GitSCM',
branches: [[name: 'refs/heads/master']],
branches: [[name: "*/${params.CLOUD_AUTO_BRANCH}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'cloud-automation']],
submoduleCfg: [],
Expand Down
4 changes: 3 additions & 1 deletion gen3-integration-tests/gen3_ci/jenkins_jobs/ci-only-modify-env-for-test-repo-pr.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
e.g., qa-anvil
String parameter JENKINS_NAMESPACE
Default value - default
String parameter CLOUD_AUTO_BRANCH
e.g., master
*/
pipeline {
agent {
Expand Down Expand Up @@ -96,7 +98,7 @@ spec:
// cloud-automation
checkout([
$class: 'GitSCM',
branches: [[name: 'refs/heads/master']],
branches: [[name: "*/${params.CLOUD_AUTO_BRANCH}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'cloud-automation']],
submoduleCfg: [],
Expand Down
4 changes: 3 additions & 1 deletion gen3-integration-tests/gen3_ci/jenkins_jobs/ci-only-mutate-manifest-for-guppy-test.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
/*
String parameter NAMESPACE
e.g., qa-anvil
String parameter CLOUD_AUTO_BRANCH
e.g., master

Archived artifacts -
*/
Expand All @@ -21,7 +23,7 @@ pipeline {
// cloud-automation
checkout([
$class: 'GitSCM',
branches: [[name: 'refs/heads/master']],
branches: [[name: "*/${params.CLOUD_AUTO_BRANCH}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'cloud-automation']],
submoduleCfg: [],
Expand Down
4 changes: 3 additions & 1 deletion gen3-integration-tests/gen3_ci/jenkins_jobs/ci-only-revoke-arborist-policy.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
e.g. main@example.org
String parameter POLICY
e.g.
String parameter CLOUD_AUTO_BRANCH
e.g., master
*/
pipeline {
agent {
Expand All @@ -21,7 +23,7 @@ pipeline {
// cloud-automation
checkout([
$class: 'GitSCM',
branches: [[name: 'refs/heads/master']],
branches: [[name: "*/${params.CLOUD_AUTO_BRANCH}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'cloud-automation']],
submoduleCfg: [],
Expand Down
4 changes: 3 additions & 1 deletion gen3-integration-tests/gen3_ci/jenkins_jobs/ci-only-run-gen3-command.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
e.g., gen3 secrets decode portal-config gitops.json | jq '.discoveryConfig.minimalFieldMapping.uid'
String parameter JENKINS_NAMESPACE
Default value - default
String parameter CLOUD_AUTO_BRANCH
e.g., master

Artifact archived - result.txt
*/
Expand Down Expand Up @@ -94,7 +96,7 @@ spec:
// cloud-automation
checkout([
$class: 'GitSCM',
branches: [[name: 'refs/heads/master']],
branches: [[name: "*/${params.CLOUD_AUTO_BRANCH}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'cloud-automation']],
submoduleCfg: [],
Expand Down
Loading
Loading