Upgrade dependencies 2025-02-03 (#6874)

[achave11-ucsc]

Connected issue: #6874

Checklist

Author

• Target branch is develop
• Name of PR branch matches upgrades/yyyy-mm-dd
• On ZenHub, PR is connected to the upgrade issue it resolves
• PR title matches Upgrade dependencies yyyy-mm-dd
• PR title references the connected issue

Author (upgrading deployments)

• Ran make docker_images.json and committed the resulting changes _{or this PR does not modify azul_docker_images, or any other variables referenced in the definition of that variable}
• Documented upgrading of deployments in UPGRADING.rst _{or this PR does not require upgrading deployments}
• Added u tag to commit title _{or this PR does not require upgrading deployments}
• This PR is labeled upgrade _{or does not require upgrading deployments}
• This PR is labeled deploy:shared _{or does not modify docker_images.json, and does not require deploying the shared component for any other reason}
• This PR is labeled deploy:gitlab _{or does not require deploying the gitlab component}
• This PR is labeled backup:gitlab
• This PR is labeled deploy:runner _{or does not require deploying the runner image}

Author (before every review)

• Rebased PR branch on develop, squashed old fixups
• Ran make requirements_update _{or this PR does not modify requirements*.txt, common.mk, Makefile and Dockerfile}
• Added R tag to commit title _{or this PR does not modify requirements*.txt}
• This PR is labeled reqs _{or does not modify requirements*.txt}
• make integration_test passes in personal deployment _{or this PR does not modify functionality that could affect the IT outcome}

System administrator (after approval)

• Actually approved the PR
• Labeled connected issue as no demo
• A comment to this PR details the completed security design review
• PR title is appropriate as title of merge commit
• Moved connected issue to Approved column
• PR is assigned to only the operator

Operator (before pushing merge the commit)

• Squashed PR branch and rebased onto develop
• Sanity-checked history
• Pushed PR branch to GitHub
• Ran _select dev.shared && CI_COMMIT_REF_NAME=develop make -C terraform/shared apply_keep_unused _{or this PR is not labeled deploy:shared}
• Ran _select dev.gitlab && python scripts/create_gitlab_snapshot.py --no-restart (see operator manual for details) _{or this PR is not labeled backup:gitlab}
• Ran _select dev.gitlab && CI_COMMIT_REF_NAME=develop make -C terraform/gitlab apply _{or this PR is not labeled deploy:gitlab}
• Ran _select anvildev.shared && CI_COMMIT_REF_NAME=develop make -C terraform/shared apply_keep_unused _{or this PR is not labeled deploy:shared}
• Ran _select anvildev.gitlab && python scripts/create_gitlab_snapshot.py --no-restart (see operator manual for details) _{or this PR is not labeled backup:gitlab}
• Ran _select anvildev.gitlab && CI_COMMIT_REF_NAME=develop make -C terraform/gitlab apply _{or this PR is not labeled deploy:gitlab}
• Checked the items in the next section _{or this PR is labeled deploy:gitlab}
• PR is assigned to only the system administrator _{or this PR is not labeled deploy:gitlab}

System administrator

• Background migrations for dev.gitlab are complete _{or this PR is not labeled deploy:gitlab}
• Background migrations for anvildev.gitlab are complete _{or this PR is not labeled deploy:gitlab}
• PR is assigned to only the operator

Operator (before pushing merge the commit)

• Ran _select dev.gitlab && make -C terraform/gitlab/runner _{or this PR is not labeled deploy:runner}
• Ran _select anvildev.gitlab && make -C terraform/gitlab/runner _{or this PR is not labeled deploy:runner}
• Added sandbox label
• Pushed PR branch to GitLab dev
• Pushed PR branch to GitLab anvildev
• Build passes in sandbox deployment
• Build passes in anvilbox deployment
• Reviewed build logs for anomalies in sandbox deployment
• Reviewed build logs for anomalies in anvilbox deployment
• The title of the merge commit starts with the title of this PR
• Added PR # reference to merge commit title
• Collected commit title tags in merge commit title _{but excluded any p tags}
• Moved connected issue to Merged lower column in ZenHub
• Moved blocked issues to Triage _{or no issues are blocked on the connected issue}
• Closed related Dependabot PRs with a comment referencing the corresponding commit in this PR _{or this PR does not include any such commits}
• Pushed merge commit to GitHub

Operator (after pushing the merge commit)

• Pushed merge commit to GitLab dev
• Pushed merge commit to GitLab anvildev
• Build passes on GitLab dev
• Reviewed build logs for anomalies on GitLab dev
• Build passes on GitLab anvildev
• Reviewed build logs for anomalies on GitLab anvildev
• Ran _select dev.shared && make -C terraform/shared apply _{or this PR is not labeled deploy:shared}
• Ran _select anvildev.shared && make -C terraform/shared apply _{or this PR is not labeled deploy:shared}
• Deleted PR branch from GitHub
• Deleted PR branch from GitLab dev
• Deleted PR branch from GitLab anvildev

Operator

• At least 24 hours have passed since anvildev.shared was last deployed
• Ran scripts/export_inspector_findings.py against anvildev, imported results to Google Sheet and posted screenshot of relevant¹ findings as a comment on the connected issue.
• Propagated the deploy:shared, deploy:gitlab, deploy:runner and backup:gitlab labels to the next promotion PRs _{or this PR carries none of these labels}
• Propagated any specific instructions related to the deploy:shared, deploy:gitlab, deploy:runner and backup:gitlab labels, from the description of this PR to that of the next promotion PRs _{or this PR carries none of these labels}
• PR is assigned to only the system administrator

¹A relevant finding is a high or critical vulnerability in an image
that is used within the security boundary. Images not used within the boundary
are tracked in azul.docker_images under a key starting with _.

System administrator

• No currently reported vulnerability requires immediate attention
• PR is assigned to no one

Shorthand for review comments

• L line is too long
• W line wrapping is wrong
• Q bad quotes
• F other formatting problem

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 85.54%. Comparing base (40c3bcb) to head (a71087e).
Report is 13 commits behind head on develop.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #6892   +/-   ##
========================================
  Coverage    85.54%   85.54%           
========================================
  Files          147      147           
  Lines        21226    21226           
========================================
  Hits         18158    18158           
  Misses        3068     3068           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[coveralls]

coverage: 85.563%. remained the same
when pulling a71087e on upgrades/2025-02-03
into 40c3bcb on develop.

[hannes-ucsc]

This looks like a downgrade, which we wouldn't want.

[achave11-ucsc]

I wasn't able to obtain compelling evidence from the Amazon or CIS websites regarding any of these AMI versions.
But, when running the command that operators usually run without removing additional hits, one may observe the creation date of the v01 version (2025-01-31T16:55:33) to be more recent than its v12 predecessor (2024-12-23T17:28:12.000Z).

❯ aws ec2 describe-images --owners aws-marketplace --filters="Name=name,Values=*4c096026-c6b0-440c-bd2f-6d34904e4fc6*" | jq -r '.Images[] | .CreationDate+"\t"+.ImageId+"\t"+.Name' | sort
2024-11-20T16:07:20.000Z        ami-0a21f5234dcf8057a   CIS Amazon Linux 2 Kernel 4.14 Benchmark - Level 1 - v11 -4c096026-c6b0-440c-bd2f-6d34904e4fc6
2024-12-23T17:28:12.000Z        ami-0a5d7e321c34492d7   CIS Amazon Linux 2 Kernel 4.14 Benchmark - Level 1 - v12 -4c096026-c6b0-440c-bd2f-6d34904e4fc6
2025-01-31T16:55:33.000Z        ami-005aa69a4e42cc74d   CIS Amazon Linux 2 Kernel 4.14 Benchmark - Level 1 - v01 -4c096026-c6b0-440c-bd2f-6d34904e4fc6

Perhaps the values 01 or 12 in v01 or v12 respectively, are representative of the month of the release more than an actual release number count.

[achave11-ucsc]

Include a note in operator manual about the versioning scheme of the IAM image, regarding the number representing the release month.

[hannes-ucsc]

Let's add the operator note (as discussed in PL) in this PR.

[hannes-ucsc]

The backquotes need to be doubled up.

[hannes-ucsc]

By our standards, this documentation change needs to be in a separate commit.

[hannes-ucsc]

Suggested change

	release, and not a sequential release number. For example, `v01` representing
	January of the current year is newer than `v12` (December) of the previous year.
	release, and is not a monotonically increasing value. For example, `v01` representing
	January of the current year is newer than `v12` (December) of the previous year.

[hannes-ucsc]

Suggested change

	release, and not a sequential release number. For example, `v01` representing
	January of the current year is newer than `v12` (December) of the previous year.
	release, and not a sequential release number.

The second sentence isn't generally true, a v01 release may or may not be older than a v12 release. Then again, this should be immediately apparent to a reasonable person, so I don't think the example is necessary.

[achave11-ucsc]

Transitive dependency updates were stale, ran make requirements_update again and committed the results.