Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[release-4.15] OCPBUGS-52183: UPSTREAM: <carry>: kubelet/cm: fix bug where kubelet restarts from missing cpuset cgroup #2227

Open
wants to merge 4 commits into
base: release-4.15
Choose a base branch
from
Open

[release-4.15] OCPBUGS-52183: UPSTREAM: <carry>: kubelet/cm: fix bug where kubelet restarts from missing cpuset cgroup #2227

wants to merge 4 commits into from
+111 −23

Conversation

haircommander
Copy link
Member

This is a manual cherry-pick of #2200

NONE

haircommander and others added 4 commits February 28, 2025 19:15
@haircommander
UPSTREAM: <carry>: kubelet/cm: fix bug where kubelet restarts from mi…
a627ddf 
…ssing cpuset cgroup

on None cpumanager policy, cgroupv2, and systemd cgroup manager, kubelet
could get into a situation where it believes the cpuset cgroup was created
(by libcontainer in the cgroupfs) but systemd has deleted it, as it wasn't requested
to create it. This causes one unnecessary restart, as kubelet fails with

`failed to initialize top level QOS containers: root container [kubepods] doesn't exist.`

This only causes one restart because the kubelet skips recreating the cgroup
if it already exists, but it's still a bother and is fixed this way

adapted version of kubernetes@c51195d, avoid merge conflict
this can be dropped in the 4.19 rebase

Signed-off-by: Peter Hunt <pehunt@redhat.com>
@haircommander
UPSTREAM: <carry>: kubelet/cm: move CPU reading from cm to cm/cpumanager
e669b1e 
Adapted from kubernetes@77d03e4 to avoid merge conflict
This can be dropped in the 4.19 rebase

Authored-by: Francesco Romani <fromani@redhat.com>
Signed-off-by: Peter Hunt <pehunt@redhat.com>
@ffromani @haircommander
UPSTREAM: <carry>: e2e_node: add a test to verify the kubelet starts
c39c30f 
with systemd cgroup driver and cpumanager none policy.

This was originally planned to be a correctness check for
https://issues.k8s.io/125923, but it was difficult to reproduce the bug,
so it's now a regression test against it.

Signed-off-by: Francesco Romani <fromani@redhat.com>
Signed-off-by: Peter Hunt <pehunt@redhat.com>
@haircommander
UPSTREAM: <carry>: e2e_node: use restart instead of start stop
b42515c 
Signed-off-by: Peter Hunt <pehunt@redhat.com>
@openshift-ci-robot openshift-ci-robot added backports/unvalidated-commits Indicates that not all commits come to merged upstream PRs. jira/severity-important Referenced Jira bug's severity is important for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. labels Feb 28, 2025
@openshift-ci-robot
Copy link

@haircommander: This pull request references Jira Issue OCPBUGS-52183, which is valid.

7 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.15.z) matches configured target version for branch (4.15.z)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
  • release note type set to "Release Note Not Required"
  • dependent bug Jira Issue OCPBUGS-45931 is in the state Closed (Done-Errata), which is one of the valid states (VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA))
  • dependent Jira Issue OCPBUGS-45931 targets the "4.16.z" version, which is one of the valid target versions: 4.16.0, 4.16.z
  • bug has dependents

Requesting review from QA contact:
/cc @cpmeadors

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

This is a manual cherry-pick of #2200

NONE

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. label Feb 28, 2025
@openshift-ci-robot
Copy link

@haircommander: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

@openshift-ci openshift-ci bot requested a review from cpmeadors February 28, 2025 20:58
@haircommander haircommander mentioned this pull request Feb 28, 2025
Closed
@openshift-ci openshift-ci bot requested review from deads2k and mrunalp February 28, 2025 20:58
@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Feb 28, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: haircommander
Once this PR has been reviewed and has the lgtm label, please assign deads2k for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@haircommander
Copy link
Member Author

failure looks unrelated
/retest

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Mar 3, 2025

@haircommander: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/unit b42515c link true /test unit

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cpmeadors cpmeadors Awaiting requested review from cpmeadors

@deads2k deads2k Awaiting requested review from deads2k

@mrunalp mrunalp Awaiting requested review from mrunalp

Assignees
No one assigned
Labels
backports/unvalidated-commits Indicates that not all commits come to merged upstream PRs. jira/severity-important Referenced Jira bug's severity is important for the branch this PR is targeting. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@haircommander @openshift-ci-robot @ffromani