Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kubeadm: graduate the UnversionedKubeletConfigMap FG to GA #110327

Merged
merged 2 commits into from Jun 7, 2022
Merged

kubeadm: graduate the UnversionedKubeletConfigMap FG to GA #110327

merged 2 commits into from Jun 7, 2022

Conversation

neolit123
Copy link
Member

@neolit123 neolit123 commented Jun 1, 2022 •
edited

What type of PR is this?

/kind deprecation cleanup feature

What this PR does / why we need it:

cmd/kubeadm

  • lock the FG to true by default
  • cleanup wrappers and logic related to versioned vs unversioned
    naming of API objects (CMs and RBAC)
  • update unit tests

test/e2e_kubeadm

  • don't require clusteconfiguration.kubernetesVersion
  • use the helper functions GetConfigMap, ExpectRole, ExpectRoleBinding
    (these were used before UnversionedKubeletConfig went Beta)

Which issue(s) this PR fixes:

xref kubernetes/kubeadm#1582

Special notes for your reviewer:

NONE

Does this PR introduce a user-facing change?

kubeadm: graduate the kubeadm specific feature gate UnversionedKubeletConfigMap to GA and lock it to "true" by default. The kubelet related ConfigMap and RBAC rules are now locked to have a simplified naming "*kubelet-config" instead of the legacy naming "*kubelet-config-x.yy", where "x.yy" was the version of the control plane. If you have previously used the old naming format with UnversionedKubeletConfigMap=false, you must manually copy the config map from kube-system/kubelet-config-x.yy to kube-system/kubelet-config before upgrading to 1.25.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

- [KEP]: https://github.com/kubernetes/enhancements/issues/2915

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. kind/deprecation Categorizes issue or PR as related to a feature/enhancement marked for deprecation. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. kind/feature Categorizes issue or PR as related to a new feature. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Jun 1, 2022
@neolit123 neolit123 mentioned this pull request Jun 1, 2022
Closed
12 tasks
@k8s-ci-robot k8s-ci-robot added area/kubeadm area/test sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. sig/testing Categorizes an issue or PR as relevant to SIG Testing. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jun 1, 2022
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: neolit123

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 1, 2022
@neolit123 neolit123 changed the title kubeadm: graduate the UnversionedKubeletConfig FG to GA kubeadm: graduate the UnversionedKubeletConfigMap FG to GA Jun 1, 2022
@neolit123
Copy link
Member Author

/hold for review
/cc @pacoxu

/cc @fabriziopandini @sbueringer
FYI related to CAPI.

@k8s-ci-robot k8s-ci-robot requested a review from pacoxu June 1, 2022 13:31
@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 1, 2022
@neolit123
Copy link
Member Author

/triage accepted
/priority important-soon
/milestone v1.25

@k8s-ci-robot k8s-ci-robot added the triage/accepted Indicates an issue or PR is ready to be actively worked on. label Jun 1, 2022
@k8s-ci-robot k8s-ci-robot added this to the v1.25 milestone Jun 1, 2022
@k8s-ci-robot k8s-ci-robot added priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Jun 1, 2022
@neolit123
kubeadm: graduate the UnversionedKubeletConfig FG to GA
9f6df1d 
- lock the FG to true by default
- cleanup wrappers and logic related to versioned vs unversioned
naming of API objects (CMs and RBAC)
- update unit tests
@neolit123
e2e_kubeadm: cleanup tests for UnversionedKubeletConfig
403ca0c 
- don't require clusteconfiguration.kubernetesVersion
- use the helper functions GetConfigMap, ExpectRole, ExpectRoleBinding
(these were used before UnversionedKubeletConfig went Beta)
@neolit123 neolit123 force-pushed the 1.25-ga-unversioned-kubelet-config branch from e8f9ea0 to 403ca0c Compare June 1, 2022 14:17
@neolit123
Copy link
Member Author

/retest

@designirisagent designirisagent mentioned this pull request Jun 1, 2022
Closed
@pacoxu
Copy link
Member

pacoxu commented Jun 6, 2022 •
edited

If user upgraded the cluster in v1.24 using kubeadm-v1.24 upgrade apply v1.24.x --feature-gates=UnversionedKubeletConfigMap=false, the user have to run kubeadm-v1.24 upgrade apply v1.24.x --feature-gates=UnversionedKubeletConfigMap=true before runningkubeadm-v1.25 upgrade apply v1.25.x.
The user will fail to upgrade directly to v1.25 as kubelet-config configmap does not exist in his cluster.
This is not expected.

We should lock the feature gate to true in v1.25, but we have to keep the logic of legacy code until we remove this feature gate.

  • v1.24: we allow kubeadm-v1.24 upgrade apply v1.24.x --feature-gates=UnversionedKubeletConfigMap=false
    • user can use only kubelet-config-1.24 configmap.
  • v1.25: we disallow kubeadm-v1.25 upgrade apply v1.25.x --feature-gates=UnversionedKubeletConfigMap=false
    • user have to switch to use kubelet-config configmap after v1.25.
  • v1.26 or v1.27: we remove the feature gate and the legacy logic.

@neolit123
Copy link
Member Author

If user upgraded the cluster in v1.24 using kubeadm-v1.24 upgrade apply v1.24.x --feature-gates=UnversionedKubeletConfigMap=false, the user have to run kubeadm-v1.24 upgrade apply v1.24.x --feature-gates=UnversionedKubeletConfigMap=true before runningkubeadm-v1.25 upgrade apply v1.25.x.
The user will fail to upgrade directly to v1.25 as kubelet-config configmap does not exist in his cluster.
This is not expected.

maybe we can add text in the release note about it.

e.g. "if you were not using the new naming format (UnversionedKubeletConfigMap=false) you must manually create / copy the new unversioned kubelet config map 'kube-system/kubelet-config'" before upgrading to 1.25"

We should lock the feature gate to true in v1.25, but we have to keep the logic of legacy code until we remove this feature gate.

hm, but the kubeadm 1.25 binary is expected to upgrade to 1.25.x versions and not 1.24.x.
i think we just have to instruct the users that 1.25 needs the "kubelet-config" naming.

@pacoxu
Copy link
Member

pacoxu commented Jun 6, 2022

hm, but the kubeadm 1.25 binary is expected to upgrade to 1.25.x versions and not 1.24.x.
i think we just have to instruct the users that 1.25 needs the "kubelet-config" naming.

That is acceptable.
/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 6, 2022
@RA489
Copy link

RA489 commented Jun 6, 2022 •
edited

/lgtm

@neolit123
Copy link
Member Author

neolit123 commented Jun 6, 2022 •
edited

our CI is currently broken due to failing prow pods, we might as well wait until that is resolved before merging:
https://k8s-testgrid.appspot.com/sig-cluster-lifecycle-kubeadm

also this is still an issue in CI:
kubernetes/kubeadm#2671 (comment)

@neolit123
Copy link
Member Author

/hold cancel
CI is green

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 7, 2022
@k8s-ci-robot k8s-ci-robot merged commit 9fad5c9 into kubernetes:master Jun 7, 2022
@Tassatux Tassatux mentioned this pull request Nov 22, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@fabriziopandini fabriziopandini Awaiting requested review from fabriziopandini

@RA489 RA489 Awaiting requested review from RA489

@pacoxu pacoxu Awaiting requested review from pacoxu

@sbueringer sbueringer Awaiting requested review from sbueringer

Assignees

@pacoxu pacoxu

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/kubeadm area/test cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. kind/deprecation Categorizes issue or PR as related to a feature/enhancement marked for deprecation. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Milestone
v1.25
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@neolit123 @k8s-ci-robot @pacoxu @RA489