将 kubernetes 从 1.23.9 升级至 1.24.3，container runtime 也从 Docker 切换到 containerd，但 直接kubelet 无法启动：
# kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.3", GitCommit:"aef86a93758dc3cb2c658dd9657ab4ad4afc21cb", GitTreeState:"clean", BuildDate:"2022-07-13T14:30:46Z", GoVersion:"go1.18.3", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.9", GitCommit:"c1de2d70269039fe55efb98e737d9a29f9155246", GitTreeState:"clean", BuildDate:"2022-07-13T14:19:57Z", GoVersion:"go1.17.11", Compiler:"gc", Platform:"linux/amd64"}
# systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: activating (auto-restart) (Result: exit-code) since Tue 2022-08-16 17:38:22 CST; 3s ago
Docs: https://kubernetes.io/docs/home/
Process: 120499 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (code=exited, status=1/FAILURE)
Main PID: 120499 (code=exited, status=1/FAILURE)
Aug 16 17:38:22 MyServer kubelet[120499]: Insecure values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, >
Aug 16 17:38:22 MyServer kubelet[120499]: --tls-min-version string Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, Vers>
Aug 16 17:38:22 MyServer kubelet[120499]: --tls-private-key-file string File containing x509 private key matching --tls-cert-file. (DEPRECATED: This par>
Aug 16 17:38:22 MyServer kubelet[120499]: --topology-manager-policy string Topology Manager policy to use. Possible values: 'none', 'best-effort', 'restric>
Aug 16 17:38:22 MyServer kubelet[120499]: --topology-manager-scope string Scope to which topology hints applied. Topology Manager collects hints from Hint>
Aug 16 17:38:22 MyServer kubelet[120499]: -v, --v Level number for the log level verbosity
Aug 16 17:38:22 MyServer kubelet[120499]: --version version[=true] Print version information and quit
Aug 16 17:38:22 MyServer kubelet[120499]: --vmodule pattern=N,... comma-separated list of pattern=N settings for file-filtered logging (only works>
Aug 16 17:38:22 MyServer kubelet[120499]: --volume-plugin-dir string The full path of the directory in which to search for additional third party vol>
Aug 16 17:38:22 MyServer kubelet[120499]: --volume-stats-agg-period duration Specifies interval for kubelet to calculate and cache the volume disk usage for
检查下kubelet报错日志
-- Subject: A start job for unit kubelet.service has finished successfully -- Defined-By: systemd -- Support: http://www.ubuntu.com/support -- -- A start job for unit kubelet.service has finished successfully. -- -- The job identifier is 25798. Aug 16 17:38:12 MyServer kubelet[120267]: Error: failed to parse kubelet flag: unknown flag: --network-plugin Aug 16 17:38:12 MyServer kubelet[120267]: Usage: Aug 16 17:38:12 MyServer kubelet[120267]: kubelet [flags] Aug 16 17:38:12 MyServer kubelet[120267]: Flags: Aug 16 17:38:12 MyServer kubelet[120267]: --add-dir-header
注意这一句：Error: failed to parse kubelet flag: unknown flag: –network-plugin
处理方法：
修改 /var/lib/kubelet/kubeadm-flags.env 配置文件
KUBELET_KUBEADM_ARGS=”–network-plugin=cni –pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.6″
修改为
KUBELET_KUBEADM_ARGS=”–container-runtime=remote –container-runtime-endpoint=unix:///var/run/containerd/containerd.sock –pod-infra-container-image=registry.aliyuncs.com/google_containers
/pause:3.6″
重启kubelet，解决问题
捎带手把集群升级一下，非常耐斯
