基于vmware16 和 ubuntu20.04, 搭建单节点 kubernetes 1.22.2

admin管理员组

文章数量:1565349

---

安装环境

• Windows： 10 家庭版

• VMware Pro：16

• Ubuntu版本： 20.04.3

• Docker版本： 20.10.7

• Kubernetes版本： 1.22.2

• Calico版本： v3.23

vmware

参考在win10上使用VMware Pro16 安装 CentOS 7.9 设置静态IP 可以访问外网_琦彦-CSDN博客中安装vmware 部分

比较简单, 不再赘述

ubuntu

镜像下载

Ubuntu 20.04.3 LTS (Focal Fossa)

https://mirrors.bupt.edu/ubuntu-releases/20.04.3/ubuntu-20.04.3-desktop-amd64.iso

安装过程比较简单, 不再赘述

我的虚拟机信息

按照提示, 一步步安装

安装完成的画面

行万里路，此处相逢，共话云原生之道。

偶逗趣事，明月清风，与君同坐。

琦彦，在云原生百宝箱公众号等你，与你一起探讨应用迁移，GitOps，二次开发，解决方案，CNCF生态，以及生活况味。

安装Ubuntu后必做的几件事情

静态IP

配置网络修改/etc/netplan/00-installer-config.yaml如下，我这网卡是ens33

# Let NetworkManager manage all devices on this system
network:
  version: 2
  renderer: NetworkManager
  ethernets:
    ens33:  # change your's                            
      dhcp4: no
      addresses: [192.168.172.129/24]      # change your's   
      gateway4: 192.168.172.2             # change your's     
      nameservers:
        addresses: [114.114.114.114]      # change your's

保存后运行

netplan apply

可以使用ip a查看修改情况，然后取消系统自带的systemd-resolved.service这个dns解析服务，是可选的

允许root使用ssh远程登录终端

测试版本使用的Ubuntu 20.04，Ubuntu 16.4以上版本都可以这样配置。

最简单的方法：直接安装openssh-server，然后测试时候可以ssh登录，如果不可以，请使用下面的方法进行配置。

sudo apt install openssh-server

1. 设置root密码

执行命令后，依次输入当前登录用户密码，要设置的root密码，确认root密码

sudo passwd root

为了方便将密码设置为123456即可

 这样还不能远程登录，需要修改配置文件

2. 修改ssh配置文件

如果没有安装ssh-server，执行安装命令，已经安装的跳过即可

sudo apt install openssh-server

修改配置文件/etc/ssh/sshd_config

在vim中搜索定位PermitRootLogin，可直接查找：

/PermitRootLogin

修改以下配置： 33 #LoginGraceTime 2m 34 #PermitRootLogin prohibit-password 35 #StrictModes yes 36 #MaxAuthTries 6 37 #MaxSessions 10

修改为：

 LoginGraceTime 2m
 PermitRootLogin yes
 StrictModes yes
 #MaxAuthTries 6
 #MaxSessions 10

3. 重启ssh，使配置生效

sudo service ssh restart

现在可是使用root用户登录ssh了！！！

A stop job is running for snappy daemon

关机或重启会遇到A stop job is running for snappy daemon这问题，

为了能够快速重启或关机修改配置文件

vim /etc/systemd/system.conf

修改

DefaultTimeoutStartSec=3s
DefaultTimeoutStopSec=3s

执行下面这条命令使其生效

systemctl daemon-reload

关闭防火墙

查看防火墙状态 inactive说明是未激活

sudo ufw status

开机不启动防火墙，重启即可生效

sudo ufw disable

关闭Swap

Linux的Swap内存交换机制是一定要关闭的，否则会因为内存交换而影响性能以及稳定性。 Kubernetes 1.8开始要求关闭系统的Swap，如果不关闭，默认配置下kubelet将无法启动。 执行swapoff -a可临时关闭，但系统重启后恢复 编辑/etc/fstab，注释掉包含swap的那一行即可，重启后可永久关闭，如下所示

Cenos7

vim /etc/fstab
​
/dev/mapper/centos-root / xfs defaults 0 0
UUID=20ca01ff-c5eb-47bc-99a0-6527b8cb246e /boot xfs defaults 0 0
# /dev/mapper/centos-swap swap

Ubuntu20.4

或直接执行

sed -i '/ swap / s/^/#/' /etc/fstab

备注

linux下通过sed命令直接修改文件内容

1、我想替换文件中的 11为 12,可以使用

*sed 's/11/12/g' xxx.log 这种方式，其实并没有修改build.xml文件的内容。如果想保存修改，通常的做法就需要重定向到另外的一个文件

sed 's/11/12/g' xxx.log > xxx.log .tmp 这样，xxx.log .tmp文件就是修改后的文件. 如果无误，那么就可以用 mv xxx.log .tmp xxx.log 覆盖原文件。*

2、如果想直接修改源文件，而没有这样的过程，可以用下面的命令 sed -i 's/11/12/g' xxx.log

3、还有一个更简单的方法 sed -in-place -e 's/abc/cba/g' build.xml

关闭成功后，使用top命令查看，如下图所示表示正常

 或者使用free -mh命令查看，如下图所示表示正常

安装docker

1、删除docker相关组件

sudo apt-get autoremove docker docker-ce docker-engine  docker.io  containerd runc

2、更新apt-get

sudo apt-get update

3、安装 apt 依赖包，用于通过HTTPS来获取仓库

sudo apt-get install apt-transport-https ca-certificates curl gnupg-agent software-properties-common

4、添加 Docker 的官方 GPG 密钥

curl -fsSL https://download.docker/linux/ubuntu/gpg | sudo apt-key add -

5、设置稳定版仓库(添加到/etc/apt/sources.list中)

sudo add-apt-repository "deb [arch=amd64] https://download.docker/linux/ubuntu $(lsb_release -cs) stable"

6、更新apt-get

sudo apt-get update

7、查询docker-ce版本

sudo apt-cache policy docker-ce
​
# 示例如下
$ sudo apt-cache policy docker-ce
docker-ce:
  Installed: 5:20.10.8~3-0~ubuntu-focal
  Candidate: 5:23.0.1-1~ubuntu.20.04~focal
  Version table:
     5:23.0.1-1~ubuntu.20.04~focal 500
        500 https://download.docker/linux/ubuntu focal/stable amd64 Packages
     5:23.0.0-1~ubuntu.20.04~focal 500
        500 https://download.docker/linux/ubuntu focal/stable amd64 Packages
     5:20.10.23~3-0~ubuntu-focal 500
        500 https://download.docker/linux/ubuntu focal/stable amd64 Packages
     5:20.10.22~3-0~ubuntu-focal 500
        500 https://download.docker/linux/ubuntu focal/stable amd64 Packages
     5:20.10.21~3-0~ubuntu-focal 500
        500 https://download.docker/linux/ubuntu focal/stable amd64 Packages
     5:20.10.20~3-0~ubuntu-focal 500
        500 https://download.docker/linux/ubuntu focal/stable amd64 Packages
     5:20.10.19~3-0~ubuntu-focal 500
        500 https://download.docker/linux/ubuntu focal/stable amd64 Packages
     5:20.10.18~3-0~ubuntu-focal 500
        500 https://download.docker/linux/ubuntu focal/stable amd64 Packages
     5:20.10.17~3-0~ubuntu-focal 500
        500 https://download.docker/linux/ubuntu focal/stable amd64 Packages
     5:20.10.16~3-0~ubuntu-focal 500
        500 https://download.docker/linux/ubuntu focal/stable amd64 Packages
     5:20.10.15~3-0~ubuntu-focal 500
        500 https://download.docker/linux/ubuntu focal/stable amd64 Packages
     5:20.10.14~3-0~ubuntu-focal 500
        500 https://download.docker/linux/ubuntu focal/stable amd64 Packages
     5:20.10.13~3-0~ubuntu-focal 500
        500 https://download.docker/linux/ubuntu focal/stable amd64 Packages
     5:20.10.12~3-0~ubuntu-focal 500
        500 https://download.docker/linux/ubuntu focal/stable amd64 Packages
     5:20.10.11~3-0~ubuntu-focal 500
        500 https://download.docker/linux/ubuntu focal/stable amd64 Packages
     5:20.10.10~3-0~ubuntu-focal 500
        500 https://download.docker/linux/ubuntu focal/stable amd64 Packages
     5:20.10.9~3-0~ubuntu-focal 500
        500 https://download.docker/linux/ubuntu focal/stable amd64 Packages
    5:20.10.8~3-0~ubuntu-focal 500
        500 https://download.docker/linux/ubuntu focal/stable amd64 Packages
        100 /var/lib/dpkg/status
     5:20.10.7~3-0~ubuntu-focal 500
        500 https://download.docker/linux/ubuntu focal/stable amd64 Packages
     5:20.10.6~3-0~ubuntu-focal 500
        500 https://download.docker/linux/ubuntu focal/stable amd64 Packages
     5:20.10.5~3-0~ubuntu-focal 500
        500 https://download.docker/linux/ubuntu focal/stable amd64 Packages
​

8、安装指定版本

sudo apt-get install  docker-ce=5:20.10.7~3-0~ubuntu-focal docker-ce-cli=5:20.10.7~3-0~ubuntu-focal

9、验证安装是否成功

docker -version

更新 cgroupdriver 为systemd

sudo tee /etc/docker/daemon.json <<-'EOF'
{
  "registry-mirrors": ["https://uy35zvn6.mirror.aliyuncs"],
  "exec-opts": ["native.cgroupdriver=systemd"]
}
EOF
​
systemctl daemon-reload
systemctl restart docker

验证

root@fly-virtual-machine:~# docker version
Client:
 Version:           20.10.7
 API version:       1.41
 Go version:        go1.13.8
 Git commit:        20.10.7-0ubuntu1~20.04.1
 Built:             Wed Aug  4 22:52:25 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true
​
Server:
 Engine:
  Version:          20.10.7
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.8
  Git commit:       20.10.7-0ubuntu1~20.04.1
  Built:            Wed Aug  4 19:07:47 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.5.2-0ubuntu1~20.04.2
  GitCommit:        
 runc:
  Version:          1.0.0~rc95-0ubuntu1~20.04.2
  GitCommit:        
 docker-init:
  Version:          0.19.0
  GitCommit:      

安装Kubernetes

iptables配置

将桥接的IPv4/IPv6流量传递到iptables的链

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF
​
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
​
sudo sysctl --system

bridge-nf

bridge-nf 使得 netfilter 可以对 Linux 网桥上的 IPv4/ARP/IPv6 包过滤。比如，设置net.bridge.bridge-nf-call-iptables＝1后，二层的网桥在转发包时也会被 iptables的 FORWARD 规则所过滤。常用的选项包括：

• net.bridge.bridge-nf-call-arptables：是否在 arptables 的 FORWARD 中过滤网桥的 ARP 包

• net.bridge.bridge-nf-call-ip6tables：是否在 ip6tables 链中过滤 IPv6 包

• net.bridge.bridge-nf-call-iptables：是否在 iptables 链中过滤 IPv4 包

• net.bridge.bridge-nf-filter-vlan-tagged：是否在 iptables/arptables 中过滤打了 vlan 标签的包。

防火墙是保护服务器和基础设施安全的重要工具。在 Linux 生态系统中，iptables 是使 用很广泛的防火墙工具之一，它基于内核的包过滤框架（packet filtering framework） netfilter。

Linux 上最常用的防火墙工具是 iptables。iptables 与协议栈内有包过滤功能的 hook 交 互来完成工作。这些内核 hook 构成了 netfilter 框架。

每个进入网络系统的包（接收或发送）在经过协议栈时都会触发这些 hook，程序 可以通过注册 hook 函数的方式在一些关键路径上处理网络流量。iptables 相关的内核模 块在这些 hook 点注册了处理函数，因此可以通过配置 iptables 规则来使得网络流量符合 防火墙规则。

备注: 深入理解 iptables 和 netfilter 架构

apt 包更新, 安装apt-transport-https\ca-certificates\curl

Update the apt package index and install packages needed to use the Kubernetes apt repository:

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl

添加GPG 密钥

sudo curl -s https://mirrors.aliyun/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add -

备注:

GnuPG，简称 GPG，来自 http://www.gnupg，是 GPG 标准的一个免费实现。不管是 Linux 还是 Windows 平台，都可以使用。GPGneng 可以为文件生成签名、管理密匙以及验证签名。

添加 Kubernetes apt 存储库

sudo tee /etc/apt/sources.list.d/kubernetes.list <<-'EOF'
deb https://mirrors.aliyun/kubernetes/apt kubernetes-xenial main
EOF

更新apt 包, 安装kubelet, kubeadm and kubectl

Update apt package index, install kubelet, kubeadm and kubectl

sudo apt-get update
sudo apt-get install -y kubelet=1.22.2-00 kubeadm=1.22.2-00 kubectl=1.22.2-00 
sudo apt-mark hold kubelet kubeadm kubectl

指定版本 apt-get install -y kubelet=1.22.2-00 kubeadm=1.22.2-00 kubectl=1.22.2-00 

最新版本 apt-get install -y kubelet kubeadm kubectl

备注:

apt-mark用法

apt-mark [选项] {auto|manual} 软件包1 [软件包2 …]

apt-mark常用命令

• auto – 标记指定软件包为自动安装

• manual – 标记指定软件包为手动安装

• minimize-manual – Mark all dependencies of meta packages as automatically installed.

• hold – 标记指定软件包为保留(held back)，阻止软件自动更新

• unhold – 取消指定软件包的保留(held back)标记，解除阻止自动更新

• showauto – 列出所有自动安装的软件包

• showmanual – 列出所有手动安装的软件包

• showhold – 列出设为保留的软件包

使用kubeadm init初始化集群

kubeadm init \
 --image-repository registry.aliyuncs/google_containers \
 --kubernetes-version v1.22.2 \
 --pod-network-cidr=192.168.0.0/16 \
 --apiserver-advertise-address=192.168.172.129

apiserver-advertise-address： 表示你的kubernetes集群的控制平面节点的 API server 的广播地址

pod-network-cidr：表示你的kubernetes集群的pod网段  

成功输出信息

root@fly-virtual-machine:/etc/netplan# kubeadm init \
>  --image-repository registry.aliyuncs/google_containers \
>  --kubernetes-version v1.22.2 \
>  --pod-network-cidr=192.168.0.0/16 \
>  --apiserver-advertise-address=192.168.172.129
[init] Using Kubernetes version: v1.22.2
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [fly-virtual-machine kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.172.129]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [fly-virtual-machine localhost] and IPs [192.168.172.129 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [fly-virtual-machine localhost] and IPs [192.168.172.129 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 11.020909 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.22" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node fly-virtual-machine as control-plane by adding the labels: [node-role.kubernetes.io/master(deprecated) node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node fly-virtual-machine as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: 6igmn8.d4zk3hmr0rr0j7k2
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy
​
Your Kubernetes control-plane has initialized successfully!
​
To start using your cluster, you need to run the following as a regular user:
​
  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config
​
Alternatively, if you are the root user, you can run:
​
  export KUBECONFIG=/etc/kubernetes/admin.conf
​
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/
​
Then you can join any number of worker nodes by running the following on each as root:
​
kubeadm join 192.168.172.129:6443 --token 6igmn8.d4zk3hmr0rr0j7k2 \
    --discovery-token-ca-cert-hash sha256:78f0796dee6bedf5f7250843be190cc3b63b97c5bccb91839f74a1e8b07efac6 

复制 kubeconfig配置文件

$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config

去除master节点的污点

$ kubectl taint nodes --all node-role.kubernetes.io/master-

当创建单机版的 k8s 时，这个时候 master 节点是默认不允许调度 pod 的，需要执行

• kubectl taint nodes --all node-role.kubernetes.io/master-

命令将 master 标记为可调度.

安装calico cni 插件

$ kubectl create -f https://projectcalico.docs.tigera.io/archive/v3.23/manifests/tigera-operator.yaml
$ kubectl create -f https://projectcalico.docs.tigera.io/archive/v3.23/manifests/custom-resources.yaml

在创建此清单之前，请阅读其内容并确保其设置适合您的环境。

例如，您可能需要更改默认 IP 池 CIDR 以匹配您的 pod 网络 CIDR。

# 如果上一步你设置的--pod-network-cidr=10.244.0.0/16，可以通过以下方式替换

$ export POD_SUBNET=10.244.0.0/16
$ sed -i "s#192\.168\.0\.0/16#${POD_SUBNET}#" custom-resources.yaml

验证集群状态

执行命令kubectl get cs查看一下集群状态，确认个组件都处于healthy状态 如下所示表示正常

[root@master-1 ~]# kubectl get cs
NAME                         STATUS      MESSAGE             ERROR
controller-manager           Healthy     ok                  
scheduler                    Healthy     ok                  
etcd-0                       Healthy     {"health":"true"} 

问题: scheduler/controller-manager： dial tcp 127.0.0.1:10251: connect: connection refused

部署完master节点以后，执行kubectl get cs命令来检测组件的运行状态时，报如下错误：

root@fly-virtual-machine:/etc/netplan# kubectl get cs
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME                 STATUS      MESSAGE                                                                                       ERROR
scheduler            Unhealthy   Get "http://127.0.0.1:10251/healthz": dial tcp 127.0.0.1:10251: connect: connection refused   
etcd-0               Healthy     {"health":"true","reason":""}                                                                 
controller-manager   Healthy     ok                                                                                            
root@fly-virtual-machine:/etc/netplan# vim /etc/kubernetes/manifests/kube-scheduler.yaml 
root@fly-virtual-machine:/etc/netplan# systemctl restart kubelet.service
​

原因分析

出现这种情况，是/etc/kubernetes/manifests/下的kube-controller-manager.yaml和kube-scheduler.yaml设置的默认端口是0导致的，解决方式是注释掉对应的port即可，操作如下：

然后在master节点上重启kubelet，systemctl restart kubelet.service，然后重新查看就正常了

port=0是做什么用的? 关闭非安全端口

Kubernetes还原

集群初始化如果遇到问题，可以使用下面的命令进行清理

# 1.卸载服务

kubeadm reset

# 2.删除相关容器  #删除镜像

docker rm $(docker  ps -aq) -f
docker rmi $(docker images -aq) -f

# 3.删除上一个集群相关的文件

rm -rf  /var/lib/etcd
rm -rf  /etc/kubernetes
rm -rf $HOME/.kube
rm -rf /var/etcd
rm -rf /var/lib/kubelet/
rm -rf /run/kubernetes/
rm -rf ~/.kube/

# 4.清除网络

systemctl stop kubelet
systemctl stop docker
rm -rf /var/lib/cni/*
rm -rf /var/lib/kubelet/*
rm -rf /etc/cni/*
ifconfig cni0 down
ifconfig flannel.1 down
ifconfig docker0 down
ip link delete cni0
ip link delete flannel.1
systemctl start docker

# 5.卸载工具

apt autoremove -y kubelet kubectl kubeadm kubernetes-cni
删除/var/lib/kubelet/目录，删除前先卸载

for m in $(sudo tac /proc/mounts | sudo awk '{print $2}'|sudo grep /var/lib/kubelet);do
 
sudo umount $m||true
 
done
 
# 6.删除所有的数据卷

sudo docker volume rm $(sudo docker volume ls -q)

# 7.再次显示所有的容器和数据卷，确保没有残留

sudo docker ps -a
 
sudo docker volume ls

Kubernetes测试

部署 Deployment

kubectl apply -f <https://k8s.io/examples/application/deployment.yaml>
​
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 2 # tells deployment to run 2 pods matching the template
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

部署 NodePort

$ kubectl apply -f - <<EOF
apiVersion: v1
kind: Service
metadata:
  name: my-nginx
spec:
  selector:
    app: nginx
  type: NodePort
  ports:
    - protocol: TCP
      port: 80
EOF

验证

$ kubectl get all -owide
NAME                                    READY   STATUS    RESTARTS   AGE   IP             NODE       NOMINATED NODE   READINESS GATES
pod/nginx-deployment-66b6c48dd5-cqjqs   1/1     Running   0          85s   10.244.39.26   master-1   <none>           <none>
pod/nginx-deployment-66b6c48dd5-s6g49   1/1     Running   0          85s   10.244.39.20   master-1   <none>           <none>
​
NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE    SELECTOR
service/kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP        231d   <none>
service/my-nginx     NodePort    10.100.97.211   <none>        80:31932/TCP   23s    app=nginx
​
NAME                               READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES         SELECTOR
deployment.apps/nginx-deployment   2/2     2            2           85s   nginx        nginx:1.14.2   app=nginx
​
NAME                                          DESIRED   CURRENT   READY   AGE   CONTAINERS   IMAGES         SELECTOR
replicaset.apps/nginx-deployment-66b6c48dd5   2         2         2       85s   nginx        nginx:1.14.2   app=nginx,pod-template-hash=66b6c48dd5

​

集群内访问

$ curl 10.100.97.211
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
​
<p>For online documentation and support please refer to
<a href="http://nginx/">nginx</a>.<br/>
Commercial support is available at
<a href="http://nginx/">nginx</a>.</p>
​
<p><em>Thank you for using nginx.</em></p>
</body>
</html>

集群外访问

通过kubectl get services查看nginx服务对外暴露的端口是30573, 浏览器访问如下

 

常见异常

kubeadm init初始化集群异常

问题1: running with swap on is not supported. Please disable swap

root@fly-virtual-machine:/etc/docker# kubeadm init \
>  --image-repository registry.aliyuncs/google_containers \
>  --kubernetes-version v1.22.2 \
>  --pod-network-cidr=192.168.0.0/16 \
>  --apiserver-advertise-address=192.168.172.129
[init] Using Kubernetes version: v1.22.2
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
    [ERROR Swap]: running with swap on is not supported. Please disable swap
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher

参考上文: 关闭Swap章节

问题2 : [kubelet-check] Initial timeout of 40s passed.

[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
​
    Unfortunately, an error has occurred:
        timed out waiting for the condition
​
    This error is likely caused by:
        - The kubelet is not running
        - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
​
    If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
        - 'systemctl status kubelet'
        - 'journalctl -xeu kubelet'
​
    Additionally, a control plane component may have crashed or exited when started by the container runtime.
    To troubleshoot, list all containers using your preferred container runtimes CLI.
​
    Here is one example how you may list all Kubernetes containers running in docker:
        - 'docker ps -a | grep kube | grep -v pause'
        Once you have found the failing container, you can inspect its logs with:
        - 'docker logs CONTAINERID'
​
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher

通过journalctl -n 20 | grep kubelet命令查看日志

原因定位: 使用kubectl访问api-server服务失败，node “XXX“ not found

问题解决:

（1）使用kubectl

kubectl get node
The connection to the server 192.168.172.129:6443 was refused - did you specify the right host or port?

（2）查看kubelet日志

journalctl -xeu kubelet
 kubelet.go:2183] node "fly-k8s" not found

（3） 说明node节点无法找到 fly-k8s，查看kubelet配置的ip

cat /etc/kubernetes/kubelet.conf

（4）查看宿主机ip

ifconfig

（5）两者一致，说明ip是正确的，这说明宿主机的/ets/hosts 配置有问题。如果ip不一致，则需要修改kubelet、kubeadm、api-server对应的ip

cat /etc/hosts

通过命令vim /etc/hosts 修改为 fly-k8s

（6）检测kubectl是否可以连接api-server

root@fly-virtual-machine:/usr/local/k8s/test# kubectl get node
NAME                  STATUS   ROLES                  AGE   VERSION
fly-k8s   Ready    control-plane,master   64m   v1.22.2

本文标签： 节点Kubernetes