k8s內網安裝部署（二）
阿新 • • 發佈：2020-11-26
續上篇
https://www.cnblogs.com/wangql/p/13397034.html
一、kubeadm安裝
1.kube-proxy開啟ipvs的前置條件
modprobe br_netfilter //載入net filter模組 cat > /etc/sysconfig/modules/ipvs.modules <<EOF #!/bin/bash modprobe -- ip_vs modprobe -- ip_vs_rr modprobe -- ip_vs_wrr modprobe -- ip_vs_sh modprobe -- nf_conntrack_ipv4 EOF chmod755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack_ipv4
2.安裝docker軟體
下載地址：https://mirrors.aliyun.com/docker-ce/linux/centos/7/x86_64/test/Packages/
docker-ce-17.03.3.ce-1.el7.x86_64.rpm
docker-ce-selinux-17.03.3.ce-1.el7.noarch.rpm
yum install -y yum-utils device-mapper-persistent-data lvm2 bind-utils yum -y install docker-ce ## 建立 /etc/docker 目錄 mkdir /etc/docker # 配置 daemon. 加速（內網可以不配置，只配置自己私有倉庫） cat > /etc/docker/daemon.json <<EOF { "insecure-registries":["192.168.4.88:5000"] #這裡我用的是內網的倉庫 } EOF mkdir -p /etc/systemd/system/docker.service.d # 重啟docker服務 systemctl daemon-reload && systemctl restart docker && systemctl enable docker
聯網安裝方法：
yum install -y yum-utils device-mapper-persistent-data lvm2 yum-config-manager \ --add-repo \ http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo //匯入阿里映象倉庫 yum update -y && yum install -y docker-ce ## 建立 /etc/docker 目錄 mkdir /etc/docker # 配置 daemon. cat > /etc/docker/daemon.json <<EOF { "exec-opts": ["native.cgroupdriver=systemd"], "log-driver": "json-file", "log-opts": { "max-size": "100m" } } EOF mkdir -p /etc/systemd/system/docker.service.d # 重啟docker服務 systemctl daemon-reload && systemctl restart docker && systemctl enable docker
重啟一下系統看看核心有沒有變
3.安裝 Kubeadm （主從配置）
把包做成yum源
yum -y install kubeadm-1.15.1 kubectl-1.15.1 kubelet-1.15.1 systemctl enable kubelet.service
解壓映象
tar -xvf kubeadm-basic.images.tar.gz #需要安裝包的話可在我的公眾號【大隆愛分享】獲取
4.初始化主節點
注：叢集初始化如果遇到問題，可以使用下面的命令進行清理：
kubeadm reset
1】配置私有倉庫地址
[root@k8s-master01 flannel]# cat /etc/docker/daemon.json { "exec-opts": ["native.cgroupdriver=systemd"], "log-driver": "json-file", "log-opts": { "max-size": "100m" }, "insecure-registries":["192.168.4.88:5000"] //加自己的私有倉庫地址 }
初始化主機點（只需要主做）
kubeadm config print init-defaults > kubeadm-config.yaml vim kubeadm-config.yaml 12 advertiseAddress: 192.168.4.10 //當前伺服器節點地址 32 imageRepository: 192.168.4.88:5000 //自己私有倉庫地址 34 kubernetesVersion: v1.15.1 //版本號 36 dnsDomain: cluster.local 37 podSubnet: "10.244.0.0/16" //新增這一行pod的網段 38 serviceSubnet: 10.96.0.0/12 //預設即可 --- //新增下面的，預設把排程方式改為IP VS apiVersion: kubeproxy.config.k8s.io/v1alpha1 kind: KubeProxyConfiguration featureGates: SupportIPVSProxyMode: true mode: ipvs
kubeadm init --config=kubeadm-config.yaml --experimental-upload-certs | tee kubeadm-init.log //指定yaml檔案以及頒發證書 把所有資訊都寫到kubeadm-init.log中 ............ ........... Your Kubernetes control-plane has initialized successfully! //代表初始化成功 To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes by running the following on each as root: kubeadm join 192.168.4.10:6443 --token abcdef.0123456789abcdef \ --discovery-token-ca-cert-hash sha256:bb6ae2db244800ce95a72e47e715a01dbc1aa712d0fec5a252e572b5a33cd083
執行
cd /etc/kubernetes/pki/ mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config //拷貝叢集管理員的配置檔案 sudo chown $(id -u):$(id -g) $HOME/.kube/config //授權 當前屬主屬者 [root@k8s-master01 ~]# kubectl get node //檢視當前節點 NAME STATUS ROLES AGE VERSION k8s-master01 NotReady master 4m37s v1.15.1
5.部署網路
mkdir install-k8s mv kubeadm-config.yaml kubeadm-init.log install-k8s/ //把重要檔案放到這個裡面 cd install-k8s/ mkdir core mv kubeadm-* core/ mkdir plugin cd plugin/ mkdir flannel cd flannel/
下載地址： wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml [root@k8s-master01 flannel]# vim kube-flannel.yml 172 image: 192.168.4.88:5000/flannel:v1 //映象地址 186 image: 192.168.4.88:5000/flannel:v1 //裡面的全都要改 192 - --iface=eth0 //指定網絡卡，都要改 建立flannel.yml kubectl apply -f kube-flannel.yml 都是Running說明成功 [root@k8s-master01 flannel]# kubectl get pod -n kube-system NAME READY STATUS RESTARTS AGE coredns-6f5f787f5b-cch5j 1/1 Running 0 15m coredns-6f5f787f5b-fscnt 1/1 Running 0 15m etcd-k8s-master01 1/1 Running 0 15m kube-apiserver-k8s-master01 1/1 Running 0 14m kube-controller-manager-k8s-master01 1/1 Running 0 15m kube-flannel-ds-amd64-q4hnk 1/1 Running 0 10m kube-proxy-pfhj2 1/1 Running 0 15m kube-scheduler-k8s-master01 1/1 Running 0 15m [root@k8s-master01 flannel]# kubectl get node NAME STATUS ROLES AGE VERSION k8s-master01 Ready master 17m v1.15.1
6. node節點加入
日誌最後一行在從節點執行即可
在這個檔案裡 kubeadm-init.log
kubeadm join 192.168.4.10:6443 --token abcdef.0123456789abcdef \ --discovery-token-ca-cert-hash sha256:bb6ae2db244800ce95a72e47e715a01dbc1aa712d0fec5a252e572b5a33cd083
7.節點下載方法
kubeadm config print init-defaults >kubeadm.conf 將配置檔案的imageRepository: 修改為自己的私有倉 imageRepository: docker.emarbox.com/google_containers kubernetesVersion 改為自有版本 kubernetesVersion: v1.15.1 kubeadm config images list --config kubeadm.conf kubeadm config images pull --config kubeadm.conf
8.節點操作
下載映象：這些映象在我的映象倉庫裡 docker pull 192.168.4.88:5000/flannel:v1 docker pull 192.168.4.88:5000/pause:3.1 docker pull 192.168.4.88:5000/kube-proxy:v1.15.1
9.報錯解決
報錯資訊
error execution phase preflight: couldn't validate the identity of the API Server: abort connecting to API servers after timeout of 5m0s
報錯原因： 與API伺服器認證失敗，八成token失效了， 檢視token kubeadm token list 建立token kubeadm token create kubeadm token list openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //' kubeadm join 192.168.4.10:6443 --token abcdef.0123456789abcdef \ 把這個token換掉 --discovery-token-ca-cert-hash sha256:eb1e1a3ce9e819ebafdf73b8a4819e2e40d9da6dfdb0272a4ab1925be3fc12f3 //重新加入試試
node節點不能檢視
[root@k8s-node02 ~]# kubectl get node The connection to the server localhost:8080 was refused - did you specify the right host or port? 將主節點（master節點）中的【/etc/kubernetes/admin.conf】檔案拷貝到從節點相同目錄下: scp /etc/kubernetes/admin.conf 192.168.4.63:/etc/kubernetes/. node上 echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> ~/.bash_profile source ~/.bash_profile
二、移除節點
Master上：
[root@k8s-master01 ~]# kubectl drain k8s-node02 --delete-local-data --force --ignore-daemonsets node/k8s-node02 cordoned WARNING: ignoring DaemonSet-managed Pods: kube-system/kube-flannel-ds-amd64-l4j57, kube-system/kube-proxy-9d9nv node/k8s-node02 drained [root@k8s-master01 ~]# kubectl delete node k8s-node02 node "k8s-node02" deleted [root@k8s-master01 ~]# kubectl get node NAME STATUS ROLES AGE VERSION k8s-master01 Ready master 4d19h v1.15.1 k8s-node01 Ready <none> 5m49s v1.15.1
加回來：
[root@k8s-node02 docker.service.d]# systemctl stop kubelet [root@k8s-node02 docker.service.d]# rm -rf /etc/kubernetes/* [root@k8s-node02 docker.service.d]# kubeadm join 192.168.4.10:6443 --token v2xaat.qip3csxdge8vicxj --discovery-token-ca-cert-hash sha256:eb1e1a3ce9e819ebafdf73b8a4819e2e40d9da6dfdb0272a4ab1925be3fc12f3 [root@k8s-node02 docker.service.d]# kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master01 Ready master 4d19h v1.15.1 k8s-node01 Ready <none> 21m v1.15.1 k8s-node02 Ready <none> 18s v1.15.1
還有好多沒來的及整理，會慢慢更新，歡迎點贊關注。