k8s---apiserver，controller-manager,scheduler部署
阿新 • • 發佈：2019-03-19
std str cor app 可用 eof cd-k enc gin
#證書自簽名腳本 root@k8s-master: ~/k8s/k8s-cert 14:06:06 $ cat k8s-cert.sh cat > ca-config.json <<EOF { "signing": { "default": { "expiry": "87600h" }, "profiles": { "kubernetes": { "expiry": "87600h", "usages": [ "signing","key encipherment", "server auth", "client auth" ] } } } } EOF cat > ca-csr.json <<EOF { "CN": "kubernetes", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN","L": "Beijing", "ST": "Beijing", "O": "k8s", "OU": "System" } ] } EOF cfssl gencert -initca ca-csr.json | cfssljson -bare ca - #----------------------- #hosts內容slb節點ip，master節點ip，下邊是node節點ip（node節點多寫一寫冗余IP地址為後續使用） cat > server-csr.json <<EOF {"CN": "kubernetes", "hosts": [ "10.0.0.1", "127.0.0.1", "192.168.1.63", "192.168.1.64", "192.168.1.65", "192.168.1.66", "192.168.1.60", "192.168.1.61", "192.168.1.62", "kubernetes", "kubernetes.default", "kubernetes.default.svc", "kubernetes.default.svc.cluster", "kubernetes.default.svc.cluster.local" ], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "BeiJing", "ST": "BeiJing", "O": "k8s", "OU": "System" } ] } EOF cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes server-csr.json | cfssljson -bare server #----------------------- cat > admin-csr.json <<EOF { "CN": "admin", "hosts": [], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "BeiJing", "ST": "BeiJing", "O": "system:masters", "OU": "System" } ] } EOF cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin #----------------------- cat > kube-proxy-csr.json <<EOF { "CN": "system:kube-proxy", "hosts": [], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "BeiJing", "ST": "BeiJing", "O": "k8s", "OU": "System" } ] } EOF cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy root@k8s-master: ~/k8s/k8s-cert 14:06:12 $ 1. kube-apiserver 2. kube-controller-manager 3. kube-scheduler 配置文件 -> systemd管理組件 -> 啟動 ==================================================kube-apiserver==================================================
# 創建 TLS Bootstrapping Token
#BOOTSTRAP_TOKEN=$(head -c 16 /dev/urandom | od -An -t x | tr -d ‘ ‘)
BOOTSTRAP_TOKEN=0fb61c46f8991b718eb38d27b605b008
cat > token.csv <<EOF
${BOOTSTRAP_TOKEN},kubelet-bootstrap,10001,"system:kubelet-bootstrap"
EOF
mv token.csv /opt/kubernetes/cfg
2，部署kube-apiserver (1)創建apiserver的文件存放目錄 root@k8s-master: ~/soft 14:09:51 $ mkdir /opt/kubernetes/{bin,ssl,cfg} -p (2)解壓tar包，將核心組件復制到/opt/kubernetes/bin下kube-apiserver，kube-controller-manager，kube-scheduler root@k8s-master: ~/soft 14:11:12 $ tar zxvf kubernetes-server-linux-amd64.tar.gz root@k8s-master: ~ 14:17:18 $ cd /root/soft/kubernetes/server/bin root@k8s-master: ~/soft/kubernetes/server/bin 14:19:23 $ cp kube-apiserver kube-controller-manager kube-scheduler /opt/kubernetes/bin/ (3)kube-apiserver配置文件腳本 root@k8s-master: ~/k8s 14:21:49 $ cat apiserver.sh #!/bin/bash #master主機節點ip地址，傳入變量 MASTER_ADDRESS=$1 #etcd所有節點ip地址 ETCD_SERVERS=$2 cat <<EOF >/opt/kubernetes/cfg/kube-apiserver #true開啟日誌默認寫到/var/log/messages,第二選項flase，並在下邊指定log寫入目錄--logs-dir=/opt/kubernetes/logs KUBE_APISERVER_OPTS="--logtostderr=true \\ #日誌登記，登記越高日誌越少 --v=4 \--etcd-servers=${ETCD_SERVERS} \--bind-address=${MASTER_ADDRESS} \--secure-port=6443 \--advertise-address=${MASTER_ADDRESS} \--allow-privileged=true \#負載均衡節點ip範文，下邊是端口 --service-cluster-ip-range=10.0.0.0/24 \--service-node-port-range=30000-50000 \--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction \--authorization-mode=RBAC,Node \--kubelet-https=true \#互相通信的token，身份認證標識 --enable-bootstrap-token-auth \--token-auth-file=/opt/kubernetes/cfg/token.csv \#apiserver的ssl自簽名證書 --tls-cert-file=/opt/kubernetes/ssl/server.pem \--tls-private-key-file=/opt/kubernetes/ssl/server-key.pem \--client-ca-file=/opt/kubernetes/ssl/ca.pem \--service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \#下邊是etcd的ssl自簽名證書。因為都是https --etcd-cafile=/opt/etcd/ssl/ca.pem \--etcd-certfile=/opt/etcd/ssl/server.pem \--etcd-keyfile=/opt/etcd/ssl/server-key.pem" EOF #配置systemctl管理apiserver cat <<EOF >/usr/lib/systemd/system/kube-apiserver.service [Unit] Description=Kubernetes API Server Documentation=https://github.com/kubernetes/kubernetes [Service] EnvironmentFile=-/opt/kubernetes/cfg/kube-apiserver ExecStart=/opt/kubernetes/bin/kube-apiserver \$KUBE_APISERVER_OPTS Restart=on-failure [Install] WantedBy=multi-user.target EOF systemctl daemon-reload systemctl enable kube-apiserver systemctl restart kube-apiserver root@k8s-master: ~/k8s 14:21:51 $ (4)將apiserver自簽名證書移動到/opt/kubernetes/ssl root@k8s-master: /opt/kubernetes/ssl 15:11:40 $ pwd /opt/kubernetes/ssl root@k8s-master: /opt/kubernetes/ssl 15:11:42 $ ls ca-key.pem ca.pem server-key.pem server.pem root@k8s-master: /opt/kubernetes/ssl 15:11:42 $ (5)復制apiserver自簽名證書到/opt/kubernetes/ssl 執行腳本 root@k8s-master: ~/k8s 14:47:17 $ ./apiserver.sh 192.168.1.63 https://192.168.1.63:2379,https://192.168.1.65:2379,https://192.168.1.66:2379 (6)驗證apiserver是否啟動成功 root@k8s-master: /opt/kubernetes/ssl 15:12:50 $ netstat -lntup |grep 8080 tcp 0 0 127.0.0.1:8080 0.0.0.0:* LISTEN 3972/kube-apiserver root@k8s-master: /opt/kubernetes/ssl 15:12:52 root@k8s-master: /opt/kubernetes/ssl 15:13:21 $ ps -ef|grep kube root 3972 1 1 11:57 ? 00:03:14 /opt/kubernetes/bin/kube-apiserver --logtostderr=true --v=4 --etcd-servers=https://192.168.1.63:2379,https://192.168.1.65:2379,https://192.168.1.66:2379 --bind-address=192.168.1.63 --secure-port=6443 --advertise-address=192.168.1.63 --allow-privileged=true --service-cluster-ip-range=10.0.0.0/24 --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction --authorization-mode=RBAC,Node --kubelet-https=true --enable-bootstrap-token-auth --token-auth-file=/opt/kubernetes/cfg/token.csv --service-node-port-range=30000-50000 --tls-cert-file=/opt/kubernetes/ssl/server.pem --tls-private-key-file=/opt/kubernetes/ssl/server-key.pem --client-ca-file=/opt/kubernetes/ssl/ca.pem --service-account-key-file=/opt/kubernetes/ssl/ca-key.pem --etcd-cafile=/opt/etcd/ssl/ca.pem --etcd-certfile=/opt/etcd/ssl/server.pem --etcd-keyfile=/opt/etcd/ssl/server-key.pem root 4091 1 0 12:07 ? 00:01:00 /opt/kubernetes/bin/kube-scheduler --logtostderr=true --v=4 --master=127.0.0.1:8080 --leader-elect root 4096 1 0 12:07 ? 00:01:32 /opt/kubernetes/bin/kube-controller-manager --logtostderr=true --v=4 --master=127.0.0.1:8080 --leader-elect=true --address=127.0.0.1 --service-cluster-ip-range=10.0.0.0/24 --cluster-name=kubernetes --cluster-signing-cert-file=/opt/kubernetes/ssl/ca.pem --cluster-signing-key-file=/opt/kubernetes/ssl/ca-key.pem --root-ca-file=/opt/kubernetes/ssl/ca.pem --service-account-private-key-file=/opt/kubernetes/ssl/ca-key.pem --experimental-cluster-signing-duration=87600h0m0s root 5899 5763 0 15:13 pts/0 00:00:00 grep --color=auto kube root@k8s-master: /opt/kubernetes/ssl 15:13:26 $ ####報錯排查方式 $ source /opt/kubernetes/cfg/kube-apiserver $ /opt/kubernetes/bin/kube-apiserver $KUBE_APISERVER_OPTS ==================================================kube-controller-manager================================================== root@k8s-master: ~/k8s 15:15:44 $ cat controller-manager.sh #!/bin/bash #傳參master節點ip地址 MASTER_ADDRESS=$1 cat <<EOF >/opt/kubernetes/cfg/kube-controller-manager KUBE_CONTROLLER_MANAGER_OPTS="--logtostderr=true \\ --v=4 \#master-apiserver運行端口8080所有引用傳參變量 --master=${MASTER_ADDRESS}:8080 \#選舉,自動做高可用 --leader-elect=true \#這個服務只在本地運行所以能跟apiserver通信就可以了 --address=127.0.0.1 \--service-cluster-ip-range=10.0.0.0/24 \--cluster-name=kubernetes \#頒發證書 --cluster-signing-cert-file=/opt/kubernetes/ssl/ca.pem \--cluster-signing-key-file=/opt/kubernetes/ssl/ca-key.pem \--root-ca-file=/opt/kubernetes/ssl/ca.pem \--service-account-private-key-file=/opt/kubernetes/ssl/ca-key.pem \--experimental-cluster-signing-duration=87600h0m0s" EOF ##使用systemctl管理controller工具 cat <<EOF >/usr/lib/systemd/system/kube-controller-manager.service [Unit] Description=Kubernetes Controller Manager Documentation=https://github.com/kubernetes/kubernetes [Service] EnvironmentFile=-/opt/kubernetes/cfg/kube-controller-manager ExecStart=/opt/kubernetes/bin/kube-controller-manager \$KUBE_CONTROLLER_MANAGER_OPTS Restart=on-failure [Install] WantedBy=multi-user.target EOF systemctl daemon-reload systemctl enable kube-controller-manager systemctl restart kube-controller-manager root@k8s-master: ~/k8s 15:15:46 $ ==================================================kube-scheduler================================================== root@k8s-master: ~/k8s 15:46:04 $ cat scheduler.sh #!/bin/bash MASTER_ADDRESS=$1 ##scheduler四行，定義日誌，指定masterip，自動選舉 cat <<EOF >/opt/kubernetes/cfg/kube-scheduler KUBE_SCHEDULER_OPTS="--logtostderr=true \\ --v=4 \--master=${MASTER_ADDRESS}:8080 \--leader-elect" EOF cat <<EOF >/usr/lib/systemd/system/kube-scheduler.service [Unit] Description=Kubernetes Scheduler Documentation=https://github.com/kubernetes/kubernetes [Service] EnvironmentFile=-/opt/kubernetes/cfg/kube-scheduler ExecStart=/opt/kubernetes/bin/kube-scheduler \$KUBE_SCHEDULER_OPTS Restart=on-failure [Install] WantedBy=multi-user.target EOF systemctl daemon-reload systemctl enable kube-scheduler systemctl restart kube-scheduler ==================================================kubectl================================================== 部署完成驗證 root@k8s-master: ~/k8s 15:47:29 $ cp /root/soft/kubernetes/server/bin/kubectl /usr/bin/ #檢查當前集群節點的健康狀態 $ kubectl get cs NAME STATUS MESSAGE ERROR scheduler Healthy ok controller-manager Healthy ok etcd-0 Healthy {"health":"true"} etcd-2 Healthy {"health":"true"} etcd-1 Healthy {"health":"true"} root@k8s-master: ~/k8s 15:48:14 $ ###ps：cs為縮寫
k8s---apiserver，controller-manager,scheduler部署