2. 程式人生 > >kubernetes分散式安裝部署-簡介

kubernetes分散式安裝部署-簡介

阿新 • • 發佈：2018-11-16

環境說明：3臺1C1G（我自己這樣的配置，有條件的可以調整大點）虛擬機器且操作是系統centos7系統，三臺虛擬機器IP地址為：

1）192.168.56.11 主機名字：linux-node1.example.com

2）192.168.56.12 主機名字：linux-node2.example.com

3）192.168.56.13 主機名字：linux-node3.example.com

開始安裝作業系統前：net.ifnames=0 biosdevname=0（目的是安裝好的作業系統網絡卡名字為：eth0）

設定網路設定：
 vi/etc/sysconfig/network-scripts/ifcfg-eth0

Systemctl disable firewalld

Systemctl disable networkmanager

Vi /etc/hostnames名字改成：linux-node1.example.com 、linux-node2.example.com、linux-node3.example.com

Vi/etc/selinux/config  內容改成：disable

更改dns：vi/etc/resol

nameserver 192.168.56.2 

免金鑰登入（三臺虛擬機器做互信）：

ssh-keygen –t rsa

Ssh-copy-id linux-node1

 

系統環境初始化

1.安裝Docker

第一步：使用國內Docker源

[[email protected] ~]# cd /etc/yum.repos.d/

[[email protected] yum.repos.d]# wget \

 https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

第二步：Docker安裝：

[[email protected] ~]# yum install -y docker-ce

第三步：啟動後臺程序：

[[email protected]

~]# systemctl start docker

2.準備部署目錄

    mkdir -p /opt/kubernetes/{cfg,bin,ssl,log}

3.準備軟體包

百度網盤下載地址：

[https://pan.baidu.com/s/1zs8sCouDeCQJ9lghH1BPiw](https://pan.baidu.com/s/1zs8sCouDeCQJ9lghH1BPiw)

4.解壓軟體包

 # tar zxf kubernetes.tar.gz

 # tar zxf kubernetes-server-linux-amd64.tar.gz

 # tar zxf kubernetes-client-linux-amd64.tar.gz

 # tar zxf kubernetes-node-linux-amd64.tar.gz

手動製作CA證書

1.安裝 CFSSL

[[email protected] ~]# cd /usr/local/src

[[email protected] src]# wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64

[[email protected] src]# wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64

[[email protected] src]# wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64

[[email protected] src]# chmod +x cfssl*

[[email protected] src]# mv cfssl-certinfo_linux-amd64 /opt/kubernetes/bin/cfssl-certinfo

[[email protected] src]# mv cfssljson_linux-amd64  /opt/kubernetes/bin/cfssljson

[[email protected] src]# mv cfssl_linux-amd64  /opt/kubernetes/bin/cfssl

複製cfssl命令檔案到k8s-node1和k8s-node2節點。如果實際中多個節點，就都需要同步複製。

[[email protected] ~]# scp /opt/kubernetes/bin/cfssl* 192.168.56.12: /opt/kubernetes/bin

[[email protected] ~]# scp /opt/kubernetes/bin/cfssl* 192.168.56.13: /opt/kubernetes/bin

2.初始化cfssl

[[email protected] src]# mkdir ssl && cd ssl

[[email protected] ssl]# cfssl print-defaults config > config.json

[[email protected] ssl]# cfssl print-defaults csr > csr.json

3.建立用來生成 CA 檔案的 JSON 配置檔案

[[email protected] ssl]# vim ca-config.json

{
  "signing": {
    "default": {
      "expiry": "8760h"
    },
    "profiles": {
      "kubernetes": {
        "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ],

        "expiry": "8760h"
      }
    }
  }
}

4.建立用來生成 CA 證書籤名請求（CSR）的 JSON 配置檔案

[[email protected] ssl]# vim ca-csr.json

{"CN": "kubernetes",
  "key": { "algo": "rsa",
    "size": 2048 },
"names": [ {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]

}

5.生成CA證書（ca.pem）和金鑰（ca-key.pem）

[[email protected] linux-node1 ssl]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca

[[email protected] linux-node1 ssl]# ls -l ca*

-rw-r--r-- 1 root root  290 Mar  4 13:45 ca-config.json

-rw-r--r-- 1 root root 1001 Mar  4 14:09 ca.csr

-rw-r--r-- 1 root root  208 Mar  4 13:51 ca-csr.json

-rw------- 1 root root 1679 Mar  4 14:09 ca-key.pem

-rw-r--r-- 1 root root 1359 Mar  4 14:09 ca.pem

6.分發證書

# cp ca.csr ca.pem ca-key.pem ca-config.json /opt/kubernetes/ssl

SCP證書到k8s-node1和k8s-node2節點

# scp ca.csr ca.pem ca-key.pem ca-config.json 192.168.56.12:/opt/kubernetes/ssl

# scp ca.csr ca.pem ca-key.pem ca-config.json 192.168.56.13:/opt/kubernetes/ssl

手動部署ETCD叢集

0.準備etcd軟體包

wget https://github.com/coreos/etcd/releases/download/v3.2.18/etcd-v3.2.18-linux-amd64.tar.gz

[[email protected] src]# tar zxf etcd-v3.2.18-linux-amd64.tar.gz

[[email protected] src]# cd etcd-v3.2.18-linux-amd64

[[email protected] etcd-v3.2.18-linux-amd64]# cp etcd etcdctl /opt/kubernetes/bin/

[[email protected] etcd-v3.2.18-linux-amd64]# scp etcd etcdctl 192.168.56.12:/opt/kubernetes/bin/

[[email protected] etcd-v3.2.18-linux-amd64]# scp etcd etcdctl 192.168.56.13:/opt/kubernetes/bin/

1.建立 etcd 證書籤名請求：

 [[email protected] ~]# vim etcd-csr.json

{
  "CN": "etcd",
  "hosts": [
    "127.0.0.1",
"192.168.56.11",
"192.168.56.12",
"192.168.56.13"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}

2.生成 etcd 證書和私鑰：

[[email protected] ~]# cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \

  -ca-key=/opt/kubernetes/ssl/ca-key.pem \

  -config=/opt/kubernetes/ssl/ca-config.json \

  -profile=kubernetes etcd-csr.json | cfssljson -bare etcd會生成以下證書檔案

[[email protected] ~]# ls -l etcd*

-rw-r--r-- 1 root root 1045 Mar  5 11:27 etcd.csr

-rw-r--r-- 1 root root  257 Mar  5 11:25 etcd-csr.json

-rw------- 1 root root 1679 Mar  5 11:27 etcd-key.pem

-rw-r--r-- 1 root root 1419 Mar  5 11:27 etcd.pem

3.將證書移動到/opt/kubernetes/ssl目錄下

[[email protected] ~]# cp etcd*.pem /opt/kubernetes/ssl

[[email protected] ~]# scp etcd*.pem 192.168.56.12:/opt/kubernetes/ssl

[[email protected] ~]# scp etcd*.pem 192.168.56.13:/opt/kubernetes/ssl

[[email protected] ~]# rm -f etcd.csr etcd-csr.json

4.設定ETCD配置檔案

[[email protected] ~]# vim /opt/kubernetes/cfg/etcd.conf

#[member]
ETCD_NAME="etcd-node1"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
#ETCD_SNAPSHOT_COUNTER="10000"
#ETCD_HEARTBEAT_INTERVAL="100"
#ETCD_ELECTION_TIMEOUT="1000"
ETCD_LISTEN_PEER_URLS="https://192.168.56.11:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.56.11:2379,https://127.0.0.1:2379"
#ETCD_MAX_SNAPSHOTS="5"
#ETCD_MAX_WALS="5"
#ETCD_CORS=""
#[cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.56.11:2380"
# if you use different ETCD_NAME (e.g. test),
# set ETCD_INITIAL_CLUSTER value for this name, i.e. "test=http://..."
ETCD_INITIAL_CLUSTER="etcd-node1=https://192.168.56.11:2380,etcd-node2=https://192.168.56.12:2380,etcd-node3=https://192.168.56.13:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="k8s-etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.56.11:2379"
#[security]
CLIENT_CERT_AUTH="true"
ETCD_CA_FILE="/opt/kubernetes/ssl/ca.pem"
ETCD_CERT_FILE="/opt/kubernetes/ssl/etcd.pem"
ETCD_KEY_FILE="/opt/kubernetes/ssl/etcd-key.pem"
PEER_CLIENT_CERT_AUTH="true"
ETCD_PEER_CA_FILE="/opt/kubernetes/ssl/ca.pem"
ETCD_PEER_CERT_FILE="/opt/kubernetes/ssl/etcd.pem"
ETCD_PEER_KEY_FILE="/opt/kubernetes/ssl/etcd-key.pem"

5.建立ETCD系統服務

[[email protected] ~]# vim /etc/systemd/system/etcd.service

[Unit]
Description=Etcd Server
After=network.target
[Service]
Type=simple
WorkingDirectory=/var/lib/etcd
EnvironmentFile=-/opt/kubernetes/cfg/etcd.conf
# set GOMAXPROCS to number of processors
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /opt/kubernetes/bin/etcd"
Type=notify
[Install]
WantedBy=multi-user.target

6.重新載入系統服務

[[email protected] ~]# systemctl daemon-reload

[[email protected] ~]# systemctl enable etcd

# scp /opt/kubernetes/cfg/etcd.conf 192.168.56.12:/opt/kubernetes/cfg/

# scp /etc/systemd/system/etcd.service 192.168.56.12:/etc/systemd/system/

# scp /opt/kubernetes/cfg/etcd.conf 192.168.56.13:/opt/kubernetes/cfg/

# scp /etc/systemd/system/etcd.service 192.168.56.13:/etc/systemd/system/

在所有節點上建立etcd

儲存目錄並啟動etcd

[[email protected] ~]# mkdir /var/lib/etcd

[[email protected] ~]# systemctl start etcd

[[email protected] ~]# systemctl status etcd

下面需要大家在所有的 etcd 節點重複上面的步驟，直到所有機器的 etcd 服務都已啟動。

7.驗證叢集

[[email protected] ~]# etcdctl --endpoints=https://192.168.56.11:2379 \

  --ca-file=/opt/kubernetes/ssl/ca.pem \

  --cert-file=/opt/kubernetes/ssl/etcd.pem \

  --key-file=/opt/kubernetes/ssl/etcd-key.pem cluster-health

member 435fb0a8da627a4c is healthy: got healthy result from https://192.168.56.12:2379

member 6566e06d7343e1bb is healthy: got healthy result from https://192.168.56.11:2379

member ce7b884e428b6c8c is healthy: got healthy result from https://192.168.56.13:2379

cluster is healthy

部署Kubernetes API服務部署

0.準備軟體包

[[email protected] ~]# cd /usr/local/src/kubernetes

[[email protected] kubernetes]# cp server/bin/kube-apiserver /opt/kubernetes/bin/

[[email protected] kubernetes]# cp server/bin/kube-controller-manager /opt/kubernetes/bin/

[[email protected] kubernetes]# cp server/bin/kube-scheduler /opt/kubernetes/bin/

1.建立生成CSR的 JSON 配置檔案

[[email protected] src]# vim kubernetes-csr.json

{
  "CN": "kubernetes",
  "hosts": [
    "127.0.0.1",
    "192.168.56.11",
    "10.1.0.1",
    "kubernetes",
    "kubernetes.default",
    "kubernetes.default.svc",
    "kubernetes.default.svc.cluster",
    "kubernetes.default.svc.cluster.local"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}

2.生成 kubernetes 證書和私鑰

 [[email protected] src]# cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \

   -ca-key=/opt/kubernetes/ssl/ca-key.pem \

   -config=/opt/kubernetes/ssl/ca-config.json \

   -profile=kubernetes kubernetes-csr.json | cfssljson -bare kubernetes

[[email protected] src]# cp kubernetes*.pem /opt/kubernetes/ssl/

[[email protected] ~]# scp kubernetes*.pem 192.168.56.12:/opt/kubernetes/ssl/

[[email protected] ~]# scp kubernetes*.pem 192.168.56.13:/opt/kubernetes/ssl/

3.建立 kube-apiserver 使用的客戶端 token 檔案

[[email protected] ~]#  head -c 16 /dev/urandom | od -An -t x | tr -d ' '

ad6d5bb607a186796d8861557df0d17f

[[email protected] ~]# vim /opt/kubernetes/ssl/ bootstrap-token.csv

ad6d5bb607a186796d8861557df0d17f,kubelet-bootstrap,10001,"system:kubelet-bootstrap"

4.建立基礎使用者名稱/密碼認證配置

[[email protected] ~]# vim /opt/kubernetes/ssl/basic-auth.csv

admin,admin,1

readonly,readonly,2

5.部署Kubernetes API Server

[[email protected] ~]# vim /usr/lib/systemd/system/kube-apiserver.service

[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target
[Service]
ExecStart=/opt/kubernetes/bin/kube-apiserver \
  --admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota,NodeRestriction \  --bind-address=192.168.56.11 \
  --insecure-bind-address=127.0.0.1 \
  --authorization-mode=Node,RBAC \
  --runtime-config=rbac.authorization.k8s.io/v1 \
  --kubelet-https=true \
  --anonymous-auth=false \
  --basic-auth-file=/opt/kubernetes/ssl/basic-auth.csv \
  --enable-bootstrap-token-auth \
  --token-auth-file=/opt/kubernetes/ssl/bootstrap-token.csv \
  --service-cluster-ip-range=10.1.0.0/16 \
  --service-node-port-range=20000-40000 \
  --tls-cert-file=/opt/kubernetes/ssl/kubernetes.pem \
  --tls-private-key-file=/opt/kubernetes/ssl/kubernetes-key.pem \
  --client-ca-file=/opt/kubernetes/ssl/ca.pem \
  --service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \
  --etcd-cafile=/opt/kubernetes/ssl/ca.pem \
  --etcd-certfile=/opt/kubernetes/ssl/kubernetes.pem \
  --etcd-keyfile=/opt/kubernetes/ssl/kubernetes-key.pem \
  --etcd-servers=https://192.168.56.11:2379,https://192.168.56.12:2379,https://192.168.56.13:2379 \
  --enable-swagger-ui=true \
  --allow-privileged=true \
  --audit-log-maxage=30 \
  --audit-log-maxbackup=3 \
  --audit-log-maxsize=100 \
  --audit-log-path=/opt/kubernetes/log/api-audit.log \
  --event-ttl=1h \
  --v=2 \
  --logtostderr=false \
  --log-dir=/opt/kubernetes/log
Restart=on-failure
RestartSec=5
Type=notify
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target

6.啟動API Server服務

[[email protected] ~]# systemctl daemon-reload

[[email protected] ~]# systemctl enable kube-apiserver

[[email protected] ~]# systemctl start kube-apiserver

檢視API Server服務狀態

[[email protected] ~]# systemctl status kube-apiserver

部署Controller Manager服務

[[email protected] ~]# vim /usr/lib/systemd/system/kube-controller-manager.service

[Unit]

Description=Kubernetes Controller Manager

Documentation=https://github.com/GoogleCloudPlatform/kubernetes

[Service]

ExecStart=/opt/kubernetes/bin/kube-controller-manager \

  --address=127.0.0.1 \

  --master=http://127.0.0.1:8080 \

  --allocate-node-cidrs=true \

  --service-cluster-ip-range=10.1.0.0/16 \

  --cluster-cidr=10.2.0.0/16 \

  --cluster-name=kubernetes \

  --cluster-signing-cert-file=/opt/kubernetes/ssl/ca.pem \

  --cluster-signing-key-file=/opt/kubernetes/ssl/ca-key.pem \

  --service-account-private-key-file=/opt/kubernetes/ssl/ca-key.pem \

  --root-ca-file=/opt/kubernetes/ssl/ca.pem \

  --leader-elect=true \

  --v=2 \

  --logtostderr=false \

  --log-dir=/opt/kubernetes/log

Restart=on-failure

RestartSec=5

[Install]

WantedBy=multi-user.target

3.啟動Controller Manager

[[email protected] ~]# systemctl daemon-reload

[[email protected] scripts]# systemctl enable kube-controller-manager

[[email protected] scripts]# systemctl start kube-controller-manager

4.檢視服務狀態

[[email protected] scripts]# systemctl status kube-controller-manager

部署Kubernetes Scheduler

[[email protected] ~]# vim /usr/lib/systemd/system/kube-scheduler.service

[Unit]

Description=Kubernetes Scheduler

Documentation=https://github.com/GoogleCloudPlatform/kubernetes

[Service]

ExecStart=/opt/kubernetes/bin/kube-scheduler \

  --address=127.0.0.1 \

  --master=http://127.0.0.1:8080 \

  --leader-elect=true \

  --v=2 \

  --logtostderr=false \

  --log-dir=/opt/kubernetes/log

Restart=on-failure

RestartSec=5

[Install]

WantedBy=multi-user.target

2.部署服務

[[email protected] ~]# systemctl daemon-reload

[[email protected] scripts]# systemctl enable kube-scheduler

[[email protected] scripts]# systemctl start kube-scheduler

[[email protected] scripts]# systemctl status kube-scheduler

部署kubectl 命令列工具

1.準備二進位制命令包

[[email protected] ~]# cd /usr/local/src/kubernetes/client/bin

[[email protected] bin]# cp kubectl /opt/kubernetes/bin/

2.建立 admin 證書籤名請求

[[email protected] ~]# cd /usr/local/src/ssl/

[[email protected] ssl]# vim admin-csr.json

{
  "CN": "admin",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "system:masters",
      "OU": "System"
    }
  ]
}

3.生成 admin 證書和私鑰：

[[email protected] ssl]# cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \

   -ca-key=/opt/kubernetes/ssl/ca-key.pem \

   -config=/opt/kubernetes/ssl/ca-config.json \

   -profile=kubernetes admin-csr.json | cfssljson -bare admin

[[email protected] ssl]# ls -l admin*

-rw-r--r-- 1 root root 1009 Mar  5 12:29 admin.csr

-rw-r--r-- 1 root root  229 Mar  5 12:28 admin-csr.json

-rw------- 1 root root 1675 Mar  5 12:29 admin-key.pem

-rw-r--r-- 1 root root 1399 Mar  5 12:29 admin.pem

[[email protected] src]# mv admin*.pem /opt/kubernetes/ssl/

4.設定叢集引數

[[email protected] src]# kubectl config set-cluster kubernetes \

   --certificate-authority=/opt/kubernetes/ssl/ca.pem \

   --embed-certs=true \

   --server=https://192.168.56.11:6443

Cluster "kubernetes" set.

5.設定客戶端認證引數

[[email protected] src]# kubectl config set-credentials admin \

   --client-certificate=/opt/kubernetes/ssl/admin.pem \

   --embed-certs=true \

   --client-key=/opt/kubernetes/ssl/admin-key.pem

User "admin" set.

6.設定上下文引數

[[email protected] src]# kubectl config set-context kubernetes \

   --cluster=kubernetes \

   --user=admin

Context "kubernetes" created.

7.設定預設上下文

[[email protected] src]# kubectl config use-context kubernetes

Switched to context "kubernetes".

8.使用kubectl工具

[[email protected] ~]# kubectl get cs

NAME                 STATUS    MESSAGE             ERROR

controller-manager   Healthy   ok                 

scheduler            Healthy   ok                 

etcd-1               Healthy   {"health":"true"}  

etcd-2               Healthy   {"health":"true"}  

etcd-0               Healthy   {"health":"true"}  

部署kubelet

1.二進位制包準備 將軟體包從linux-node1複製到linux-node2中去。

[[email protected] ~]# cd /usr/local/src/kubernetes/server/bin/

[[email protected] bin]# cp kubelet kube-proxy /opt/kubernetes/bin/

[[email protected] bin]# scp kubelet kube-proxy 192.168.56.12:/opt/kubernetes/bin/

[[email protected] bin]# scp kubelet kube-proxy 192.168.56.13:/opt/kubernetes/bin/

2.建立角色繫結

[[email protected] ~]# kubectl create clusterrolebinding kubelet-bootstrap --clusterrole=system:node-bootstrapper --user=kubelet-bootstrap

clusterrolebinding "kubelet-bootstrap" created

3.建立 kubelet bootstrapping kubeconfig 檔案 設定叢集引數

[[email protected] ~]# kubectl config set-cluster kubernetes \

   --certificate-authority=/opt/kubernetes/ssl/ca.pem \

   --embed-certs=true \

   --server=https://192.168.56.11:6443 \

   --kubeconfig=bootstrap.kubeconfig

Cluster "kubernetes" set.

設定客戶端認證引數

[[email protected] ~]# kubectl config set-credentials kubelet-bootstrap \

   --token=ad6d5bb607a186796d8861557df0d17f \

   --kubeconfig=bootstrap.kubeconfig  

User "kubelet-bootstrap" set.

設定上下文引數

[[email protected] ~]# kubectl config set-context default \

   --cluster=kubernetes \

   --user=kubelet-bootstrap \

   --kubeconfig=bootstrap.kubeconfig

Context "default" created.

選擇預設上下文

[[email protected] ~]# kubectl config use-context default --kubeconfig=bootstrap.kubeconfig

Switched to context "default".

[[email protected] kubernetes]# cp bootstrap.kubeconfig /opt/kubernetes/cfg

[[email protected] kubernetes]# scp bootstrap.kubeconfig 192.168.56.12:/opt/kubernetes/cfg

[[email protected] kubernetes]# scp bootstrap.kubeconfig 192.168.56.13:/opt/kubernetes/cfg

部署kubelet 1.設定CNI支援

[[email protected] ~]# mkdir -p /etc/cni/net.d

[[email protected] ~]# vim /etc/cni/net.d/10-default.conf

{
        "name": "flannel",
        "type": "flannel",
        "delegate": {
            "bridge": "docker0",
            "isDefaultGateway": true,
            "mtu": 1400
        }
}

2.建立kubelet目錄

[[email protected] ~]# mkdir /var/lib/kubelet

3.建立kubelet服務配置

[[email protected] ~]# vim /usr/lib/systemd/system/kubelet.service

[Unit]

Description=Kubernetes Kubelet

Documentation=https://github.com/GoogleCloudPlatform/kubernetes

After=docker.service

Requires=docker.service

[Service]

WorkingDirectory=/var/lib/kubelet

ExecStart=/opt/kubernetes/bin/kubelet \

  --address=192.168.56.12 \

  --hostname-override=192.168.56.12 \

  --pod-infra-container-image=mirrorgooglecontainers/pause-amd64:3.0 \

  --experimental-bootstrap-kubeconfig=/opt/kubernetes/cfg/bootstrap.kubeconfig \

  --kubeconfig=/opt/kubernetes/cfg/kubelet.kubeconfig \

  --cert-dir=/opt/kubernetes/ssl \

  --network-plugin=cni \

  --cni-conf-dir=/etc/cni/net.d \

  --cni-bin-dir=/opt/kubernetes/bin/cni \

  --cluster-dns=10.1.0.2 \

  --cluster-domain=cluster.local. \

  --hairpin-mode hairpin-veth \

  --allow-privileged=true \

  --fail-swap-on=false \

  --logtostderr=true \

  --v=2 \

  --logtostderr=false \

  --log-dir=/opt/kubernetes/log

Restart=on-failure

RestartSec=5

4.啟動Kubelet

[[email protected] ~]# systemctl daemon-reload

[[email protected] ~]# systemctl enable kubelet

[[email protected] ~]# systemctl start kubelet

5.檢視服務狀態

[[email protected] kubernetes]# systemctl status kubelet

6.檢視csr請求 注意是在linux-node1上執行。

[[email protected] ~]# kubectl get csr

NAME                                                   AGE       REQUESTOR           CONDITION

node-csr-0_w5F1FM_la_SeGiu3Y5xELRpYUjjT2icIFk9gO9KOU   1m        kubelet-bootstrap   Pending

7.批准kubelet 的 TLS 證書請求

[[email protected] ~]# kubectl get csr|grep 'Pending' | awk 'NR>0{print $1}'| xargs kubectl certificate approve

執行完畢後，檢視節點狀態已經是Ready的狀態了 [[email protected] ssl]# kubectl get node NAME STATUS ROLES AGE VERSION

部署Kubernetes Proxy

1.配置kube-proxy使用LVS

[[email protected] ~]# yum install -y ipvsadm ipset conntrack

2.建立 kube-proxy 證書請求

[[email protected] ~]# cd /usr/local/src/ssl/

[[email protected] ~]# vim kube-proxy-csr.json

{
  "CN": "system:kube-proxy",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}

3.生成證書

[[email protected]~]# cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \

   -ca-key=/opt/kubernetes/ssl/ca-key.pem \

   -config=/opt/kubernetes/ssl/ca-config.json \

   -profile=kubernetes  kube-proxy-csr.json | cfssljson -bare kube-proxy

4.分發證書到所有Node節點

[[email protected] ssl]# cp kube-proxy*.pem /opt/kubernetes/ssl/

[[email protected] ssl]# scp kube-proxy*.pem 192.168.56.12:/opt/kubernetes/ssl/

[[email protected] ssl]# scp kube-proxy*.pem 192.168.56.12:/opt/kubernetes/ssl/

5.建立kube-proxy配置檔案

[[email protected] ~]# kubectl config set-cluster kubernetes \

   --certificate-authority=/opt/kubernetes/ssl/ca.pem \

   --embed-certs=true \

   --server=https://192.168.56.11:6443 \

   --kubeconfig=kube-proxy.kubeconfig

Cluster "kubernetes" set.

[[email protected] ~]# kubectl config set-credentials kube-proxy \

   --client-certificate=/opt/kubernetes/ssl/kube-proxy.pem \

   --client-key=/opt/kubernetes/ssl/kube-proxy-key.pem \

   --embed-certs=true \

   --kubeconfig=kube-proxy.kubeconfig

User "kube-proxy" set.

[[email protected] ~]# kubectl config set-context default \

   --cluster=kubernetes \

   --user=kube-proxy \

   --kubeconfig=kube-proxy.kubeconfig

Context "default" created.

[[email protected] ~]# kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig

Switched to context "default".

6.分發kubeconfig配置檔案

[[email protected] ssl]# cp kube-proxy.kubeconfig /opt/kubernetes/cfg/

[[email protected] ~]# scp kube-proxy.kubeconfig 192.168.56.12:/opt/kubernetes/cfg/

[[email protected] ~]# scp kube-proxy.kubeconfig 192.168.56.13:/opt/kubernetes/cfg/

7.建立kube-proxy服務配置

[[email protected] bin]# mkdir /var/lib/kube-proxy

[[email protected] ~]# vim /usr/lib/systemd/system/kube-proxy.service

[Unit]
Description=Kubernetes Kube-Proxy Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target
[Service]
WorkingDirectory=/var/lib/kube-proxy
ExecStart=/opt/kubernetes/bin/kube-proxy \
  --bind-address=192.168.56.12 \
  --hostname-override=192.168.56.12 \
  --kubeconfig=/opt/kubernetes/cfg/kube-proxy.kubeconfig \
--masquerade-all \
  --feature-gates=SupportIPVSProxyMode=true \
  --proxy-mode=ipvs \
  --ipvs-min-sync-period=5s \
  --ipvs-sync-period=5s \
  --ipvs-scheduler=rr \
  --logtostderr=true \
  --v=2 \
  --logtostderr=false \
  --log-dir=/opt/kubernetes/log
Restart=on-failure
RestartSec=5
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target

8.啟動Kubernetes Proxy

[[email protected] ~]# systemctl daemon-reload

[[email protected] ~]# systemctl enable kube-proxy

[[email protected] ~]# systemctl start kube-proxy

9.檢視服務狀態 檢視kube-proxy服務狀態

[[email protected] scripts]# systemctl status kube-proxy

檢查LVS狀態

[[email protected] ~]# ipvsadm -L -n

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  10.1.0.1:443 rr persistent 10800

  -> 192.168.56.11:6443           Masq    1      0          0        

如果你在兩臺實驗機器都安裝了kubelet和proxy服務，使用下面的命令可以檢查狀態：

[[email protected] ssl]#  kubectl get node

NAME            STATUS    ROLES     AGE       VERSION

192.168.56.12   Ready     <none>    22m       v1.10.1

192.168.56.13   Ready     <none>    3m        v1.10.1

linux-node3節點請自行部署。

1.為Flannel生成證書

[[email protected] ~]# vim flanneld-csr.json

{
  "CN": "flanneld",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}

2.生成證書

[[email protected] ~]# cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \

   -ca-key=/opt/kubernetes/ssl/ca-key.pem \

   -config=/opt/kubernetes/ssl/ca-config.json \

   -profile=kubernetes flanneld-csr.json | cfssljson -bare flanneld

3.分發證書

[[email protected] ~]# cp flanneld*.pem /opt/kubernetes/ssl/

[[email protected] ~]# scp flanneld*.pem 192.168.56.12:/opt/kubernetes/ssl/

[[email protected] ~]# scp flanneld*.pem 192.168.56.13:/opt/kubernetes/ssl/

4.下載Flannel軟體包

[[email protected] ~]# cd /usr/local/src

# wget

 https://github.com/coreos/flannel/releases/download/v0.10.0/flannel-v0.10.0-linux-amd64.tar.gz

[[email protected] src]# tar zxf flannel-v0.10.0-linux-amd64.tar.gz

[[email protected] src]# cp flanneld mk-docker-opts.sh /opt/kubernetes/bin/

複製到linux-node2節點

[[email protected] src]# scp flanneld mk-docker-opts.sh 192.168.56.12:/opt/kubernetes/bin/

[[email protected] src]# scp flanneld mk-docker-opts.sh 192.168.56.13:/opt/kubernetes/bin/

複製對應指令碼到/opt/kubernetes/bin目錄下。

[[email protected] ~]# cd /usr/local/src/kubernetes/cluster/centos/node/bin/

[[email protected] bin]# cp remove-docker0.sh /opt/kubernetes/bin/

[[email protected] bin]# scp remove-docker0.sh 192.168.56.12:/opt/kubernetes/bin/

[[email protected] bin]# scp remove-docker0.sh 192.168.56.13:/opt/kubernetes/bin/

5.配置Flannel

[[email protected] ~]# vim /opt/kubernetes/cfg/flannel

FLANNEL_ETCD="-etcd-endpoints=https://192.168.56.11:2379,https://192.168.56.12:2379,https://192.168.56.13:2379"
FLANNEL_ETCD_KEY="-etcd-prefix=/kubernetes/network"
FLANNEL_ETCD_CAFILE="--etcd-cafile=/opt/kubernetes/ssl/ca.pem"
FLANNEL_ETCD_CERTFILE="--etcd-certfile=/opt/kubernetes/ssl/flanneld.pem"
FLANNEL_ETCD_KEYFILE="--etcd-keyfile=/opt/kubernetes/ssl/flanneld-key.pem"

複製配置到其它節點上

[[email protected] ~]# scp /opt/kubernetes/cfg/flannel 192.168.56.12:/opt/kubernetes/cfg/

[[email protected] ~]# scp /opt/kubernetes/cfg/flannel 192.168.56.13:/opt/kubernetes/cfg/

6.設定Flannel系統服務

[[email protected] ~]# vim /usr/lib/systemd/system/flannel.service

[Unit]
Description=Flanneld overlay address etcd agent
After=network.target
Before=docker.service
[Service]
EnvironmentFile=-/opt/kubernetes/cfg/flannel
ExecStartPre=/opt/kubernetes/bin/remove-docker0.sh
ExecStart=/opt/kubernetes/bin/flanneld ${FLANNEL_ETCD} ${FLANNEL_ETCD_KEY} ${FLANNEL_ETCD_CAFILE} ${FLANNEL_ETCD_CERTFILE} ${FLANNEL_ETCD_KEYFILE}
ExecStartPost=/opt/kubernetes/bin/mk-docker-opts.sh -d /run/flannel/docker
Type=notify
[Install]
WantedBy=multi-user.target
RequiredBy=docker.service

複製系統服務指令碼到其它節點上

# scp /usr/lib/systemd/system/flannel.service 192.168.56.12:/usr/lib/systemd/system/

# scp /usr/lib/systemd/system/flannel.service 192.168.56.13:/usr/lib/systemd/system/

Flannel CNI整合

下載CNI外掛https://github.com/containernetworking/plugins/releases

wget https://github.com/containernetworking/plugins/releases/download/v0.7.1/cni-plugins-amd64-v0.7.1.tgz

[[email protected] ~]# mkdir /opt/kubernetes/bin/cni

[[email protected] src]# tar zxf cni-plugins-amd64-v0.7.1.tgz -C /opt/kubernetes/bin/cni

# scp -r /opt/kubernetes/bin/cni/* 192.168.56.12:/opt/kubernetes/bin/cni/

# scp -r /opt/kubernetes/bin/cni/* 192.168.56.13:/opt/kubernetes/bin/cni/

建立Etcd的key

/opt/kubernetes/bin/etcdctl --ca-file /opt/kubernetes/ssl/ca.pem --cert-file /opt/kubernetes/ssl/flanneld.pem --key-file /opt/kubernetes/ssl/flanneld-key.pem \

      --no-sync -C https://192.168.56.11:2379,https://192.168.56.12:2379,https://192.168.56.13:2379 \

mk /kubernetes/network/config '{ "Network": "10.2.0.0/16", "Backend": { "Type": "vxlan", "VNI": 1 }}' >/dev/null 2>&1

啟動flannel

相關推薦

kubernetes分散式安裝部署-簡介

環境說明：3臺1C1G（我自己這樣的配置，有條件的可以調整大點）虛擬機器且操作是系統centos7系統，三臺虛擬機器IP地址為： 1）192.168.56.11 主機名字：linux-node1.example.com 2）192.168.56.12 主機名字：linux-no

hadoop 3.1.1 分散式安裝部署 ubuntu16.04

參考連結：https://blog.didiyun.com/index.php/2018/11/20/hadoop3-1-1/ 其中需要注意的幾個點包括 hosts節點中刪除原來的127.0.0.1 localhost和127.0.0.1 master兩行。 hadoop使

基於RKE進行Kubernetes的安裝部署_Kubernetes中文社群

在進行Kubernetes的深入學習之前，首先要做的一件事情就是Kubernetes環境的安裝部署。由於Kubernetes本身的複雜性，因此決定安裝過程相當較為複雜，在安裝過程中需要考慮諸多的因素。在本文中，將使用Kubernetes的輕量級工具——Rancher Kubernetes En

Centos7 下Kubernetes叢集安裝部署

[[email protected] ~]# vi /etc/etcd/etcd.conf #[Member] #ETCD_CORS="" ETCD_DATA_DIR="/var/lib/etcd/default.etcd" #ETCD_WAL_DIR="" ETCD_LISTEN_PEER_UR

Kubernetes對象之Pod詳解(附安裝部署方法)

虛擬化技術 kubernetes docker 首先介紹一下K8S是什麽：(引用自K8S中文社區)Kubernetes是容器集群管理系統，是一個開源的平臺，可以實現容器集群的自動化部署、自動擴縮容、維護等功能。通過Kubernetes你可以：快速部署應用快速擴展應用無縫對接新的應用功能節省資源，優

Kubernetes（K8s）安裝部署過程（一）--證書安裝

更改 目錄 hand /etc 主題 nbsp kubecon 安裝部署 post 一、安裝前主題環境準備 　　1、docker安裝 　　建議使用官網yum源安裝，添加yum源之後，直接yum install docker即可 　　2、關閉所有節點的selinux

Kubernetes（K8s）安裝部署過程（三）--創建高可用etcd集群

方式安裝 10.9 修改配置 取消 roo initial code clas list 這裏的etcd集群復用我們測試的3個節點，3個node都要安裝並啟動，註意修改配置文件 1、TLS認證文件分發：etcd集群認證用，除了本機有，分發到其他node節

Kubernetes（K8s）安裝部署過程（四）--Master節點安裝

emc 不用 ces 成功 sts var 獲取 art health 再次明確下架構： 三臺虛擬機 centos 7.4系統，docker為17版本，ip為10.10.90.105到107，其中105位master，接下來的master相關組件安裝到此機器上。 etc

Kubernetes（K8s）安裝部署過程（六）--node節點部署

sch wan tool pods systemd tps stat mis type hi，everybody，我回來了，之前安裝到flannel之後，文章一直沒有更新，甚至不少小夥伴都來加qq詢問是否繼續更新了， 這裏說明下原因，我在部署1.91node的時候的確出現

Kubernetes1.91（K8s）安裝部署過程（八）-- kubernetes-dashboard安裝

addons quest 集群 heapster 管理 min sele author aps kubernets-dashboard顧名思義是操作面板安裝，也就是可視化管理機器，同意我們用鏡像結合配置文件部署。 1、下載鏡像： docker pull reg

kubernetes 中DNS的安裝部署測試

rgs sid proto args GC con target iat 通信 [toc] 安裝DNS後，pod就可以通過dns來解析service，從而實現通信 kubernetes 版本： kubectl version 我當前的版本是1.9.0

Sqoop簡介及安裝部署

cdh5 ive not 簡介 ins one generate and lin 簡介： Apache Sqoop是專為Apache Hadoop和結構化數據存儲如關系數據庫之間的數據轉換工具的有效工具。你可以使用Sqoop從外部結構化數據存儲的數據導入到Hadoop分布

Apache Flume簡介及安裝部署

display 根據 buffers type hello headers agent tran 結構圖 概述 Flume 是 Cloudera 提供的一個高可用的，高可靠的，分布式的海量日誌采集、聚合和傳輸的軟件。 Flume 的核心是把數據從數據源(source)收集

kubernetes集群安裝部署

std iptable 接口 dock ole 日誌級別 name -m entos 部署環境說明： 1、CENTOS 7.4 2、Docker version 17.05.0 3、etcd Version: 3.3.8

Kubernetes安裝部署

adding ces 1.5 playbook com pac book 所有組 github k8s安裝部署方法 手動部署：部署k8s自身，讓k8s相關組件都運行為系統守護進程 GitHub上托管的Ansible playbook kubeadm安裝部署

使用 kubeadm 安裝部署 kubernetes 1.9-部署heapster插件

master aml con inf uber kubectl net raw 重新 1.先到外網下載好鏡像倒進各個節點 2.下載yaml文件和創建應用 mkdir -p ~/k8s/heapster cd ~/k8s/heapster wget https://raw.

kubernetes+docker+dashboard安裝部署詳細步驟

fire col owin selinux 打通 dct des 主機名 -c 對docker和kubernetes都不熟悉，搜了很多指導，在kubernetes集群中安裝dashboard的時候總是會遇到各種問題，花了兩天時間試了很多方法，終於安裝成功，主體步驟來自

熱部署簡介及在eclipse安裝插件JRebel進行熱部署

ready tps ebe 變化 project bsp connect art nbsp 一、熱部署簡介 1、熱部署與熱加載在應用運行的時候升級軟件，無需重新啟動的方式有兩種，熱部署和熱加載。它們之間的區別是:(1)、部署方式: 熱部署在服務器運行時重新部署項目。熱加載在

使用 Kubeadm 安裝部署 Kubernetes 1.12.1 集群

etc firewall emctl linu http nbsp set -s code 手工搭建 Kubernetes 集群是一件很繁瑣的事情，所以就產生了很多輔助性安裝工具，如 Kubeadm ，Kubespray，RKE 等組件，我最終選擇了官方的 Kubeadm

Centos7安裝部署Kubernetes（K8s）集群

signed node use ots 新的 b- 分區 str ext Kubernetes集群的安裝有多種方式：下載源碼包編譯安裝、下載編譯好的二進制包安裝、使用kubeadm工具安裝等。本文是以二進制文件方式安裝Kubernetes集群。系統環境 主機名 IP地