glusterfs（分散式儲存）是企業中的主流部署形式

準備：

將所有節點寫入：
192.168.1.21 server1
192.168.1.111 server2，
將節點的hostname名字改成對應的server

glusterfs資料盤需要和系統盤隔開：所以沒有盤的可以掛載一個盤，有的請忽略
格式化磁碟：

fdisk /dev/xvdb
mkfs.ext4 /dev/xvdb1
mkdir /glusterfs/data -p
echo '/dev/xvdb1 /glusterfs/data ext4 defaults 1 2' >> /etc/fstab
mount -a && mount
 

安裝：

centos7安裝擴充套件源

vim /etc/yum.repos.d/gluster.repo

[glusterfs]

name=glusterfs

#baseurl=http://download.gluster.org/pub/gluster/glusterfs/3.4/3.4.7/CentOS/epel-6.5/x86_64/
baseurl= http://buildlogs.centos.org/centos/7/storage/x86_64/gluster-3.12/

enabled=1

gpgcheck=0

使用yum安裝

yum install glusterfs-server

centos6:

wget https://buildlogs.centos.org/centos/6/storage/x86_64/gluster-3.12/glusterfs-server-3.12.14-1.el6.x86_64.rpm
wget https://buildlogs.centos.org/centos/6/storage/x86_64/gluster-3.12/glusterfs-api-3.12.14-1.el6.x86_64.rpm
 wget https://buildlogs.centos.org/centos/6/storage/x86_64/gluster-3.12/glusterfs-cli-3.12.14-1.el6.x86_64.rpm
wget https://buildlogs.centos.org/centos/6/storage/x86_64/gluster-3.12/glusterfs-client-xlators-3.12.14-1.el6.x86_64.rpm
wget https://buildlogs.centos.org/centos/6/storage/x86_64/gluster-3.12/glusterfs-fuse-3.12.14-1.el6.x86_64.rpm
wget https://buildlogs.centos.org/centos/6/storage/x86_64/gluster-3.12/glusterfs-geo-replication-3.12.14-1.el6.x86_64.rpm
wget https://buildlogs.centos.org/centos/6/storage/x86_64/gluster-3.12/glusterfs-libs-3.12.14-1.el6.x86_64.rpm
wget https://buildlogs.centos.org/centos/6/storage/x86_64/gluster-3.12/glusterfs-3.12.14-1.el6.x86_64.rpm
wget https://buildlogs.centos.org/centos/6/storage/x86_64/gluster-3.12/userspace-rcu-0.7.16-2.el6.x86_64.rpm
 

安裝：
強制安裝：

rpm -ivh  *  --force --nodeps 

安裝完畢後啟動：

service glusterd start  

新增節點：

gluster peer probe server2

檢視狀態：

gluster peer status

刪除節點：

gluster peer detach server2

建立資料卷：

mkdir /home/test-volume
gluster volume create gv1 replica 2 server1:/home/test-volume server2:/home/test-volume

如果報錯的話需要，清掉/home/test-volume 裡面的檔案重來：

cd /home/test-volume
rm -rf .glusterfs
setfattr -x trusted.glusterfs.volume-id ./
setfattr -x trusted.gfid ./

檢視資料卷狀態：

 gluster volume info

使用：

首先我們得在也好使用的伺服器安裝客戶端
新增hosts

192.168.1.21 server1
192.168.1.111 server2

安裝客戶端

yum install glusterfs-fuse

掛載glusterfs

mount -t glusterfs server1:/gv1 /mnt/

[[email protected] gluster-yaml]# cat glusterfs-endpoints.json
{
  "kind": "Endpoints",
  "apiVersion": "v1",
  "metadata": {
    "name": "glusterfs-cluster"
  },
  "subsets": [
    {
      "addresses": [
        {
          "ip": "192.168.1.21"
        }
      ],
      "ports": [
        {
          "port": 1
        }
      ]
    },
    {
      "addresses": [
        {
          "ip": "192.168.1.111"
        }
      ],
      "ports": [
        {
          "port": 1
        }
      ]
    }
  ]
}

建立glusterfs服務：
[[email protected] gluster-yaml]# cat glusterfs-service.json
{
  "kind": "Service",
  "apiVersion": "v1",
  "metadata": {
    "name": "glusterfs-cluster"
  },
  "spec": {
    "ports": [
      {"port": 1}
    ]
  }
}

建立測試用例：

[[email protected] gluster-yaml]# cat nginx-deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        volumeMounts:
        - name: glusterfsvol
          mountPath: /usr/share/nginx/html
        ports:
        - containerPort: 80
      volumes:
      - name: glusterfsvol
        glusterfs:
          endpoints: glusterfs-cluster
          path: gv1
          readOnly: false

---

apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    app: nginx
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  type: NodePort

建立服務

kubectl create -f glusterfs-endpoints.json
kubectl create -f glusterfs-service.json
kubectl create -f nginx-deployment.yaml

檢測服務：

[[email protected] gluster-yaml]# kubectl get ep
NAME                ENDPOINTS                         AGE
glusterfs-cluster   192.168.1.111:1,192.168.1.21:1    26m
nginx-service       172.17.102.4:80,172.17.102.5:80   10m

kubectl get svc
glusterfs-cluster   ClusterIP   10.10.10.117   <none>        1/TCP          25m
nginx-service       NodePort    10.10.10.109   <none>        80:33725/TCP   10m

此時我們可以cur 10.10.10.109 了，看看是否會出現我們在test-volume中編輯的內容了