nfs 作為 k8s 的網路儲存驅動，可以滿足持久儲存業務的需求，支援多節點讀也寫。
下面採用 k8s pv與pvc來配套使用nfs資源。
建立pv，並建立與之關聯的pvc，如下：

-[[email protected] pvtest]$ kubectl get pv
NAME            CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM                       STORAGECLASS   REASON    AGE
nfs-pv          100Mi      RWX            Retain           Bound     chenqiang-pv-test/nfs-pvc                            12
 
m
nfs-server-pv   100Gi      RWX            Retain           Bound     default/nfs-server-pvc                               2h
-[[email protected] pvtest]$ kubectl get pvc
NAME             STATUS    VOLUME          CAPACITY   ACCESS MODES   STORAGECLASS   AGE
nfs-server-pvc   Bound     nfs-server-pv   100
 
Gi      RWX                           2h
-[[email protected] pvtest]$ kubectl -n chenqiang-pv-test get pvc
NAME      STATUS    VOLUME    CAPACITY   ACCESS MODES   STORAGECLASS   AGE
nfs-pvc   Bound     nfs-pv    100Mi      RWX                           12m
-[[email protected] pvtest]$ kubectl -n chenqiang-pv-test get pv
NAME            CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM                       STORAGECLASS   REASON    AGE
nfs-pv          100
 
Mi      RWX            Retain           Bound     chenqiang-pv-test/nfs-pvc                            12m
nfs-server-pv   100Gi      RWX            Retain           Bound     default/nfs-server-pvc                               2h

注意：pv是沒有namespace的概念，也就沒有租戶的概念，但 pvc 有租戶的概念，當需要在某個 namespace 下使用 pvc 時，需要指定該 pvc 所屬 namespace。這個我們可以通過上面的演示看出。

建立名字為 nfs-pv 的 pv，假設 yaml 命名為 nfs-pv.yaml 如下：

apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-pv
  namespace: chenqiang-pv-test
spec:
  capacity:
    storage: 100Mi
  accessModes:
    - ReadWriteMany
  nfs:
    # FIXME: use the right IP
    server: 10.130.44.20
    path: "/test/mysql-nfs01"

建立名字為 nfs-pvc 的 pvc, 假設 yaml 命名為 nfs-pvc.yaml 如下：

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nfs-pvc
  namespace: chenqiang-pv-test
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: ""
  resources:
    requests:
      storage: 90Mi

建立使用pod

      containers:
      - name: nginx-hello
        image: docker-registry.saicstack.com/chenqiang/nginx-hello:v2.0
        ports:
        - containerPort: 80
        securityContext:
          privileged: true
        volumeMounts:
        # name must match the volume name below
        - name: nfs
          mountPath: "/chenqiang/pv-test"
      volumes:
      - name: nfs
        persistentVolumeClaim:
          claimName: nfs-pvc

看下 pod 是否起來：

-[[email protected] pvtest]$ kubectl -n chenqiang-pv-test get po
NAME                                      READY     STATUS              RESTARTS   AGE
nginx-hello-deployment-6f9f4d7bcc-86kb5   1/1       Running             0          1h

如果在建立 pod 的時，沒有先建立 pvc，那麼為出現如下的 event log:

Events:
  Type     Reason                 Age               From                  Message
  ----     ------                 ----              ----                  -------
  Warning  FailedMount            58m (x4 over 1h)  kubelet, 10.130.33.8  Unable to mount volumes for pod "nginx-hello-deployment-6f9f4d7bcc-86kb5_chenqiang-pv-test(7b18c376-7382-11e8-9a03-74eacb756039)": timeout expired waiting for volumes to attach/mount for pod "chenqiang-pv-test"/"nginx-hello-deployment-6f9f4d7bcc-86kb5". list of unattached/unmounted volumes=[nfs]

當 pvc 建立好後，繼續檢視log如下：

Events:
  Type     Reason                 Age               From                  Message
  ----     ------                 ----              ----                  -------
  Warning  FailedMount            58m (x4 over 1h)  kubelet, 10.130.33.8  Unable to mount volumes for pod "nginx-hello-deployment-6f9f4d7bcc-86kb5_chenqiang-pv-test(7b18c376-7382-11e8-9a03-74eacb756039)": timeout expired waiting for volumes to attach/mount for pod "chenqiang-pv-test"/"nginx-hello-deployment-6f9f4d7bcc-86kb5". list of unattached/unmounted volumes=[nfs]
  Normal   SuccessfulMountVolume  58m               kubelet, 10.130.33.8  MountVolume.SetUp succeeded for volume "nfs-pv"
  Normal   Pulled                 58m               kubelet, 10.130.33.8  Container image "docker-registry.saicstack.com/chenqiang/nginx-hello:v2.0" already present on machine
  Normal   Created                58m               kubelet, 10.130.33.8  Created container
  Normal   Started                58m               kubelet, 10.130.33.8  Started container

當希望建立 2 個 pod 的時候，會發現有一個啟動不了。

-[[email protected] pvtest]$ kubectl -n chenqiang-pv-test get po -o wide
NAME                                      READY     STATUS              RESTARTS   AGE       IP               NODE
nginx-hello-deployment-6f9f4d7bcc-86kb5   1/1       Running             0          1h        172.12.180.144   10.130.33.8
nginx-hello-deployment-6f9f4d7bcc-lbt2n   0/1       ContainerCreating   0          1h        <none>           10.130.33.13

檢視該 pod event log:

Events:
  Type     Reason       Age                From                   Message
  ----     ------       ----               ----                   -------
  Warning  FailedMount  6m (x25 over 1h)   kubelet, 10.130.33.13  Unable to mount volumes for pod "nginx-hello-deployment-6f9f4d7bcc-lbt2n_chenqiang-pv-test(7b0fcbd3-7382-11e8-80b5-74eacb7559f1)": timeout expired waiting for volumes to attach/mount for pod "chenqiang-pv-test"/"nginx-hello-deployment-6f9f4d7bcc-lbt2n". list of unattached/unmounted volumes=[nfs]
  Warning  FailedMount  12s (x39 over 1h)  kubelet, 10.130.33.13  (combined from similar events): MountVolume.SetUp failed for volume "nfs-pv" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/7b0fcbd3-7382-11e8-80b5-74eacb7559f1/volumes/kubernetes.io~nfs/nfs-pv --scope -- mount -t nfs 10.130.44.20:/test/mysql-nfs01 /var/lib/kubelet/pods/7b0fcbd3-7382-11e8-80b5-74eacb7559f1/volumes/kubernetes.io~nfs/nfs-pv
Output: Running scope as unit run-11918.scope.
mount: wrong fs type, bad option, bad superblock on 10.130.44.20:/test/mysql-nfs01,
       missing codepage or helper program, or other error
       (for several filesystems (e.g. nfs, cifs) you might
       need a /sbin/mount.<type> helper program)

       In some cases useful info is found in syslog - try
       dmesg | tail or so.

根據錯誤提示，檢視/sbin/mount.<type>檔案，果然發現沒有/sbin/mount.nfs的檔案，安裝 nfs-utils 即可

解決辦法：

apt-get install nfs-common

或者

yum install nfs-utils

安裝之後，/sbin/ 下面多了幾個 mount 檔案，比如 mount.nfs 和 mount.nfs4等：

[[email protected]]# cat /sbin/mo
modinfo     modprobe    mount.ceph  mount.nfs   mount.nfs4  mountstats 

再次 kubectl get po 試下，發現起來了：

-[[email protected] pvtest]$ kubectl -n chenqiang-pv-test get po -o wide
NAME                                      READY     STATUS    RESTARTS   AGE       IP               NODE
nginx-hello-deployment-6f9f4d7bcc-86kb5   1/1       Running   0          1h        172.12.180.144   10.130.33.8
nginx-hello-deployment-6f9f4d7bcc-lbt2n   1/1       Running   0          1h        172.12.232.103   10.130.33.13

接下來測試一下如何同時寫：
分別進入這兩個pod，並分別執行如下程式。

在 pod nginx-hello-deployment-6f9f4d7bcc-86kb5 中執行：

sh-4.1# for i in `seq 1 1000`; do echo $i >> a.test; done

在 pod nginx-hello-deployment-6f9f4d7bcc-lbt2n 中執行：

sh-4.1# for i in `seq 1000 10000`; do echo $i >> a.test; done

經過一段的同時寫後，擷取部分結果如下：

995
855
997
998
999
1000
1001
1002
1003
1004
1005

我們發現，兩個pod在同時開始寫的時候，是有一個先後順序的，比如第一個是從 1寫到999，等第一個寫完了，第二個再從 1000 寫到 9999 這是加了檔案鎖的緣故。
如果不加任何延時，由於寫的太快，中途可能會丟失資料，需要加些延時處理，比如：
另測試一組再次驗證：

pod 1 中執行：

for i in `seq 1 1000`; do echo $i >> a.test; sleep 1; done

pod 2 中執行：

for i in `seq 20001 21000`; do echo $i >> a.test; sleep 1; done

先敲下 pod2 中的命令，然後立馬敲下 pod1 中的命令，發現會錯開寫，部分結果如下：

20001
20002
20003
1
20004
2
20005
3
20006
4
20007
5
20008
6
20009
7
20010
8
20011
9
20012
10