由於專案組業務需求，在k8s中執行的微服務日誌要進行收集。

1、需要將k8s中的每個pod微服務掛載一個卷，（每個pod都會在一個node上，在node節點上建立一個資料夾，如：/logs/ 將這個目錄掛載到每個微服務的pod中，使之實現對映）

apiVersion: apps/v1
kind: Deployment
metadata:
  name: neusipo-cloud-gateway-deployment
  namespace: gateway
spec:
  selector:
    matchLabels:
      app: neusipo-cloud-gateway
  replicas: 3
  minReadySeconds: 10
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 0
    type: RollingUpdate
  template:
    metadata:
      name: neusipo-cloud-gateway
      labels:
        app: neusipo-cloud-gateway
    spec:
      containers:
      - name: neusipo-cloud-gateway
        image: cr.registry.cnipaig1.cloud/neusipo/neusipo-cloud-gateway-test:1.0  #這裡的映象地址是阿里雲映象倉庫地址
 

        args: ["--spring.profiles.active=test"]   #配置檔案中的測試配置
        env:
        - name: SERVER_PORT
          value: "18080"
        resources:
          requests:
            cpu: 4
            memory: 4Gi
          limits:
            cpu: 4
            memory: 4Gi
        volumeMounts:   #pod中掛載
 

        - name: gateway-logs   #name與下方本機name保持一致
          mountPath: /sipo-data/neusipo-test/   #pod中的掛載地址
          readOnly: false
      volumes:    #本機卷
      - name: gateway-logs     
        hostPath:
          path: /logs/   #本機建立的掛載目錄

2、執行命令，建立pod。

kubectl apply -f deployment-demo.yaml -n gateway

3、檢視是否建立成功

[root@iZqj001np4kv147ax0tr54Z ~]# kubectl get pods -n gateway
NAME                                                READY   STATUS    RESTARTS   AGE
neusipo-cloud-gateway-deployment-66b6c96fbf-9ncht   1/1     Running   0          4h16m
neusipo-cloud-gateway-deployment-66b6c96fbf-r7wcd   1/1     Running   0          4h16m
neusipo-cloud-gateway-deployment-66b6c96fbf-wwjpk   1/1     Running   0          4h16m

看到STATUS狀態為Running 時說明容器已啟動。

4、此時驗證pod中是否已經掛載上本機目錄。

[root@iZqj001np4kv147ax0tr54Z ~]# kubectl exec -it  neusipo-cloud-gateway-deployment-66b6c96fbf-9ncht -n gateway -- bash
[root@neusipo-cloud-gateway-deployment-66b6c96fbf-9ncht data]# df -h
Filesystem      Size  Used Avail Use% Mounted on
overlay          40G   17G   21G  45% /
tmpfs            64M     0   64M   0% /dev
tmpfs           7.8G     0  7.8G   0% /sys/fs/cgroup
/dev/vda1        40G   17G   21G  45% /sipo-data/neusipo-test   #這裡就代表已經掛載上本機所建立的目錄了
shm              64M     0   64M   0% /dev/shm
tmpfs           7.8G   12K  7.8G   1% /run/secrets/kubernetes.io/serviceaccount
tmpfs           7.8G     0  7.8G   0% /proc/acpi
tmpfs           7.8G     0  7.8G   0% /proc/scsi
tmpfs           7.8G     0  7.8G   0% /sys/firmware
[root@neusipo-cloud-gateway-deployment-66b6c96fbf-9ncht data]# 

此時檢視pod服務啟動後是否有日誌列印，通過對映到node節點上建立的目錄。（檢視pod在那個node節點上，登入node節點，進入建立的目錄，檢視是否pod日誌映射出來）

[root@neusipo-cloud-gateway-deployment-66b6c96fbf-9ncht data]# cd /sipo-data/neusipo-test/
[root@neusipo-cloud-gateway-deployment-66b6c96fbf-9ncht neusipo-test]# ls
gateway
[root@neusipo-cloud-gateway-deployment-66b6c96fbf-9ncht neusipo-test]# cd gateway/
[root@neusipo-cloud-gateway-deployment-66b6c96fbf-9ncht gateway]# ls
logs
[root@neusipo-cloud-gateway-deployment-66b6c96fbf-9ncht gateway]# cd logs/
[root@neusipo-cloud-gateway-deployment-66b6c96fbf-9ncht logs]# ls
error  info　

[root@iZqj001np4kv147ax0tr54Z ~]# kubectl get pods -n gateway -o wide
NAME                                                READY   STATUS    RESTARTS   AGE     IP             NODE                                         NOMINATED NODE   READINESS GATES
neusipo-cloud-gateway-deployment-66b6c96fbf-9ncht   1/1     Running   0          4h26m   172.20.1.91    cn-beijing-gzj1-d01.i-qj001np4kv147gu45x2l   <none>           <none>
neusipo-cloud-gateway-deployment-66b6c96fbf-r7wcd   1/1     Running   0          4h26m   172.20.0.198   cn-beijing-gzj1-d01.i-qj001np4kv147gu45x2k   <none>           <none>
neusipo-cloud-gateway-deployment-66b6c96fbf-wwjpk   1/1     Running   0          4h26m   172.20.1.60    cn-beijing-gzj1-d01.i-qj001np4kv147gu45x2j   <none>           <none>
[root@iZqj001np4kv147ax0tr54Z ~]# kubectl get node -o wide
NAME                                         STATUS   ROLES    AGE    VERSION            INTERNAL-IP    EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION               CONTAINER-RUNTIME
cn-beijing-gzj1-d01.i-qj001np4kv147ax0tr54   Ready    master   198d   v1.14.8-aliyun.1   10.160.22.68   <none>        CentOS Linux 7 (Core)   3.10.0-957.21.3.el7.x86_64   docker://18.9.2
cn-beijing-gzj1-d01.i-qj001np4kv147ax0tr55   Ready    master   198d   v1.14.8-aliyun.1   10.160.22.69   <none>        CentOS Linux 7 (Core)   3.10.0-957.21.3.el7.x86_64   docker://18.9.2
cn-beijing-gzj1-d01.i-qj001np4kv147ax0tr56   Ready    master   198d   v1.14.8-aliyun.1   10.160.22.70   <none>        CentOS Linux 7 (Core)   3.10.0-957.21.3.el7.x86_64   docker://18.9.2
cn-beijing-gzj1-d01.i-qj001np4kv147gu45x2j   Ready    <none>   198d   v1.14.8-aliyun.1   10.160.22.73   <none>        CentOS Linux 7 (Core)   3.10.0-957.21.3.el7.x86_64   docker://18.9.2
cn-beijing-gzj1-d01.i-qj001np4kv147gu45x2k   Ready    <none>   198d   v1.14.8-aliyun.1   10.160.22.72   <none>        CentOS Linux 7 (Core)   3.10.0-957.21.3.el7.x86_64   docker://18.9.2
cn-beijing-gzj1-d01.i-qj001np4kv147gu45x2l   Ready    <none>   198d   v1.14.8-aliyun.1   10.160.22.71   <none>        CentOS Linux 7 (Core)   3.10.0-957.21.3.el7.x86_64   docker://18.9.2

從這裡看到其中一個pod在10.160.22.73這個node節點上，登入node節點檢視是否有對映的日誌

[root@iZqj001np4kv147gu45x2jZ logs]# pwd
/logs
[root@iZqj001np4kv147gu45x2jZ logs]# ls
gateway
[root@iZqj001np4kv147gu45x2jZ logs]# cd gateway/
[root@iZqj001np4kv147gu45x2jZ gateway]# ls
logs
[root@iZqj001np4kv147gu45x2jZ gateway]# cd logs/
[root@iZqj001np4kv147gu45x2jZ logs]# ls
error  info

如果node節點出現了和pod中一樣的目錄則說明，掛載成功並對映成功，（其他微服務同理就行）

5、日誌映射出來後，需要搭建filebeat映象，建立filebeat的pod容器，容器需要建立為DaemonSet形式的（DaemonSet所建立的容器類似於守護程序，它會在每個node節點都建立一個pod），使filebeat容器可以從所有的node節點日誌目錄拉取日誌到pod中，併發送給elk進行日誌分析。在master節點上下載filebeat壓縮包，並解壓。

[root@iZqj001np4kv147ax0tr54Z ~]# ls
deployment-demo.yaml              filebeat-7.13.1-linux-x86_64.tar.gz         daemonset-demo.yamlfilebeat-7.13.1-linux-x86_64      daemonset-demo.yaml        dockerfile-filebeat

建立filebeat的docker映象

[root@iZqj001np4kv147ax0tr54Z ~]# cat dockerfile-filebeat 
FROM cr.registry.cnipaig1.cloud/neusipo/centos:7.9.2009
ENV LANG en_US.UTF-8
ENV TZ Asia/Shanghai

ADD filebeat-7.13.1-linux-x86_64.tar.gz /data/

WORKDIR /data/filebeat-7.13.1-linux-x86_64

ENTRYPOINT ["./filebeat", "-c", "filebeat.yml", "-e"]

上傳到阿里雲的映象倉庫，阿里雲容器映象服務文件中已寫如何上傳，照著一步步執行就可。

構建filebeat的yaml檔案

apiVersion: apps/v1
kind: DaemonSet   #這裡為DaemonSet模式
metadata:
  name: neusipo-filebeat
  namespace: filebeat
spec:
  selector:
    matchLabels:
      app: neusipo-filebeat
  template:
    metadata:
      name: neusipo-filebeat
      labels:
        app: neusipo-filebeat
    spec:
      containers:
      - name: neusipo-filebeat
        image: cr.registry.cnipaig1.cloud/neusipo/filebeat:v2
        command: ["./filebeat", "-c", "/data/filebeat/conf/filebeat.yml", "-e"]   #command模組可以強制替換容器裡預設的啟動命令
        resources:
          requests:
            cpu: 4
            memory: 4Gi
          limits:
            cpu: 4
            memory: 4Gi
        volumeMounts:
          - name: filebeat-log
            mountPath: /sipo-data/neusipo-test/    #這裡為掛載node目錄到pod中的目錄
            readOnly: false
          - name: neusipo-filebeat
            mountPath: /data/filebeat/conf/       #這裡為pod容器內filebeat配置檔案目錄
            readOnly: true
      volumes:
      - name: filebeat-log
        hostPath:
          path: /logs/          #node節點目錄
      - name: neusipo-filebeat
        configMap:               #將pod容器內的filebeat配置檔案對映到configmap中
          name: neusipo-filebeat    #configmap內filebeat的name
          items:
            - key: filebeat.yml      #對映在configmap中的filebeat的配置檔名
              path: filebeat.yml       #pod中filebeat的配置檔名

建立neusipo-filebeat的configmap，對映的檔案是從filebeat壓縮包解壓出來的配置檔案

[root@iZqj001np4kv147ax0tr54Z ~]# kubectl create configmap neusipo-filebeat -n filebeat
[root@iZqj001np4kv147ax0tr54Z ~]# kubectl get configmap -n filebeat
NAME               DATA   AGE
neusipo-filebeat   1      3h55m   #這裡DATA出現1，說明檔案已經寫入

建立namesapce

[root@iZqj001np4kv147ax0tr54Z ~]# kubectl create configmap ns filebeat
[root@iZqj001np4kv147ax0tr54Z ~]# kubectl get ns |grep filebeat
filebeat              Active   4h25m

建立filebeat的pod容器，並檢視是否建立成功

kubectl apply -f daemonset-demo.yaml -n filebeat

[root@iZqj001np4kv147ax0tr54Z ~]# kubectl get pods -n filebeat
NAME                     READY   STATUS    RESTARTS   AGE
neusipo-filebeat-64mhf   1/1     Running   0          3h55m
neusipo-filebeat-k9ncp   1/1     Running   0          3h55m
neusipo-filebeat-lsvcb   1/1     Running   0          3h55m

進入容器檢視是否掛載上node目錄，並檢視是否拉取到微服務的日誌

[root@iZqj001np4kv147ax0tr54Z ~]# kubectl exec -it neusipo-filebeat-k9ncp -n filebeat /bin/bash
[root@neusipo-filebeat-k9ncp filebeat-7.13.1-linux-x86_64]# df -h
Filesystem      Size  Used Avail Use% Mounted on
overlay          40G   17G   21G  45% /
tmpfs            64M     0   64M   0% /dev
tmpfs           7.8G     0  7.8G   0% /sys/fs/cgroup
/dev/vda1        40G   17G   21G  45% /sipo-data/neusipo-test
shm              64M     0   64M   0% /dev/shm
tmpfs           7.8G   12K  7.8G   1% /run/secrets/kubernetes.io/serviceaccount
tmpfs           7.8G     0  7.8G   0% /proc/acpi
tmpfs           7.8G     0  7.8G   0% /proc/scsi
tmpfs           7.8G     0  7.8G   0% /sys/firmware
[root@neusipo-filebeat-k9ncp filebeat-7.13.1-linux-x86_64]# cd /sipo-data/neusipo-test/
[root@neusipo-filebeat-k9ncp neusipo-test]# ls
gateway
[root@neusipo-filebeat-k9ncp neusipo-test]# cd gateway/
[root@neusipo-filebeat-k9ncp gateway]# ls
logs
[root@neusipo-filebeat-k9ncp gateway]# cd logs/
[root@neusipo-filebeat-k9ncp logs]# ls
error  info
[root@neusipo-filebeat-k9ncp logs]# 

後續只需要更改configmap中的filebeat配置檔案，就可以了。

kubectl edit configmap neusipo-filebeat -n filebeat

完成！！！