深度解析Istio系列之Istio-proxy初始化篇
註:以下講述的按理環境場景是基於Kubernetes環境基礎上部署的Istio環境。
涉及到Envoy概念介紹請參考深度解析Istio系列之流量控制篇。本文重點針對Envoy初始化場景進行拆解。
Istio-proxy(Envoy)作為Istio數據平面的重要組件,基於sidecar方式與業務應用混合部署到同一pod,為應用提供代理服務。Pilot作為控制平面組件,基於元數據的抽象層,屏蔽底層具體容器環境(Kubernetes或者docker),同時為Envoy的策略執行提供有效的數據支撐。那麽Envoy如何獲取Pilot所在地址的信息呢?
下面通過官網BookInfo案例中Productpage服務的配置文件來說明。
Pod容器說明
執行kubectl get deploy productpage-v1 -o yaml,獲取productpage部署模板,如下所示:
apiVersion: extensions/v1beta1 kind: Deployment metadata: annotations: deployment.kubernetes.io/revision: "2" kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"extensions/v1beta1","kind":"Deployment","metadata":{"annotations":{},"creationTimestamp":null,"name":"productpage-v1","namespace":"default"},"spec":{"replicas":1,"strategy":{},"template":{"metadata":{"annotations":{"sidecar.istio.io/status":"{\"version\":\"50128f63e7b050c58e1cdce95b577358054109ad2aff4bc4995158c06924a43b\",\"initContainers\":[\"istio-init\"],\"containers\":[\"istio-proxy\"],\"volumes\":[\"istio-envoy\",\"istio-certs\"],\"imagePullSecrets\":null}"},"creationTimestamp":null,"labels":{"app":"productpage","version":"v1"}},"spec":{"containers":[{"image":"istio/examples-bookinfo-productpage-v1:1.8.0","imagePullPolicy":"IfNotPresent","name":"productpage","ports":[{"containerPort":9080}],"resources":{}},{"args":["proxy","sidecar","--configPath","/etc/istio/proxy","--binaryPath","/usr/local/bin/envoy","--serviceCluster","productpage","--drainDuration","45s","--parentShutdownDuration","1m0s","--discoveryAddress","istio-pilot.istio-system:15007","--discoveryRefreshDelay","1s","--zipkinAddress","zipkin.istio-system:9411","--connectTimeout","10s","--proxyAdminPort","15000","--controlPlaneAuthPolicy","NONE"],"env":[{"name":"POD_NAME","valueFrom":{"fieldRef":{"fieldPath":"metadata.name"}}},{"name":"POD_NAMESPACE","valueFrom":{"fieldRef":{"fieldPath":"metadata.namespace"}}},{"name":"INSTANCE_IP","valueFrom":{"fieldRef":{"fieldPath":"status.podIP"}}},{"name":"ISTIO_META_POD_NAME","valueFrom":{"fieldRef":{"fieldPath":"metadata.name"}}},{"name":"ISTIO_META_INTERCEPTION_MODE","value":"REDIRECT"},{"name":"ISTIO_METAJSON_LABELS","value":"{\"app\":\"productpage\",\"version\":\"v1\"}\n"}],"image":"docker.io/istio/proxyv2:1.0.5","imagePullPolicy":"IfNotPresent","name":"istio-proxy","ports":[{"containerPort":15090,"name":"http-envoy-prom","protocol":"TCP"}],"resources":{"requests":{"cpu":"10m"}},"securityContext":{"readOnlyRootFilesystem":true,"runAsUser":1337},"volumeMounts":[{"mountPath":"/etc/istio/proxy","name":"istio-envoy"},{"mountPath":"/etc/certs/","name":"istio-certs","readOnly":true}]}],"initContainers":[{"args":["-p","15001","-u","1337","-m","REDIRECT","-i","*","-x","","-b","9080","-d",""],"image":"docker.io/istio/proxy_init:1.0.5","imagePullPolicy":"IfNotPresent","name":"istio-init","resources":{},"securityContext":{"capabilities":{"add":["NET_ADMIN"]},"privileged":true}}],"volumes":[{"emptyDir":{"medium":"Memory"},"name":"istio-envoy"},{"name":"istio-certs","secret":{"optional":true,"secretName":"istio.default"}}]}}},"status":{}} creationTimestamp: 2018-12-18T01:51:18Z generation: 2 labels: app: productpage version: v1 name: productpage-v1 namespace: default resourceVersion: "17402700" selfLink: /apis/extensions/v1beta1/namespaces/default/deployments/productpage-v1 uid: 695ba22c-0267-11e9-8475-0050569c62d0 spec: progressDeadlineSeconds: 600 replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: app: productpage version: v1 strategy: rollingUpdate: maxSurge: 1 maxUnavailable: 1 type: RollingUpdate template: metadata: annotations: sidecar.istio.io/status: ‘{"version":"50128f63e7b050c58e1cdce95b577358054109ad2aff4bc4995158c06924a43b","initContainers":["istio-init"],"containers":["istio-proxy"],"volumes":["istio-envoy","istio-certs"],"imagePullSecrets":null}‘ creationTimestamp: null labels: app: productpage version: v1 spec: containers: - image: istio/examples-bookinfo-productpage-v1:1.8.0 imagePullPolicy: IfNotPresent name: productpage ports: - containerPort: 9080 protocol: TCP resources: {} terminationMessagePath: /dev/termination-log terminationMessagePolicy: File - args: - proxy - sidecar - --configPath - /etc/istio/proxy - --binaryPath - /usr/local/bin/envoy - --serviceCluster - productpage - --drainDuration - 45s - --parentShutdownDuration - 1m0s - --discoveryAddress - istio-pilot.istio-system:15007 - --discoveryRefreshDelay - 1s - --zipkinAddress - zipkin.istio-system:9411 - --connectTimeout - 10s - --proxyAdminPort - "15000" - --controlPlaneAuthPolicy - NONE env: - name: POD_NAME valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.namespace - name: INSTANCE_IP valueFrom: fieldRef: apiVersion: v1 fieldPath: status.podIP - name: ISTIO_META_POD_NAME valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.name - name: ISTIO_META_INTERCEPTION_MODE value: REDIRECT - name: ISTIO_METAJSON_LABELS value: | {"app":"productpage","version":"v1"} image: docker.io/istio/proxyv2:1.0.5 imagePullPolicy: IfNotPresent name: istio-proxy ports: - containerPort: 15090 name: http-envoy-prom protocol: TCP resources: requests: cpu: 10m securityContext: readOnlyRootFilesystem: true runAsUser: 1337 terminationMessagePath: /dev/termination-log terminationMessagePolicy: File volumeMounts: - mountPath: /etc/istio/proxy name: istio-envoy - mountPath: /etc/certs/ name: istio-certs readOnly: true dnsPolicy: ClusterFirst initContainers: - args: - -p - "15001" - -u - "1337" - -m - REDIRECT - -i - ‘*‘ - -x - "" - -b - "9080" - -d - "" image: docker.io/istio/proxy_init:1.0.5 imagePullPolicy: IfNotPresent name: istio-init resources: {} securityContext: capabilities: add: - NET_ADMIN privileged: true terminationMessagePath: /dev/termination-log terminationMessagePolicy: File restartPolicy: Always schedulerName: default-scheduler securityContext: {} terminationGracePeriodSeconds: 30 volumes: - emptyDir: medium: Memory name: istio-envoy - name: istio-certs secret: defaultMode: 420 optional: true secretName: istio.default status: availableReplicas: 1 conditions: - lastTransitionTime: 2018-12-18T01:51:19Z lastUpdateTime: 2018-12-18T01:51:19Z message: Deployment has minimum availability. reason: MinimumReplicasAvailable status: "True" type: Available - lastTransitionTime: 2018-12-18T01:51:18Z lastUpdateTime: 2018-12-18T03:48:52Z message: ReplicaSet "productpage-v1-7b96bbf89f" has successfully progressed. reason: NewReplicaSetAvailable status: "True" type: Progressing observedGeneration: 2 readyReplicas: 1 replicas: 1 updatedReplicas: 1
模板中可見,除業務應用外,還包括istio-init,istio-proxy兩個容器。istio-init作為Init Container類型,在應用容器啟動之前啟動,用來執行初始化任務。Istio-init主要初始化iptables規則,用於服務間調用時的請求攔截及轉發。Istio-proxy是代理容器(即Envoy),容器中包含兩個進程,分別是Pilot-agent進程和Envoy進程。前者負責生成Envoy啟動所需的配置文件,啟動Envoy進程。後者在服務調用發生時負責具體的策略執行。
Envoy配置信息詳解
Envoy啟動後,執行kubectl exec -it productpage-v1-7b96bbf89f-pj28l -c istio-proxy /bin/sh,進入istio-proxy容器內部,在/etc/istio/proxy目錄下存在envoy-rev0.json配置文件,如圖2所示:
該文件是Pilot是Pilot-agent進程根據服務啟動參數以及Kubernetes Server API的信息生成,配置文件框架如圖3所示:
Node包含的信息如圖4所示:
這裏重點說明cluster屬性,該屬性值為productpage,說明當前的istio-proxy服務為productpage應用的代理服務,iptables將調用productpage服務請求攔截後,直接轉發到該istio-proxy進行處理。
stats_config包含的信息如圖5所示:
static_config重點設定部分屬性的命名規則。
admin包含的信息如圖6所示:
admin類似Envoy內部的一個管理器,暴露出的15000端口只能在容器內部被訪問。
通過curl http://127.0.0.1:15000/help訪問能獲取其他接口信息,如下圖7所示:
dynamic_resources包含的信息如圖8所示:
dynamic_resources中記錄pilot調取XDS接口采用ADS聚合發現服務,這種方式能更好的保證從pilot上獲取到的數據的一致性。
static_resources包含的部分信息如圖9所示:
static_resources name屬性值為xds-grpc,與上述dynamic_resources模塊的信息裏cluster_name的值一致。同時xds-grpc對象信息中包括pilot組件的服務地址“istio-pilot.istio-system:15010”,即為istio-proxy從pilot獲取動態信息的服務地址。Istio-proxy(Envoy)通過該地址調取XDS接口,獲取服務信息。除此之外,static_resources中提供zipkin服務的地址,zipkin是用於服務分布式跟蹤。
綜上可知Envoy啟動時,pilot-agent進程生成Envoy啟動所需的配置文件,Envoy基於配置文件中攜帶的pilot地址信息動態獲取服務信息,並且基於pilot提供的XDS接口,動態獲取服務的listener,cluster,endpoint和route信息。
深度解析Istio系列之Istio-proxy初始化篇