微服務之分布式跟蹤系統(springboot+zipkin+mysql)
通過上一節《微服務之分布式跟蹤系統(springboot+zipkin)》我們簡單熟悉了zipkin的使用,但是收集的數據都保存在內存中重啟後數據丟失,不過zipkin的Storage除了內存,還有Cassandra、MYSQL、ElasticSearch。
二、zipkin的各種Storage配置簡介
zipkin存在一些公用的配置,同時存在一些私有的配置(詳細信息地址為:https://github.com/openzipkin/zipkin/tree/master/zipkin-server#configuration-for-the-ui),此處不做配置說明的翻譯(因為都比較簡單易懂),其公用的配置如下所示:
- *`QUERY_PORT`: Listen port for the http api and web ui; Defaults to 9411
- *`QUERY_LOG_LEVEL`: Log level written to the console; Defaults to INFO
- *`QUERY_LOOKBACK`: How many milliseconds queries can look back from endTs;Defaults to 7 days
- *`STORAGE_TYPE`: SpanStore implementation: one of `mem`, `mysql`, `cassandra`,`elasticsearch`
- *`COLLECTOR_PORT`: Listen port for the scribe thrift api; Defaults to 9410
- *`COLLECTOR_SAMPLE_RATE`: Percentage of traces to retain, defaults to alwayssample (1.0).
(1)Cassandra Storage配置
- * `CASSANDRA_KEYSPACE`: The keyspace to use. Defaults to "zipkin".
- * `CASSANDRA_CONTACT_POINTS`: Comma separated list of hosts / ip addresses part of Cassandra cluster. Defaults to localhost
- * `CASSANDRA_LOCAL_DC`: Name of the datacenter that will be considered "local" for latency load balancing. When unset, load-balancing is round-robin.
- * `CASSANDRA_ENSURE_SCHEMA`: Ensuring cassandra has the latest schema. If enabled tries to execute scripts in the classpath prefixed with `cassandra-schema-cql3`. Defaults to true
- * `CASSANDRA_USERNAME` and `CASSANDRA_PASSWORD`: Cassandra authentication. Will throw an exception on startup if authentication fails. No default
- * `CASSANDRA_USE_SSL`: Requires `javax.net.ssl.trustStore` and `javax.net.ssl.trustStorePassword`, defaults to false.
(2)MySQL Storage配置
- * `MYSQL_DB`: The database to use. Defaults to "zipkin".
- * `MYSQL_USER` and `MYSQL_PASS`: MySQL authentication, which defaults to empty string.
- * `MYSQL_HOST`: Defaults to localhost
- * `MYSQL_TCP_PORT`: Defaults to 3306
- * `MYSQL_MAX_CONNECTIONS`: Maximum concurrent connections, defaults to 10
- * `MYSQL_USE_SSL`: Requires `javax.net.ssl.trustStore` and `javax.net.ssl.trustStorePassword`, defaults to false.
(3)Elasticsearch Storage配置
- * `ES_CLUSTER`: The name of the elasticsearch cluster to connect to. Defaults to "elasticsearch".
- * `ES_HOSTS`: A comma separated list of elasticsearch hostnodes to connect to. When in host:port
- format, they should use the transport port, not the http port. To use http, specify
- base urls, ex. http://host:9200. Defaults to "localhost:9300". When not using http,
- Only one of the hosts needs to be available to fetch the remaining nodes in the
- cluster. It is recommended to set this to all the master nodes of the cluster.
- If the http URL is an AWS-hosted elasticsearch installation (e.g.
- https://search-domain-xyzzy.us-west-2.es.amazonaws.com) then Zipkin will attempt to
- use the default AWS credential provider (env variables, system properties, config
- files, or ec2 profiles) to sign outbound requests to the cluster.
- * `ES_PIPELINE`: Only valid when the destination is Elasticsearch 5.x. Indicates the ingest
- pipeline used before spans are indexed. No default.
- * `ES_MAX_REQUESTS`: Only valid when the transport is http. Sets maximum in-flight requests from
- this process to any Elasticsearch host. Defaults to 64.
- * `ES_AWS_DOMAIN`: The name of the AWS-hosted elasticsearch domain to use. Supercedes any set
- `ES_HOSTS`. Triggers the same request signing behavior as with `ES_HOSTS`, but
- requires the additional IAM permission to describe the given domain.
- * `ES_AWS_REGION`: An optional override to the default region lookup to search for the domain
- given in `ES_AWS_DOMAIN`. Ignored if only `ES_HOSTS` is present.
- * `ES_INDEX`: The index prefix to use when generating daily index names. Defaults to zipkin.
- * `ES_DATE_SEPARATOR`: The date separator to use when generating daily index names. Defaults to ‘-‘.
- * `ES_INDEX_SHARDS`: The number of shards to split the index into. Each shard and its replicas
- are assigned to a machine in the cluster. Increasing the number of shards
- and machines in the cluster will improve read and write performance. Number
- of shards cannot be changed for existing indices, but new daily indices
- will pick up changes to the setting. Defaults to 5.
三、zipkin環境準備與啟動
在本節中,以MySQL為例進行說明,由於目前只是Mysql5.6和5.7進行測試過,所以本次我選擇Mysql5.7版本。
(1) 初始化數據庫
安裝好Mysql5.7後新建zipkin的數據庫,然後執行下面的SQL語句新建表:
- CREATETABLE IF NOT EXISTS zipkin_spans (
- `trace_id_high` BIGINT NOT NULL DEFAULT 0COMMENT ‘If non zero, this means the trace uses 128 bit traceIds instead of 64bit‘,
- `trace_id` BIGINT NOT NULL,
- `id` BIGINT NOT NULL,
- `name` VARCHAR(255) NOT NULL,
- `parent_id` BIGINT,
- `debug` BIT(1),
- `start_ts` BIGINT COMMENT ‘Span.timestamp():epoch micros used for endTs query and to implement TTL‘,
- `duration` BIGINT COMMENT ‘Span.duration():micros used for minDuration and maxDuration query‘
- )ENGINE=InnoDB ROW_FORMAT=COMPRESSED CHARACTER SET=utf8 COLLATE utf8_general_ci;
- ALTERTABLE zipkin_spans ADD UNIQUE KEY(`trace_id_high`, `trace_id`, `id`) COMMENT‘ignore insert on duplicate‘;
- ALTERTABLE zipkin_spans ADD INDEX(`trace_id_high`, `trace_id`, `id`) COMMENT ‘forjoining with zipkin_annotations‘;
- ALTERTABLE zipkin_spans ADD INDEX(`trace_id_high`, `trace_id`) COMMENT ‘forgetTracesByIds‘;
- ALTERTABLE zipkin_spans ADD INDEX(`name`) COMMENT ‘for getTraces and getSpanNames‘;
- ALTERTABLE zipkin_spans ADD INDEX(`start_ts`) COMMENT ‘for getTraces ordering andrange‘;
- CREATETABLE IF NOT EXISTS zipkin_annotations (
- `trace_id_high` BIGINT NOT NULL DEFAULT 0COMMENT ‘If non zero, this means the trace uses 128 bit traceIds instead of 64bit‘,
- `trace_id` BIGINT NOT NULL COMMENT ‘coincideswith zipkin_spans.trace_id‘,
- `span_id` BIGINT NOT NULL COMMENT ‘coincideswith zipkin_spans.id‘,
- `a_key` VARCHAR(255) NOT NULL COMMENT‘BinaryAnnotation.key or Annotation.value if type == -1‘,
- `a_value` BLOB COMMENT‘BinaryAnnotation.value(), which must be smaller than 64KB‘,
- `a_type` INT NOT NULL COMMENT‘BinaryAnnotation.type() or -1 if Annotation‘,
- `a_timestamp` BIGINT COMMENT ‘Used toimplement TTL; Annotation.timestamp or zipkin_spans.timestamp‘,
- `endpoint_ipv4` INT COMMENT ‘Null whenBinary/Annotation.endpoint is null‘,
- `endpoint_ipv6` BINARY(16) COMMENT ‘Null whenBinary/Annotation.endpoint is null, or no IPv6 address‘,
- `endpoint_port` SMALLINT COMMENT ‘Null whenBinary/Annotation.endpoint is null‘,
- `endpoint_service_name` VARCHAR(255) COMMENT‘Null when Binary/Annotation.endpoint is null‘
- )ENGINE=InnoDB ROW_FORMAT=COMPRESSED CHARACTER SET=utf8 COLLATE utf8_general_ci;
- ALTERTABLE zipkin_annotations ADD UNIQUE KEY(`trace_id_high`, `trace_id`, `span_id`,`a_key`, `a_timestamp`) COMMENT ‘Ignore insert on duplicate‘;
- ALTERTABLE zipkin_annotations ADD INDEX(`trace_id_high`, `trace_id`, `span_id`)COMMENT ‘for joining with zipkin_spans‘;
- ALTERTABLE zipkin_annotations ADD INDEX(`trace_id_high`, `trace_id`) COMMENT ‘forgetTraces/ByIds‘;
- ALTERTABLE zipkin_annotations ADD INDEX(`endpoint_service_name`) COMMENT ‘forgetTraces and getServiceNames‘;
- ALTERTABLE zipkin_annotations ADD INDEX(`a_type`) COMMENT ‘for getTraces‘;
- ALTERTABLE zipkin_annotations ADD INDEX(`a_key`) COMMENT ‘for getTraces‘;
- ALTERTABLE zipkin_annotations ADD INDEX(`trace_id`, `span_id`, `a_key`) COMMENT ‘fordependencies job‘;
- CREATETABLE IF NOT EXISTS zipkin_dependencies (
- `day` DATE NOT NULL,
- `parent` VARCHAR(255) NOT NULL,
- `child` VARCHAR(255) NOT NULL,
- `call_count` BIGINT
- )ENGINE=InnoDB ROW_FORMAT=COMPRESSED CHARACTER SET=utf8 COLLATE utf8_general_ci;
- ALTERTABLE zipkin_dependencies ADD UNIQUE KEY(`day`, `parent`, `child`);
(2) 啟動實例
執行命令:java -jar zipkin-server-1.17.1-exec.jar --STORAGE_TYPE=mysql--MYSQL_DB=zipkin --MYSQL_USER=root --MYSQL_PASS=root --MYSQL_HOST=localhost--MYSQL_TCP_PORT=3306,啟動成功如下圖所示:
(3) 查看運行效果
通過上圖,我們發現zipkin使用springboot,並且啟動的端口為9411,然後我們通過瀏覽器訪問,效果如下:
四、分布式跟蹤系統實踐(springboot+zipkin+mysql)
4.1場景設置與分析
現在有一個服務A調用服務B,服務B又分別調用服務C和D,整個鏈路過程的關系圖如下所示:
4.2 代碼編寫
具體代碼和上一節代碼相同,源代碼下載地址:https://github.com/dreamerkr/mircoservice.git文件夾springboot+zipkin下面。
4.3運行效果
(1)分別啟動每個服務,然後訪問服務1,瀏覽器訪問(http://localhost:8081/service1/test)
(2)輸入zipkin地址,每次trace的列表
點擊其中的trace,可以看trace的樹形結構,包括每個服務所消耗的時間:
點擊每個span可以獲取延遲信息:
同時可以查看服務之間的依賴關系:
同時查看zipkin數據庫表已經存在數據:
微服務之分布式跟蹤系統(springboot+zipkin+mysql)