docker 原理之本地儲存
導讀
> 在前面的文章docker 原理之儲存驅動中簡單的介紹了 Docker 的儲存驅動,這篇文章接著講儲存,目前的 docker 版本中預設的是overlay2
,所以這篇文章就以overlay2
為例帶大家看看,在我們執行docker build
,docker pull
,docker run
等命令時本地儲存有何變化。 > > 這篇文章比較長,如果看不完可以收藏起來後續需要用到的時候再查閱,稱的上是乾貨滿滿,作者自己整理也花了較長的時間。 > > 原文:https://russellgao.cn/docker-local-storage/
背景
- 檢視 docker
Storage Driver
docker info | grep "Storage Driver"
命令。 - docker 的預設安裝目錄為:
/var/lib/docker
,如果要修改可以通過修改啟動時的配置檔案(預設為/usr/lib/systemd/system/docker.service
) 中的ExecStart
,
檢視 docker 啟動時的配置檔案:
修改 docker 的儲存目錄:
修改(增加)--graph
即可。
本地目錄
[root@iZuf685opgs9oyozju9i2bZ docker]# ll
總用量 48
drwx------ 2 root root 4096 11月 11 08:49 builder
drwx--x--x 4 root root 4096 11月 11 08:49 buildkit
drwx------ 3 root root 4096 12月 2 09:25 containers
drwx------ 3 root root 4096 11月 11 08:49 image
drwxr-x--- 3 root root 4096 11月 11 08:49 network
drwx------ 9 root root 4096 12月 2 09:25 overlay2
drwx------ 4 root root 4096 11月 11 08:49 plugins
drwx------ 2 root root 4096 11月 11 08:49 runtimes
drwx------ 2 root root 4096 11月 11 08:49 swarm
drwx------ 2 root root 4096 11月 11 13:32 tmp
drwx------ 2 root root 4096 11月 11 08:49 trust
drwx------ 2 root root 4096 11月 11 08:49 volumes
可以用tree
進行展開
[root@iZuf685opgs9oyozju9i2bZ docker]# tree -L 2
.
├── builder
│ └── fscache.db
├── buildkit
│ ├── cache.db
│ ├── content
│ ├── executor
│ ├── metadata.db
│ └── snapshots.db
├── containers
│ └── 9bd6ac07a8c962e2403203e1c45f4fb54733f9953cf318b34fc3f155bf2c0c59
├── image
│ └── overlay2
├── network
│ └── files
├── overlay2
│ ├── 00b65b9c288df8c0ae7cdacba531a7dc5cb006e6c768e19ee36055717b782acc
│ ├── 1e53dddb1a0bb04ee4ebd24a8edb94b96e2fd471a72bf1b8608096b38cb16646
│ ├── 1e53dddb1a0bb04ee4ebd24a8edb94b96e2fd471a72bf1b8608096b38cb16646-init
│ ├── 5da215c4f218cbb1d9825fa111c21bf381dc35a9e6c7c6cd5c3ea952316031e4
│ ├── 8cbfb8b74c887e780747c8e6f4b3b9223a513ff6d69770bac16abb76da4e314f
│ ├── f1cf8b173467c98e08f3d276d7ccd8f9892c7c71dec2c4b335c39c6f175ae744
│ └── l
├── plugins
│ ├── storage
│ └── tmp
├── runtimes
├── swarm
├── tmp
├── trust
└── volumes
└── metadata.db
26 directories, 5 files
這篇文章以分析儲存為主,涉及到的目錄有image
,containers
,overlay2
,其他的目錄放在後續的文章討論。
在真正開始之前,先想想幾個問題(這也是我自己問我自己的問題) :
- docker build 的過程是怎樣的?
- docker pull 和 docker build 產生的映象存放在哪了?
- docker run 執行一個容器的時候過程是怎麼樣的?
帶著這些問題我們以一個例子進行說明russellgao/openresty:1.17.8.2-5-alpine
image
image 目錄主要存放的映象相關的資訊,我們執行docker pull russellgao/openresty:1.17.8.2-5-alpine
看看:
[root@iZuf685opgs9oyozju9i2bZ docker]# docker pull russellgao/openresty:1.17.8.2-5-alpine
1.17.8.2-5-alpine: Pulling from russellgao/openresty
df20fa9351a1: Already exists
5682af42731d: Pull complete
7c6cb2b54a9d: Pull complete
aa74dc345098: Pull complete
Digest: sha256:224ced85b5f8b679a8664a39b69c1b8feb09f8ba4343d834bd5b69433081389e
Status: Downloaded newer image for openresty/openresty:1.17.8.2-5-alpine
可以看到 pull 了 4 層下來了,我們看看image
目錄:
[root@iZuf685opgs9oyozju9i2bZ docker]# tree image
image
└── overlay2
├── distribution
│ ├── diffid-by-digest
│ │ └── sha256
│ │ ├── 5682af42731d652bd98d2456ed3da4f0595ed5d9e5b13ac8bb9590bb74f72eb8
│ │ ├── 7c6cb2b54a9d9d40c4a03dd6615b1c8e791feb5d81464a7702a9bb921f7a73e9
│ │ ├── aa74dc3450985aee599c181d650da8f8880ca1d6e2bc01a43831ca59b6e2a7b6
│ │ └── df20fa9351a15782c64e6dddb2d4a6f50bf6d3688060a34c4014b0d9a752eb4c
│ └── v2metadata-by-diffid
│ └── sha256
│ ├── 1680a9f16b18732726d0656b6d6ff9611a3c4460ca870827b537a87bbe10cc22
│ ├── 50644c29ef5a27c9a40c393a73ece2479de78325cae7d762ef3cdc19bf42dd0a
│ ├── 8521b614863046bf4bb604e3586feeca8b7ce1372f1d6664a5545e85ad9ca472
│ └── 9c572ba82b91e3ac35c7351bdacc6876c67f5d9bc69c5e51e8b2deeafae95e4f
├── imagedb
│ ├── content
│ │ └── sha256
│ │ └── 1ddc7a18ba0bcc20c61447f391bfff98ac559eea590e7ac59b5b5f588f1f47ed
│ └── metadata
│ └── sha256
│ └── 1ddc7a18ba0bcc20c61447f391bfff98ac559eea590e7ac59b5b5f588f1f47ed
│ └── lastUpdated
├── layerdb
│ ├── mounts
│ │ └── 9bd6ac07a8c962e2403203e1c45f4fb54733f9953cf318b34fc3f155bf2c0c59
│ │ ├── init-id
│ │ ├── mount-id
│ │ └── parent
│ ├── sha256
│ │ ├── 228fb92e31891f472e9857ee11d13c404ff7c88e808b05ce4ebdc80d785d71f3
│ │ │ ├── cache-id
│ │ │ ├── diff
│ │ │ ├── parent
│ │ │ ├── size
│ │ │ └── tar-split.json.gz
│ │ ├── 50644c29ef5a27c9a40c393a73ece2479de78325cae7d762ef3cdc19bf42dd0a
│ │ │ ├── cache-id
│ │ │ ├── diff
│ │ │ ├── size
│ │ │ └── tar-split.json.gz
│ │ ├── 5f72760956a669e4c9b33aa3f2f04baa84b0f4cf1e11676049981bafcbba74da
│ │ │ ├── cache-id
│ │ │ ├── diff
│ │ │ ├── parent
│ │ │ ├── size
│ │ │ └── tar-split.json.gz
│ │ └── fe267088885017d5e9a4621e68617a7f35e58dc2d0d747927882da21059854e3
│ │ ├── cache-id
│ │ ├── diff
│ │ ├── parent
│ │ ├── size
│ │ └── tar-split.json.gz
│ └── tmp
└── repositories.json
21 directories, 33 files
看看repositories.json中是內容 :
[root@iZuf685opgs9oyozju9i2bZ docker]# cat image/overlay2/repositories.json | jq .
{
"Repositories": {
"openresty/openresty": {
"openresty/openresty:1.17.8.2-5-alpine": "sha256:1ddc7a18ba0bcc20c61447f391bfff98ac559eea590e7ac59b5b5f588f1f47ed",
"openresty/openresty@sha256:224ced85b5f8b679a8664a39b69c1b8feb09f8ba4343d834bd5b69433081389e": "sha256:1ddc7a18ba0bcc20c61447f391bfff98ac559eea590e7ac59b5b5f588f1f47ed"
},
"russellgao/openresty": {
"russellgao/openresty:1.17.8.2-5-alpine": "sha256:1ddc7a18ba0bcc20c61447f391bfff98ac559eea590e7ac59b5b5f588f1f47ed",
"russellgao/openresty@sha256:84f53dc7517e9b6695fc8fd74916a1eb5970a92fc24a984f99bfb81508f3d261": "sha256:1ddc7a18ba0bcc20c61447f391bfff98ac559eea590e7ac59b5b5f588f1f47ed"
}
}
}
repositories.json中記錄了這個機器上所有的映象,可以看到這裡有兩個映象openresty/openresty:1.17.8.2-5-alpine
和russellgao/openresty:1.17.8.2-5-alpine
,但其實只有一個映象,因為後面的imageid
是相同的,這個可以docker images
驗證一下
[root@iZuf685opgs9oyozju9i2bZ docker]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
openresty/openresty 1.17.8.2-5-alpine 1ddc7a18ba0b 2 months ago 104MB
russellgao/openresty 1.17.8.2-5-alpine 1ddc7a18ba0b 2 months ago 104MB
openresty/openresty:1.17.8.2-5-alpine
和russellgao/openresty:1.17.8.2-5-alpine
只是映象1ddc7a18ba0b
的 tag 。
那麼1ddc7a18ba0b
映象是怎麼組成的呢?image/overlay2/
下面除了 repositories.json 還有3個目錄distribution
,imagedb
,layerdb
,作用分別如下:
- distribution: 主要和映象倉庫的互動相關
- imagedb: 儲存了映象的元資料
- layerdb: 儲存了映象layer(層) 的資料
image/overlay2/
儲存的是資料的連結,真正的映象資料是存放在overlay2
目錄下,先看看distribution
:
distribution
[root@iZuf685opgs9oyozju9i2bZ docker]# tree image/overlay2/distribution/
image/overlay2/distribution/
├── diffid-by-digest
│ └── sha256
│ ├── 5682af42731d652bd98d2456ed3da4f0595ed5d9e5b13ac8bb9590bb74f72eb8
│ ├── 7c6cb2b54a9d9d40c4a03dd6615b1c8e791feb5d81464a7702a9bb921f7a73e9
│ ├── aa74dc3450985aee599c181d650da8f8880ca1d6e2bc01a43831ca59b6e2a7b6
│ └── df20fa9351a15782c64e6dddb2d4a6f50bf6d3688060a34c4014b0d9a752eb4c
└── v2metadata-by-diffid
└── sha256
├── 1680a9f16b18732726d0656b6d6ff9611a3c4460ca870827b537a87bbe10cc22
├── 50644c29ef5a27c9a40c393a73ece2479de78325cae7d762ef3cdc19bf42dd0a
├── 8521b614863046bf4bb604e3586feeca8b7ce1372f1d6664a5545e85ad9ca472
└── 9c572ba82b91e3ac35c7351bdacc6876c67f5d9bc69c5e51e8b2deeafae95e4f
4 directories, 8 files
請注意看image/overlay2/distribution/diffid-by-digest/sha256
下面,回過頭再看看 docker pull 的過程,這裡的就是digestid
,docker pull 的時候也是通過digestid
實現的,這個id對應的是docker repository
中的blob id
,在docker repository
的blobs
目錄下可以找到。
可以檢視具體的檔案,如cat image/overlay2/distribution/diffid-by-digest/sha256/5682af42731d652bd98d2456ed3da4f0595ed5d9e5b13ac8bb9590bb74f72eb8
[root@iZuf685opgs9oyozju9i2bZ docker]# cat image/overlay2/distribution/diffid-by-digest/sha256/5682af42731d652bd98d2456ed3da4f0595ed5d9e5b13ac8bb9590bb74f72eb8
sha256:9c572ba82b91e3ac35c7351bdacc6876c67f5d9bc69c5e51e8b2deeafae95e4f
不難發現它們之間是相互引用的,可以實現diffid
和digest
的相互轉換。
imagedb
[root@iZuf685opgs9oyozju9i2bZ docker]# tree image/overlay2/imagedb/
image/overlay2/imagedb/
├── content
│ └── sha256
│ └── 1ddc7a18ba0bcc20c61447f391bfff98ac559eea590e7ac59b5b5f588f1f47ed
└── metadata
└── sha256
└── 1ddc7a18ba0bcc20c61447f391bfff98ac559eea590e7ac59b5b5f588f1f47ed
└── lastUpdated
5 directories, 2 files
可以看到imagedb
是以映象為單位進行儲存的,看一下content
下面的具體內容 :
[root@iZuf685opgs9oyozju9i2bZ docker]# cat image/overlay2/imagedb/content/sha256/1ddc7a18ba0bcc20c61447f391bfff98ac559eea590e7ac59b5b5f588f1f47ed | jq .
{
"architecture": "amd64",
"config": {
"Hostname": "",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/openresty/luajit/bin:/usr/local/openresty/nginx/sbin:/usr/local/openresty/bin"
],
"Cmd": [
"/usr/local/openresty/bin/openresty",
"-g",
"daemon off;"
],
"ArgsEscaped": true,
"Image": "sha256:0b827067ad09ab8a0b9a73a45f5b1c408b84db1ca6883a4c544078ed43b8b5e3",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": null,
"OnBuild": null,
"Labels": {
"maintainer": "Evan Wies <[email protected]>",
"resty_add_package_builddeps": "",
"resty_add_package_rundeps": "",
"resty_config_deps": "--with-pcre --with-cc-opt='-DNGX_LUA_ABORT_AT_PANIC -I/usr/local/openresty/pcre/include -I/usr/local/openresty/openssl/include' --with-ld-opt='-L/usr/local/openresty/pcre/lib -L/usr/local/openresty/openssl/lib -Wl,-rpath,/usr/local/openresty/pcre/lib:/usr/local/openresty/openssl/lib' ",
"resty_config_options": " --with-compat --with-file-aio --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_geoip_module=dynamic --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-http_xslt_module=dynamic --with-ipv6 --with-mail --with-mail_ssl_module --with-md5-asm --with-pcre-jit --with-sha1-asm --with-stream --with-stream_ssl_module --with-threads ",
"resty_config_options_more": "",
"resty_eval_post_make": "",
"resty_eval_pre_configure": "",
"resty_image_base": "alpine",
"resty_image_tag": "3.12",
"resty_openssl_patch_version": "1.1.1f",
"resty_openssl_url_base": "https://www.openssl.org/source",
"resty_openssl_version": "1.1.1g",
"resty_pcre_version": "8.44",
"resty_version": "1.17.8.2"
},
"StopSignal": "SIGQUIT"
},
"container": "0ae35046dd1afef0f1f525360939abc524dbd469a92470c5836dfbb7dc666923",
"container_config": {
"Hostname": "0ae35046dd1a",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/openresty/luajit/bin:/usr/local/openresty/nginx/sbin:/usr/local/openresty/bin"
],
"Cmd": [
"/bin/sh",
"-c",
"#(nop) ",
"STOPSIGNAL SIGQUIT"
],
"ArgsEscaped": true,
"Image": "sha256:0b827067ad09ab8a0b9a73a45f5b1c408b84db1ca6883a4c544078ed43b8b5e3",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": null,
"OnBuild": null,
"Labels": {
"maintainer": "Evan Wies <[email protected]>",
"resty_add_package_builddeps": "",
"resty_add_package_rundeps": "",
"resty_config_deps": "--with-pcre --with-cc-opt='-DNGX_LUA_ABORT_AT_PANIC -I/usr/local/openresty/pcre/include -I/usr/local/openresty/openssl/include' --with-ld-opt='-L/usr/local/openresty/pcre/lib -L/usr/local/openresty/openssl/lib -Wl,-rpath,/usr/local/openresty/pcre/lib:/usr/local/openresty/openssl/lib' ",
"resty_config_options": " --with-compat --with-file-aio --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_geoip_module=dynamic --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-http_xslt_module=dynamic --with-ipv6 --with-mail --with-mail_ssl_module --with-md5-asm --with-pcre-jit --with-sha1-asm --with-stream --with-stream_ssl_module --with-threads ",
"resty_config_options_more": "",
"resty_eval_post_make": "",
"resty_eval_pre_configure": "",
"resty_image_base": "alpine",
"resty_image_tag": "3.12",
"resty_openssl_patch_version": "1.1.1f",
"resty_openssl_url_base": "https://www.openssl.org/source",
"resty_openssl_version": "1.1.1g",
"resty_pcre_version": "8.44",
"resty_version": "1.17.8.2"
},
"StopSignal": "SIGQUIT"
},
"created": "2020-09-18T16:25:11.080239395Z",
"docker_version": "18.06.0-ce",
"history": [
{
"created": "2020-05-29T21:19:46.192045972Z",
"created_by": "/bin/sh -c #(nop) ADD file:c92c248239f8c7b9b3c067650954815f391b7bcb09023f984972c082ace2a8d0 in / "
},
{
"created": "2020-05-29T21:19:46.363518345Z",
"created_by": "/bin/sh -c #(nop) CMD [\"/bin/sh\"]",
"empty_layer": true
},
{
"created": "2020-09-18T16:18:11.417409263Z",
"created_by": "/bin/sh -c #(nop) LABEL maintainer=Evan Wies <[email protected]>",
"empty_layer": true
},
{
"created": "2020-09-18T16:18:11.510209288Z",
"created_by": "/bin/sh -c #(nop) ARG RESTY_IMAGE_BASE=alpine",
"empty_layer": true
},
{
"created": "2020-09-18T16:18:11.601623273Z",
"created_by": "/bin/sh -c #(nop) ARG RESTY_IMAGE_TAG=3.12",
"empty_layer": true
},
{
"created": "2020-09-18T16:18:11.688966243Z",
"created_by": "/bin/sh -c #(nop) ARG RESTY_VERSION=1.17.8.2",
"empty_layer": true
},
{
"created": "2020-09-18T16:18:11.783539793Z",
"created_by": "/bin/sh -c #(nop) ARG RESTY_OPENSSL_VERSION=1.1.1g",
"empty_layer": true
},
{
"created": "2020-09-18T16:18:11.885734193Z",
"created_by": "/bin/sh -c #(nop) ARG RESTY_OPENSSL_PATCH_VERSION=1.1.1f",
"empty_layer": true
},
{
"created": "2020-09-18T16:18:11.977916317Z",
"created_by": "/bin/sh -c #(nop) ARG RESTY_OPENSSL_URL_BASE=https://www.openssl.org/source",
"empty_layer": true
},
{
"created": "2020-09-18T16:18:12.075117786Z",
"created_by": "/bin/sh -c #(nop) ARG RESTY_PCRE_VERSION=8.44",
"empty_layer": true
},
{
"created": "2020-09-18T16:18:12.169065223Z",
"created_by": "/bin/sh -c #(nop) ARG RESTY_J=1",
"empty_layer": true
},
{
"created": "2020-09-18T16:18:12.268734856Z",
"created_by": "/bin/sh -c #(nop) ARG RESTY_CONFIG_OPTIONS= --with-compat --with-file-aio --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_geoip_module=dynamic --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-http_xslt_module=dynamic --with-ipv6 --with-mail --with-mail_ssl_module --with-md5-asm --with-pcre-jit --with-sha1-asm --with-stream --with-stream_ssl_module --with-threads ",
"empty_layer": true
},
{
"created": "2020-09-18T16:18:12.363332751Z",
"created_by": "/bin/sh -c #(nop) ARG RESTY_CONFIG_OPTIONS_MORE=",
"empty_layer": true
},
{
"created": "2020-09-18T16:18:12.462196367Z",
"created_by": "/bin/sh -c #(nop) ARG RESTY_LUAJIT_OPTIONS=--with-luajit-xcflags='-DLUAJIT_NUMMODE=2 -DLUAJIT_ENABLE_LUA52COMPAT'",
"empty_layer": true
},
{
"created": "2020-09-18T16:18:12.553726712Z",
"created_by": "/bin/sh -c #(nop) ARG RESTY_ADD_PACKAGE_BUILDDEPS=",
"empty_layer": true
},
{
"created": "2020-09-18T16:18:12.644512619Z",
"created_by": "/bin/sh -c #(nop) ARG RESTY_ADD_PACKAGE_RUNDEPS=",
"empty_layer":