相変わらずARM板でDockerしているわけですが、調子が悪くなってしまったので、再度環境を入れなおしておりました。
起動しなくなったり、途中でハングしたりするようになって、まったくうまくいっていないのですが、なんとかKernelの入れ替え等で対応してきました。
そして、Dockerですが、途中から以下のようなエラーが出るようになってしまいました。
ちなみに実行していた環境は以下の通り。
確認環境
$ docker version
Client:
Version: 18.05.0-ce
API version: 1.37
Go version: go1.9.5
Git commit: f150324
Built: Wed May 9 22:10:21 2018
OS/Arch: linux/arm64
Experimental: false
Orchestrator: swarm
Server:
Engine:
Version: 18.05.0-ce
API version: 1.37 (minimum version 1.12)
Go version: go1.9.5
Git commit: f150324
Built: Wed May 9 22:08:19 2018
OS/Arch: linux/arm64
Experimental: false症状
$ docker-compose up -d
ERROR: readlink /var/lib/docker/overlay2/l/Q4OSVLT53TOUIHGTVHBDO4UTFR: no such file or directory悪あがき
その１
overlay2のマウントポイントが正しく認識できていないっぽいです。ということで以下のよう対応をしました。
$ docker system pruneしかし、これでもうまくいきません。調べてみると以下のような情報が見つかりました。
その２
docker-compose: readlink /var/lib/docker/overlay2: invalid argument · Issue #1667 · docker/for-mac
Expected behavior docker-compose works Actual behavior $ docker-compose up ERROR: readlink /var/lib/docker/overlay2: inv...
github.com
No such file or directory for /var/lib/docker/overlay2 · Issue #1396 · docker/for-mac
Expected behavior Docker image builds without error Actual behavior Docker image encounters error no such file or direct...
github.com
とりあえず以下のようなコメントがあったので実行してみました。
Screen into the Docker VM
Remove/var/lib/docker
Restart Docker via the widget
$ sudo rm -rf /var/lib/docker/overlay2/$ docker-compose up -d --force-recreate
ERROR: stat /var/lib/docker/overlay2/f7e0deb141a9012fd1293568b893aeab817687d19158fe7ba575da9a26577e53: no such file or directoryダメなようです。
systemd側から見てみると以下のように問題がないように見えます。
● docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
Active: active (running) since Sun 2018-05-13 19:37:31 JST; 3s ago
Docs: https://docs.docker.com
Main PID: 4322 (dockerd)
Tasks: 20
CGroup: /system.slice/docker.service
├─4322 /usr/bin/dockerd -H fd://
└─4329 docker-containerd --config /var/run/docker/containerd/containerd.toml
May 13 19:37:29 dns02 dockerd[4322]: time="2018-05-13T19:37:29.993871009+09:00" level=error msg="Failed to l
May 13 19:37:29 dns02 dockerd[4322]: time="2018-05-13T19:37:29.997235922+09:00" level=error msg="Failed to l
May 13 19:37:30 dns02 dockerd[4322]: time="2018-05-13T19:37:30.000142633+09:00" level=error msg="Failed to l
May 13 19:37:30 dns02 dockerd[4322]: time="2018-05-13T19:37:30.365380524+09:00" level=warning msg="Could not
May 13 19:37:30 dns02 dockerd[4322]: time="2018-05-13T19:37:30.895801031+09:00" level=info msg="Default brid
May 13 19:37:31 dns02 dockerd[4322]: time="2018-05-13T19:37:31.252502539+09:00" level=info msg="Loading cont
May 13 19:37:31 dns02 dockerd[4322]: time="2018-05-13T19:37:31.395542412+09:00" level=info msg="Docker daemo
May 13 19:37:31 dns02 dockerd[4322]: time="2018-05-13T19:37:31.395842825+09:00" level=info msg="Daemon has c
May 13 19:37:31 dns02 dockerd[4322]: time="2018-05-13T19:37:31.431197309+09:00" level=info msg="API listen o
May 13 19:37:31 dns02 systemd[1]: Started Docker Application Container Engine.docker-compose upしても以下のような感じでエラーになります。やはりvolumeレイヤーが正しくないっぽいです。
$ docker-compose pull
Pulling dibbler-server ...
Pulling dnsdist ...
Pulling unbound ...
Pulling stubby ...
Pulling dnscrypt-proxy ...
Pulling nsd ...
Pulling kresd ...
Pulling bind9 ...
Pulling zabbix-agent ...
ERROR: for dibbler-server failed to register layer: lstat /var/lib/docker/overlay2/734b54acad32a1142e95d3f115270bd02de3661a8f092d50cc49abf464ae0a15: no such file or directory
ERROR: for dnsdist failed to register layer: lstat /var/lib/docker/overlay2/734b54acad32a1142e95d3f115270bd02de3661a8f092d50cc49abf464ae0a15: no such file or directory
ERROR: for unbound failed to register layer: lstat /var/lib/docker/overlay2/734b54acad32a1142e95d3f115270bd02de3661a8f092d50cc49abf464ae0a15: no such file or directory
ERROR: for stubby failed to register layer: lstat /var/lib/docker/overlay2/734b54acad32a1142e95d3f115270bd02de3661a8f092d50cc49abf464ae0a15: no such file or directory
ERROR: for dnscrypt-proxy failed to register layer: lstat /var/lib/docker/overlay2/734b54acad32a1142e95d3f115270bd02de3661a8f092d50cc49abf464ae0a15: no such file or directory
ERROR: for nsd failed to register layer: lstat /var/lib/docker/overlay2/734b54acad32a1142e95d3f115270bd02de3661a8f092d50cc49abf464ae0a15: no such file or directory
ERROR: for kresd failed to register layer: lstat /var/lib/docker/overlay2/734b54acad32a1142e95d3f115270bd02de3661a8f092d50cc49abf464ae0a15: no such file or directory
ERROR: for bind9 failed to register layer: lstat /var/lib/docker/overlay2/734b54acad32a1142e95d3f115270bd02de3661a8f092d50cc49abf464ae0a15: no such file or directory
ERROR: for zabbix-agent failed to register layer: lstat /var/lib/docker/overlay2/734b54acad32a1142e95d3f115270bd02de3661a8f092d50cc49abf464ae0a15: no such file or directory
ERROR: failed to register layer: lstat /var/lib/docker/overlay2/734b54acad32a1142e95d3f115270bd02de3661a8f092d50cc49abf464ae0a15: no such file or directoryその３
仕方がないので、一旦docker-ceを再インストールしました。んでdocker-compose upします。
$ docker-compose up -d
Pulling dibbler-server (kometchtech/dibbler-server:latest)...
latest: Pulling from kometchtech/dibbler-server
e629e6244a09: Already exists
aafc292a94ab: Already exists
08e31ee12525: Extracting [==================================================>] 116B/116B
7794b287d4af: Download complete
ERROR: failed to register layer: lstat /var/lib/docker/overlay2/734b54acad32a1142e95d3f115270bd02de3661a8f092d50cc49abf464ae0a15: no such file or directoryエラーになります。どうしてだろうと思って再度調べてみると、以下のようなissueが。
コレが解決策？
remove lingering containers
docker rm $(docker ps -a -q)remove all image
docker rmi $(docker images -q)stop docker service
service docker stop# or whatever: systctl …remove overlay2 files
sudo rm -rf /var/lib/docker/overlay2start docker service
https://github.com/docker/for-mac/issues/1396service docker start# or equally whatever: systcl…
ちゃんとイメージを削除してやる必要があるようです。というか、今までの作業では、それまでの情報が正しく削除できていなかったようです。というわけでコマンドを実行して、再度pullします。
$ docker rmi $(docker images -q)
$ docker-compose up -d --force-recreate
Pulling dibbler-server (kometchtech/dibbler-server:latest)...
latest: Pulling from kometchtech/dibbler-server
e629e6244a09: Pull complete
aafc292a94ab: Pull complete
08e31ee12525: Pull complete
7794b287d4af: Pull complete
Digest: sha256:318b2a1dda8a80e3d17cd375850002d0b4b274bdfce6aeaac775e9fbfc9de452
Status: Downloaded newer image for kometchtech/dibbler-server:latest
Pulling dnsdist (kometchtech/dnsdist:latest-alpine)...
latest-alpine: Pulling from kometchtech/dnsdist
e629e6244a09: Already exists
aafc292a94ab: Already exists
52d71dd2309e: Pull complete
Digest: sha256:629054422a58f0da02ac3b065665ada86415ee430763615e8dfb7123a3aadede
Status: Downloaded newer image for kometchtech/dnsdist:latest-alpine
Pulling unbound (kometchtech/unbound:latest-alpine)...
latest-alpine: Pulling from kometchtech/unbound
e629e6244a09: Already exists
aafc292a94ab: Already exists
8a17a3ee633b: Pull complete
3a1f7e47a627: Pull complete
8e7f7845d127: Pull complete
66ea0958cb6e: Pull complete
Digest: sha256:6b0e04927b4ea99d17d7573d22f53a17ec0af53b568f0ea41425159af7422d0d
Status: Downloaded newer image for kometchtech/unbound:latest-alpine
Pulling stubby (kometchtech/stubby:latest)...
latest: Pulling from kometchtech/stubby
e629e6244a09: Already exists
aafc292a94ab: Already exists
ea299f65ac09: Pull complete
c23329f2053a: Pull complete
Digest: sha256:0beb0550ad59f6f3d7f5e8357aa1dc6dd641b95efcf684a7cd406e590eddc752
Status: Downloaded newer image for kometchtech/stubby:latest
Pulling dnscrypt-proxy (kometchtech/dnscrypt-proxy:latest)...
latest: Pulling from kometchtech/dnscrypt-proxy
e629e6244a09: Already exists
aafc292a94ab: Already exists
a5c25d636686: Pull complete
Digest: sha256:5b6ef135a1fa3f625da7d6e0163722240d66cce9327540e554d0c7a3c1c83fb2
Status: Downloaded newer image for kometchtech/dnscrypt-proxy:latest
Pulling nsd (kometchtech/nsd:latest-alpine)...
latest-alpine: Pulling from kometchtech/nsd
e629e6244a09: Already exists
aafc292a94ab: Already exists
55657d61d0ed: Pull complete
374f1b0fe167: Pull complete
31c0ab897abd: Pull complete
8b125bba7dda: Pull complete
4195aa304628: Pull complete
Digest: sha256:08c558a255f382b1ef7beef4c99a16b209d6a5891c0dfd5010b647c5a928d57c
Status: Downloaded newer image for kometchtech/nsd:latest-alpine
Pulling kresd (kometchtech/kresd:latest)...
latest: Pulling from kometchtech/kresd
e629e6244a09: Already exists
aafc292a94ab: Already exists
f20790745fb8: Pull complete
d6cbe6a84ad9: Pull complete
Digest: sha256:f98aa227579bb56446d30614a6de91be56d7ef43a42a1456402ed9cff49a39a2
Status: Downloaded newer image for kometchtech/kresd:latest
Pulling bind9 (kometchtech/bind9:)...
latest: Pulling from kometchtech/bind9
e629e6244a09: Already exists
aafc292a94ab: Already exists
10c63bac784f: Pull complete
Digest: sha256:c8b6add3046f152bbc6ec067f41ae3b9a971aa4a7f6768094ddf0f9a008c42ef
Status: Downloaded newer image for kometchtech/bind9:latest
Pulling zabbix-agent (kometchtech/zabbix-agent:)...
latest: Pulling from kometchtech/zabbix-agent
e629e6244a09: Already exists
aafc292a94ab: Already exists
50a67c7be7c6: Pull complete
2ee103fc7c33: Pull complete
0aa98f8b5961: Pull complete
7e4c47f9849c: Pull complete
Digest: sha256:b97a306a77313706cf7942885ab3c5d408fba3a7a6c735a6cda9db1d9e1105ca
Status: Downloaded newer image for kometchtech/zabbix-agent:latest
Creating zabbix-agent ... done
Creating dibbler-server ... done
Creating dnsdist ... done
Creating bind9 ... done
Creating dnscrypt-proxy ... done
Creating kresd ... done
Creating nsd ... done
Creating stubby ... done
Creating unbound ... doneというわけで無事に起動するようになりました。
$ docker-compose ps
Name Command State Ports
------------------------------------------------------------------------------------------------------------
bind9 named -c /etc/bind/named.c ... Up 0.0.0.0:9853->53/tcp, 0.0.0.0:9853->53/udp
dibbler-server /usr/local/sbin/dibbler-se ... Up
dnscrypt-proxy dnscrypt-proxy -config /et ... Up 0.0.0.0:9653->53/tcp, 0.0.0.0:9653->53/udp
dnsdist dnsdist --supervised --dis ... Up
kresd /usr/local/sbin/kresd -f 4 ... Exit 1
nsd /usr/local/sbin/run.sh Up 0.0.0.0:10053->53/tcp, 0.0.0.0:10053->53/udp,
8952/tcp
stubby /usr/local/bin/stubby Up 0.0.0.0:8053->53/tcp, 0.0.0.0:8053->53/udp
unbound /usr/local/sbin/run.sh Up 0.0.0.0:9753->53/tcp, 0.0.0.0:9753->53/udp
zabbix-agent /usr/bin/docker-entrypoint.sh Up 0.0.0.0:10050->10050/tcpこのあたりはもう少し勉強が必要なようです。
