机器是rancherv2.5.9(单节点部署)下的一个集群的master节点,在电脑断电后,使用"systemctl start docker "重启docker服务发现无法启动,报错如下:
Sep 20 21:18:55 k8s-master-03 polkitd[681]: Registered Authentication Agent for unix-process:1553:34365 (system bus name :1.21 [/usr/bin/pkttyagent --notify-fd 5 --fallback], object path /org/freedesktop/PolicyKit1/AuthenticationAgent, l
Sep 20 21:18:55 k8s-master-03 systemd[1]: Starting Docker Application Container Engine...
-- Subject: Unit docker.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit docker.service has begun starting up.
Sep 20 21:18:55 k8s-master-03 dockerd[1559]: time="2021-09-20T21:18:55.467714267+08:00" level=info msg="Starting up"
Sep 20 21:18:55 k8s-master-03 dockerd[1559]: time="2021-09-20T21:18:55.468862178+08:00" level=info msg="parsed scheme: \"unix\"" module=grpc
Sep 20 21:18:55 k8s-master-03 dockerd[1559]: time="2021-09-20T21:18:55.468890040+08:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
Sep 20 21:18:55 k8s-master-03 dockerd[1559]: time="2021-09-20T21:18:55.468919616+08:00" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock <nil> 0 <nil>}] <nil> <nil>}" module=grpc
Sep 20 21:18:55 k8s-master-03 dockerd[1559]: time="2021-09-20T21:18:55.468931508+08:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
Sep 20 21:18:55 k8s-master-03 dockerd[1559]: time="2021-09-20T21:18:55.473481383+08:00" level=info msg="parsed scheme: \"unix\"" module=grpc
Sep 20 21:18:55 k8s-master-03 dockerd[1559]: time="2021-09-20T21:18:55.473516288+08:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
Sep 20 21:18:55 k8s-master-03 dockerd[1559]: time="2021-09-20T21:18:55.473536436+08:00" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock <nil> 0 <nil>}] <nil> <nil>}" module=grpc
Sep 20 21:18:55 k8s-master-03 dockerd[1559]: time="2021-09-20T21:18:55.473544771+08:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
Sep 20 21:18:55 k8s-master-03 dockerd[1559]: time="2021-09-20T21:18:55.475614489+08:00" level=warning msg="Usage of loopback devices is strongly discouraged for production use. Please use `--storage-opt dm.thinpooldev` or use `man docker
Sep 20 21:18:55 k8s-master-03 dockerd[1559]: time="2021-09-20T21:18:55.496878502+08:00" level=warning msg="Base device already exists and has filesystem ext4 on it. User specified filesystem will be ignored." storage-driver=devicemapper
Sep 20 21:18:55 k8s-master-03 dockerd[1559]: time="2021-09-20T21:18:55.505754547+08:00" level=info msg="[graphdriver] using prior storage driver: devicemapper"
Sep 20 21:18:55 k8s-master-03 dockerd[1559]: time="2021-09-20T21:18:55.505811243+08:00" level=warning msg="[graphdriver] WARNING: the devicemapper storage-driver is deprecated, and will be removed in a future release"
Sep 20 21:18:55 k8s-master-03 kernel: XFS (dm-0): Internal error XFS_WANT_CORRUPTED_RETURN at line 414 of file fs/xfs/libxfs/xfs_alloc.c. Caller xfs_alloc_ag_vextent_exact+0x153/0x2a0 [xfs]
Sep 20 21:18:55 k8s-master-03 kernel: CPU: 1 PID: 1564 Comm: dockerd Kdump: loaded Not tainted 3.10.0-1160.el7.x86_64 #1
Sep 20 21:18:55 k8s-master-03 kernel: Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/22/2020
Sep 20 21:18:55 k8s-master-03 kernel: Call Trace:
Sep 20 21:18:55 k8s-master-03 kernel: [<ffffffffb1f81340>] dump_stack+0x19/0x1b
Sep 20 21:18:55 k8s-master-03 kernel: [<ffffffffc060e52b>] xfs_error_report+0x3b/0x40 [xfs]
Sep 20 21:18:55 k8s-master-03 kernel: [<ffffffffc05cdad3>] ? xfs_alloc_ag_vextent_exact+0x153/0x2a0 [xfs]
Sep 20 21:18:55 k8s-master-03 kernel: [<ffffffffc05cac04>] xfs_alloc_fixup_trees+0x2c4/0x370 [xfs]
Sep 20 21:18:55 k8s-master-03 kernel: [<ffffffffc05cdad3>] xfs_alloc_ag_vextent_exact+0x153/0x2a0 [xfs]
Sep 20 21:18:55 k8s-master-03 kernel: [<ffffffffc05cdd1d>] xfs_alloc_ag_vextent+0xfd/0x150 [xfs]
Sep 20 21:18:55 k8s-master-03 kernel: [<ffffffffc05ce745>] xfs_alloc_vextent+0x2d5/0x510 [xfs]
Sep 20 21:18:55 k8s-master-03 kernel: [<ffffffffc05fa3a9>] xfs_ialloc_ag_alloc+0x159/0x670 [xfs]
Sep 20 21:18:55 k8s-master-03 kernel: [<ffffffffc063ca18>] ? _xfs_trans_bjoin+0x48/0x60 [xfs]
Sep 20 21:18:55 k8s-master-03 kernel: [<ffffffffc05fb989>] xfs_dialloc+0x119/0x280 [xfs]
Sep 20 21:18:55 k8s-master-03 kernel: [<ffffffffc061ce2d>] xfs_ialloc+0x6d/0x550 [xfs]
Sep 20 21:18:55 k8s-master-03 kernel: [<ffffffffc062c4b4>] ? xlog_grant_head_check+0x54/0x100 [xfs]
Sep 20 21:18:55 k8s-master-03 kernel: [<ffffffffc061eda7>] xfs_dir_ialloc+0x67/0x1c0 [xfs]
Sep 20 21:18:55 k8s-master-03 kernel: [<ffffffffb1f86482>] ? down_write+0x12/0x3d
Sep 20 21:18:55 k8s-master-03 kernel: [<ffffffffc061f352>] xfs_create+0x452/0x6a0 [xfs]
Sep 20 21:18:55 k8s-master-03 kernel: [<ffffffffc061ca03>] xfs_generic_create+0xd3/0x2a0 [xfs]
Sep 20 21:18:55 k8s-master-03 kernel: [<ffffffffc061cc04>] xfs_vn_mknod+0x14/0x20 [xfs]
Sep 20 21:18:55 k8s-master-03 kernel: [<ffffffffc061cc26>] xfs_vn_mkdir+0x16/0x20 [xfs]
Sep 20 21:18:55 k8s-master-03 kernel: [<ffffffffb1a5b03f>] vfs_mkdir+0x10f/0x1d0
Sep 20 21:18:55 k8s-master-03 kernel: [<ffffffffb1a6133a>] SyS_mkdirat+0xca/0x100
Sep 20 21:18:55 k8s-master-03 kernel: [<ffffffffb1f93f92>] system_call_fastpath+0x25/0x2a
Sep 20 21:18:55 k8s-master-03 dockerd[1559]: failed to start daemon: mkdir /var/lib/docker/image/devicemapper/distribution: structure needs cleaning
Sep 20 21:18:55 k8s-master-03 systemd[1]: docker.service: main process exited, code=exited, status=1/FAILURE
Sep 20 21:18:55 k8s-master-03 systemd[1]: Failed to start Docker Application Container Engine.
-- Subject: Unit docker.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit docker.service has failed.
--
-- The result is failed.
Sep 20 21:18:55 k8s-master-03 systemd[1]: Unit docker.service entered failed state.
Sep 20 21:18:55 k8s-master-03 systemd[1]: docker.service failed.
Sep 20 21:18:55 k8s-master-03 polkitd[681]: Unregistered Authentication Agent for unix-process:1553:34365 (system bus name :1.21, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) (disconnected from bus)
有人知道什么原因吗
嗯,如下解決辦法可以參考:
/var/lib/docker/image/devicemapper/distribution :structure needs cleaning
試試