Docker自动重启问题追踪和排查

1、线上边端环境Java服务部署在边端，docker老是重启

先看几个常用的docker指令

1.1 journalctl -u docker --since=2023-09-15【查看docker守护进程日志】

命令用于查看Docker服务的日志。该命令使用journalctl工具来访问系统的日志消息，特别是与Docker相关的日志，从哪一天日志看起，since后面跟着的是具体日期。

1.2 docker events --since=1694707200000【查看docker的事件日志】

docker events命令用于监听Docker守护进程产生的实时事件。从哪一天日志看起，since后面跟着的是unix时间戳。

1.3 docker exec -it xxx-zeienrio bash 【进入docker容器】

docker exec -it bash命令用于在正在运行的Docker容器中执行一个新的进程，即表示进度docker 容器，xxx-zeienrio为对应的容器名或者容器ID。

1.4 jstack -l 6【查看Java的堆栈信息】

jstack -l pid 命令主要用于查看Java服务的堆栈信息

1.5 uptime -s【显示系统最后一次启动的时间 】

1.6 last reboot【查看系统上次重启时间 】

1.7 查看docker事件日志

[root@YWMDLFJLTX-FK ~]# docker events --since=1564588800

2023-09-15T14:57:20.623211229+08:00 network connect 2496f8f09a28d9e92fff99f649ea230bbdc30f6e9fde3066c9ce6db1c9df1832 (container=eb929b4f91ed5c2c39b0db44b2cc376fa9557f97ede158fa7e955b0fe4238e5e, name=host, type=host)

2023-09-15T14:57:20.659141936+08:00 network connect deb5d60ba9a19242a50cd9a1d4909028b4ecba78ed6788a886890a5a2749ff87 (container=a4e405feb87fc5ba2597fcf674b3c48531b5f0e5be52fe3cf04dd653f67286f6, name=app_default, type=bridge)

2023-09-15T14:57:20.681931347+08:00 network connect 092be9f83bafa06f2d545bf5faa5c0bc149b618a3258908d0bbe99cc6a532123 (container=8a451c1f8e71ecd798f619bd7180d492cb9f3a256bb56e10fe3809837276bc91, name=bridge, type=bridge)

2023-09-15T14:57:20.768136858+08:00 container start eb929b4f91ed5c2c39b0db44b2cc376fa9557f97ede158fa7e955b0fe4238e5e (image=zeienrio-one, name=zeienrio-one)

2023-09-15T14:57:20.849794040+08:00 container start 8a451c1f8e71ecd798f619bd7180d492cb9f3a256bb56e10fe3809837276bc91 (image=zeienrio-two, name=zeienrio-two, org.label-schema.build-date=20201113, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.opencontainers.image.created=2020-11-13 00:00:00+00:00, org.opencontainers.image.licenses=GPL-2.0-only, org.opencontainers.image.title=CentOS Base Image, org.opencontainers.image.vendor=CentOS)

2023-09-15T14:57:20.872355502+08:00 container start a4e405feb87fc5ba2597fcf674b3c48531b5f0e5be52fe3cf04dd653f67286f6 (com.docker.compose.config-hash=0be7fa4a6bcdc7b51df6dc0653d62af1316a0716fd63fb4ffcedb88fe846c9f9, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=app, com.docker.compose.project.config_files=/app/docker-compose-all.yml, com.docker.compose.project.working_dir=/app, com.docker.compose.service=mysql, com.docker.compose.version=1.25.1, image=mysql:5.7, name=mysql)

2023-09-15T18:16:43.272024976+08:00 container exec_create: bash 8a451c1f8e71ecd798f619bd7180d492cb9f3a256bb56e10fe3809837276bc91 (execID=572af93d8c6a803b97c6a7a00e538e1571afe47b19adf7cad358e34347ced9d2, image=zeienrio-two, name=zeienrio-two, org.label-schema.build-date=20201113, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.opencontainers.image.created=2020-11-13 00:00:00+00:00, org.opencontainers.image.licenses=GPL-2.0-only, org.opencontainers.image.title=CentOS Base Image, org.opencontainers.image.vendor=CentOS)

2023-09-15T18:16:43.272219231+08:00 container exec_start: bash 8a451c1f8e71ecd798f619bd7180d492cb9f3a256bb56e10fe3809837276bc91 (execID=572af93d8c6a803b97c6a7a00e538e1571afe47b19adf7cad358e34347ced9d2, image=zeienrio-two, name=zeienrio-two, org.label-schema.build-date=20201113, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.opencontainers.image.created=2020-11-13 00:00:00+00:00, org.opencontainers.image.licenses=GPL-2.0-only, org.opencontainers.image.title=CentOS Base Image, org.opencontainers.image.vendor=CentOS)

2023-09-16T09:36:27.546830131+08:00 container kill 8a451c1f8e71ecd798f619bd7180d492cb9f3a256bb56e10fe3809837276bc91 (image=zeienrio-two, name=zeienrio-two, org.label-schema.build-date=20201113, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.opencontainers.image.created=2020-11-13 00:00:00+00:00, org.opencontainers.image.licenses=GPL-2.0-only, org.opencontainers.image.title=CentOS Base Image, org.opencontainers.image.vendor=CentOS, signal=15)

2023-09-16T09:36:27.949005032+08:00 container exec_die 8a451c1f8e71ecd798f619bd7180d492cb9f3a256bb56e10fe3809837276bc91 (execID=572af93d8c6a803b97c6a7a00e538e1571afe47b19adf7cad358e34347ced9d2, exitCode=137, image=zeienrio-two, name=zeienrio-two, org.label-schema.build-date=20201113, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.opencontainers.image.created=2020-11-13 00:00:00+00:00, org.opencontainers.image.licenses=GPL-2.0-only, org.opencontainers.image.title=CentOS Base Image, org.opencontainers.image.vendor=CentOS)

2023-09-16T09:36:28.073724172+08:00 network disconnect 092be9f83bafa06f2d545bf5faa5c0bc149b618a3258908d0bbe99cc6a532123 (container=8a451c1f8e71ecd798f619bd7180d492cb9f3a256bb56e10fe3809837276bc91, name=bridge, type=bridge)

2023-09-16T09:36:28.076667004+08:00 container stop 8a451c1f8e71ecd798f619bd7180d492cb9f3a256bb56e10fe3809837276bc91 (image=zeienrio-two, name=zeienrio-two, org.label-schema.build-date=20201113, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.opencontainers.image.created=2020-11-13 00:00:00+00:00, org.opencontainers.image.licenses=GPL-2.0-only, org.opencontainers.image.title=CentOS Base Image, org.opencontainers.image.vendor=CentOS)

2023-09-16T09:36:28.080854936+08:00 container die 8a451c1f8e71ecd798f619bd7180d492cb9f3a256bb56e10fe3809837276bc91 (exitCode=143, image=zeienrio-two, name=zeienrio-two, org.label-schema.build-date=20201113, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.opencontainers.image.created=2020-11-13 00:00:00+00:00, org.opencontainers.image.licenses=GPL-2.0-only, org.opencontainers.image.title=CentOS Base Image, org.opencontainers.image.vendor=CentOS)

2023-09-16T09:36:28.132078759+08:00 network connect 092be9f83bafa06f2d545bf5faa5c0bc149b618a3258908d0bbe99cc6a532123 (container=8a451c1f8e71ecd798f619bd7180d492cb9f3a256bb56e10fe3809837276bc91, name=bridge, type=bridge)

2023-09-16T09:36:28.305777210+08:00 container start 8a451c1f8e71ecd798f619bd7180d492cb9f3a256bb56e10fe3809837276bc91 (image=zeienrio-two, name=zeienrio-two, org.label-schema.build-date=20201113, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.opencontainers.image.created=2020-11-13 00:00:00+00:00, org.opencontainers.image.licenses=GPL-2.0-only, org.opencontainers.image.title=CentOS Base Image, org.opencontainers.image.vendor=CentOS)

2023-09-16T09:36:28.305786602+08:00 container restart 8a451c1f8e71ecd798f619bd7180d492cb9f3a256bb56e10fe3809837276bc91 (image=zeienrio-two, name=zeienrio-two org.label-schema.build-date=20201113, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.opencontainers.image.created=2020-11-13 00:00:00+00:00, org.opencontainers.image.licenses=GPL-2.0-only, org.opencontainers.image.title=CentOS Base Image, org.opencontainers.image.vendor=CentOS)

2023-09-16T09:36:37.966765874+08:00 container kill 8a451c1f8e71ecd798f619bd7180d492cb9f3a256bb56e10fe3809837276bc91 (image=zeienrio-two, name=zeienrio-two, org.label-schema.build-date=20201113, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.opencontainers.image.created=2020-11-13 00:00:00+00:00, org.opencontainers.image.licenses=GPL-2.0-only, org.opencontainers.image.title=CentOS Base Image, org.opencontainers.image.vendor=CentOS, signal=15)

2023-09-16T09:36:38.441671727+08:00 network disconnect 092be9f83bafa06f2d545bf5faa5c0bc149b618a3258908d0bbe99cc6a532123 (container=8a451c1f8e71ecd798f619bd7180d492cb9f3a256bb56e10fe3809837276bc91, name=bridge, type=bridge)

2023-09-16T09:36:38.444753482+08:00 container stop 8a451c1f8e71ecd798f619bd7180d492cb9f3a256bb56e10fe3809837276bc91 (image=zeienrio-two name=zeienrio-two, org.label-schema.build-date=20201113, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.opencontainers.image.created=2020-11-13 00:00:00+00:00, org.opencontainers.image.licenses=GPL-2.0-only, org.opencontainers.image.title=CentOS Base Image, org.opencontainers.image.vendor=CentOS)

2023-09-16T09:36:38.449168714+08:00 container die 8a451c1f8e71ecd798f619bd7180d492cb9f3a256bb56e10fe3809837276bc91 (exitCode=143, image=zeienrio-two, name=zeienrio-two, org.label-schema.build-date=20201113, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.opencontainers.image.created=2020-11-13 00:00:00+00:00, org.opencontainers.image.licenses=GPL-2.0-only, org.opencontainers.image.title=CentOS Base Image, org.opencontainers.image.vendor=CentOS)

2023-09-16T09:36:38.499457234+08:00 network connect 092be9f83bafa06f2d545bf5faa5c0bc149b618a3258908d0bbe99cc6a532123 (container=8a451c1f8e71ecd798f619bd7180d492cb9f3a256bb56e10fe3809837276bc91, name=bridge, type=bridge)

2023-09-16T09:36:38.692380801+08:00 container start 8a451c1f8e71ecd798f619bd7180d492cb9f3a256bb56e10fe3809837276bc91 (image=zeienrio-two, name=zeienrio-two, org.label-schema.build-date=20201113, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.opencontainers.image.created=2020-11-13 00:00:00+00:00, org.opencontainers.image.licenses=GPL-2.0-only, org.opencontainers.image.title=CentOS Base Image, org.opencontainers.image.vendor=CentOS)

2023-09-16T09:36:38.692389283+08:00 container restart 8a451c1f8e71ecd798f619bd7180d492cb9f3a256bb56e10fe3809837276bc91 (image=zeienrio-two, name=zeienrio-two, org.label-schema.build-date=20201113, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.opencontainers.image.created=2020-11-13 00:00:00+00:00, org.opencontainers.image.licenses=GPL-2.0-only, org.opencontainers.image.title=CentOS Base Image, org.opencontainers.image.vendor=CentOS)

2023-09-18T16:48:41.861040775+08:00 container top 8a451c1f8e71ecd798f619bd7180d492cb9f3a256bb56e10fe3809837276bc91 (image=zeienrio-two, name=zeienrio-two, org.label-schema.build-date=20201113, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.opencontainers.image.created=2020-11-13 00:00:00+00:00, org.opencontainers.image.licenses=GPL-2.0-only, org.opencontainers.image.title=CentOS Base Image, org.opencontainers.image.vendor=CentOS)

2023-09-18T17:03:54.933743225+08:00 container exec_create: bash 8a451c1f8e71ecd798f619bd7180d492cb9f3a256bb56e10fe3809837276bc91 (execID=6189cf0f4f7cf8b8b6b5c32dd255d8382370de00c494149b478deb69b862c0d1, image=zeienrio-two, name=zeienrio-two, org.label-schema.build-date=20201113, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.opencontainers.image.created=2020-11-13 00:00:00+00:00, org.opencontainers.image.licenses=GPL-2.0-only, org.opencontainers.image.title=CentOS Base Image, org.opencontainers.image.vendor=CentOS)

2023-09-18T17:03:54.933997357+08:00 container exec_start: bash 8a451c1f8e71ecd798f619bd7180d492cb9f3a256bb56e10fe3809837276bc91 (execID=6189cf0f4f7cf8b8b6b5c32dd255d8382370de00c494149b478deb69b862c0d1, image=zeienrio-two, name=zeienrio-two, org.label-schema.build-date=20201113, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.opencontainers.image.created=2020-11-13 00:00:00+00:00, org.opencontainers.image.licenses=GPL-2.0-only, org.opencontainers.image.title=CentOS Base Image, org.opencontainers.image.vendor=CentOS)

1.8 查看docker守护进程的日志

[root@YWMDLFJLTX-FK ~]# journalctl -u docker.service

-- Logs begin at 五 2023-09-15 22:57:11 CST, end at 三 2023-09-20 09:27:54 CST. --

9月 15 14:57:19 YWMDLFJLTX-FK systemd[1]: Starting Docker Application Container Engine...

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.371482105+08:00" level=info msg="Starting up"

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.376013089+08:00" level=info msg="[core] [Channel #1] Channel created" module=grpc

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.376026011+08:00" level=info msg="[core] [Channel #1] original dial target is: \"unix:///run/containerd/containerd.sock\"" module=grpc

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.376063936+08:00" level=info msg="[core] [Channel #1] parsed dial target is: {Scheme:unix Authority: Endpoint:run/containerd/containerd.sock URL:{Scheme:unix Opaque: User: Host: Path:/run/containerd

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.376072930+08:00" level=info msg="[core] [Channel #1] Channel authority set to \"localhost\"" module=grpc

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.376137909+08:00" level=info msg="[core] [Channel #1] Resolver state updated: {\n \"Addresses\": [\n {\n \"Addr\": \"/run/containerd/containerd.sock\",\n \"ServerName\": \"\",\n \

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.376554099+08:00" level=info msg="[core] [Channel #1] Channel switches to new LB policy \"pick_first\"" module=grpc

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.377322202+08:00" level=info msg="[core] [Channel #1 SubChannel #2] Subchannel created" module=grpc

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.377486940+08:00" level=info msg="[core] [Channel #1 SubChannel #2] Subchannel Connectivity change to CONNECTING" module=grpc

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.377511484+08:00" level=info msg="[core] [Channel #1 SubChannel #2] Subchannel picks a new address \"/run/containerd/containerd.sock\" to connect" module=grpc

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.377514389+08:00" level=info msg="[core] [Channel #1] Channel Connectivity change to CONNECTING" module=grpc

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.378550750+08:00" level=info msg="[core] [Channel #1 SubChannel #2] Subchannel Connectivity change to READY" module=grpc

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.378564968+08:00" level=info msg="[core] [Channel #1] Channel Connectivity change to READY" module=grpc

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.380211597+08:00" level=info msg="[core] [Channel #4] Channel created" module=grpc

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.380221062+08:00" level=info msg="[core] [Channel #4] original dial target is: \"unix:///run/containerd/containerd.sock\"" module=grpc

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.380250299+08:00" level=info msg="[core] [Channel #4] parsed dial target is: {Scheme:unix Authority: Endpoint:run/containerd/containerd.sock URL:{Scheme:unix Opaque: User: Host: Path:/run/containerd

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.380261804+08:00" level=info msg="[core] [Channel #4] Channel authority set to \"localhost\"" module=grpc

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.380279568+08:00" level=info msg="[core] [Channel #4] Resolver state updated: {\n \"Addresses\": [\n {\n \"Addr\": \"/run/containerd/containerd.sock\",\n \"ServerName\": \"\",\n \

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.380293998+08:00" level=info msg="[core] [Channel #4] Channel switches to new LB policy \"pick_first\"" module=grpc

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.380308510+08:00" level=info msg="[core] [Channel #4 SubChannel #5] Subchannel created" module=grpc

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.380321116+08:00" level=info msg="[core] [Channel #4 SubChannel #5] Subchannel Connectivity change to CONNECTING" module=grpc

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.380334888+08:00" level=info msg="[core] [Channel #4 SubChannel #5] Subchannel picks a new address \"/run/containerd/containerd.sock\" to connect" module=grpc

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.380371196+08:00" level=info msg="[core] [Channel #4] Channel Connectivity change to CONNECTING" module=grpc

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.380446342+08:00" level=info msg="[core] [Channel #4 SubChannel #5] Subchannel Connectivity change to READY" module=grpc

9月 15 14:57:19 YWMDLFJLTX-FK dockerd[1422]: time="2023-09-15T14:57:19.380465196+08:00" level=info msg="[core] [Channel #4] Channel Connectivity change to READY" module=grpc

1.9 查看Java的堆栈信息

2023-09-15 18:22:19

Full thread dump OpenJDK 64-Bit Server VM (25.362-b08 mixed mode):

"Attach Listener" #800 daemon prio=9 os_prio=0 tid=0x00007fed60001000 nid=0x32e waiting on condition [0x0000000000000000]

java.lang.Thread.State: RUNNABLE

Locked ownable synchronizers:

- None

"C2 CompilerThread2" #10 daemon prio=9 os_prio=0 tid=0x00007fedf432e800 nid=0x2a waiting on condition [0x0000000000000000]

java.lang.Thread.State: RUNNABLE

Locked ownable synchronizers:

- None

"C2 CompilerThread1" #9 daemon prio=9 os_prio=0 tid=0x00007fedf432c800 nid=0x29 waiting on condition [0x0000000000000000]

java.lang.Thread.State: RUNNABLE

Locked ownable synchronizers:

- None

"C2 CompilerThread0" #8 daemon prio=9 os_prio=0 tid=0x00007fedf432a000 nid=0x28 waiting on condition [0x0000000000000000]

java.lang.Thread.State: RUNNABLE

Locked ownable synchronizers:

- None

"JDWP Event Helper Thread" #7 daemon prio=10 os_prio=0 tid=0x00007fedf4326000 nid=0x27 runnable [0x0000000000000000]

java.lang.Thread.State: RUNNABLE

Locked ownable synchronizers:

- None

"JDWP Transport Listener: dt_socket" #6 daemon prio=10 os_prio=0 tid=0x00007fedf4322800 nid=0x26 runnable [0x0000000000000000]

java.lang.Thread.State: RUNNABLE

Locked ownable synchronizers:

- None

"Signal Dispatcher" #5 daemon prio=9 os_prio=0 tid=0x00007fedf4303000 nid=0x25 runnable [0x0000000000000000]

java.lang.Thread.State: RUNNABLE

Locked ownable synchronizers:

- None

"Surrogate Locker Thread (Concurrent GC)" #4 daemon prio=9 os_prio=0 tid=0x00007fedf4301000 nid=0x24 waiting on condition [0x0000000000000000]

java.lang.Thread.State: RUNNABLE

Locked ownable synchronizers:

- None

2.0 docker事件错误码分析

2.0.1 错误码：exitCode=137

日志中出现的退出码：exitCode=137表示容器的退出代码为137。在Docker中，退出代码用于指示容器终止时的状态。

具体到exitCode=137，它通常表示容器被发送了一个SIGKILL信号，导致容器被强制终止。这可能是因为容器使用了过多的内存资源，或者超出了可分配给它的限制。

值得注意的是，137是由于信号9（SIGKILL）与128之和而得到的。Docker将信号9映射到容器的退出代码，并添加128作为基础值。

当你遇到exitCode=137时，你可以考虑增加容器的内存限制，或者检查容器内部是否存在内存泄漏或其他资源消耗过量的情况。此外，你还可以查看容器的日志以获取更多关于容器停止的信息。

2.0.2 错误码：exitCode=143

日志中出现的退出码：exitCode=143表示容器的退出代码为143。在Docker中，退出代码用于指示容器终止时的状态。

具体到exitCode=143，它通常表示容器被发送了一个SIGTERM信号，这是一个正常的终止信号。SIGTERM信号是Docker在尝试停止容器时发送的信号。容器收到SIGTERM信号后，有机会进行清理和优雅地关闭。

如果你看到exitCode=143，说明容器已经接收到停止信号并成功地完成了正常的终止操作。

请注意，与其他退出代码一样，143只是一个标识符，具体含义可能因上下文而异。你可以进一步查看容器的日志和事件来了解更多关于容器停止的详细信息。

2.0.3 网络连接失败

从日志可以看出，会出现网络连接失败的情况！

2.1 分析

1、docker events和journalctl -u docker.service的日志都显示了docker 重启了，并且将docker下面的服务也重新启动了。 2、从Java的堆栈信息上面看不出异常，而且从Java服务本身的服务日志，就是docker重启前后的日志也看不出任何异常，没有任何的内存溢出，数据库连接失败等信息，且Java服务本身的业务日志也是正常在打印的，故不是Java服务本身的问题。 3、从docker事件的日志中看到exitCode=137和exitCode=143是在docker重启以后出现的，所以也不是这个问题；exitCode=137和exitCode=143的问题，是因为，Java的堆内存启动参数设置的太小了，只有256M，后面调成了1G。 4、继续排查：开始的时候并没有想到项目上会断电啥的，然后经过同事提醒，去看下系统的重启时间，可以看到系统最后一次启动时间是：2023-09-15 14:57:11，再看下系统的启动时间日志，可以看到Sep 15 22:57 - 09:40 (4+10:42) 系统进行了重启【自动换算时区】，最后结论是系统重启导致的docker重启，docker重启导致的Java服务重启；

[root@YWMDLFJLTX-FK ~]# uptime -s

2023-09-15 14:57:11

[root@YWMDLFJLTX-FK ~]# last reboot

reboot system boot 3.10.0-1160.el7. Fri Sep 15 22:57 - 09:40 (4+10:42)

reboot system boot 3.10.0-1160.el7. Tue Sep 12 18:22 - 09:40 (7+15:17)

reboot system boot 3.10.0-1160.el7. Sun Sep 3 16:47 - 09:40 (16+16:52)

reboot system boot 3.10.0-1160.el7. Sun Sep 3 16:35 - 09:40 (16+17:04)

reboot system boot 3.10.0-1160.el7. Sun Aug 27 16:36 - 09:40 (23+17:03)

reboot system boot 3.10.0-1160.el7. Tue Aug 22 18:09 - 09:40 (28+15:30)

reboot system boot 3.10.0-1160.el7. Tue Aug 22 17:49 - 09:40 (28+15:50)

reboot system boot 3.10.0-1160.el7. Mon Aug 7 17:20 - 09:40 (43+16:19)

reboot system boot 3.10.0-1160.el7. Wed Jul 26 17:10 - 09:40 (55+16:29)

reboot system boot 3.10.0-1160.el7. Wed Jul 12 17:30 - 09:40 (69+16:09)

2、总结

以上就是自己本次问题排查的过程，仅以此记录，如有错误，欢迎指正！

参考阅读

 您阅读本篇文章共花了：