Troubleshooting and monitoring Docker
You learn to find why containers crash, slow down or eat resources, using logs, metrics and events.
Level 301 · 16 lessons · 15 practice labs
Lessons
Crashes and failed starts
- Exit codes and OOMKilled
- Breaking a restart loop
- Debugging an image without a shell
- Docker engine logs and daemon.json
- Short task: get a service that exits at once running again
Logs and metrics
- Log drivers and log rotation
- docker stats and docker events
- Container metrics with cAdvisor and Prometheus
- Shipping container logs to a central place
- Short task: find and limit the container slowing the others
Performance
- Tuning CPU and memory limits
- Disk I/O and layer problems
- Designing a proper health check
- Clock drift and DNS cache problems
- Short task: show the bottleneck of an app that slowed down in a container
Wrap-up
- Capstone: midnight alert: fix three faulty containers and report
Practice labs
- Three containers, three exit codes: Inspect the exit codes and state fields of three stopped containers and explain in one line each why it stopped (memory, command not found, application error).
- The report service that dies at night: Prove with inspect and kernel logs that Kuzey Logistics’ report service is killed at its memory limit, bring the limit and the app’s memory setting into line and show the job completes.
- A restart loop that never ends: Stop the loop of a service that keeps restarting because of a stale lock file and wrong permissions on its volume without harming data, fix both causes and show the service stays up.
- Look at a dead container’s file: Copy out and inspect the configuration file of a container you can’t exec into because it won’t start, find the bad line and start the container with the fixed file.
- Inspect a shell-less container with a helper: Start a helper container sharing the network and process namespaces of a container with no shell, find its listening ports and running process and report the wrong listen address.
- A Docker service that won’t start: Find the errors of a Docker service that won’t start because of a broken daemon.json and conflicting start options from the service logs, repair the configuration and show the containers come back.
- When and why did the container stop?: Find from the docker events history at what time and through which event a container stopped, and put the sequence into a short timeline.
- docker logs comes back empty: Find from the log driver setting why docker logs shows nothing, switch to the local driver with rotation and show new entries appear.
- Ship logs to a central place: Send three containers’ logs to the lab’s central log collector tagged with the service name, and prove multi-line error entries arrive unbroken and stay readable locally too.
- The container slowing the others: Find the container eating the resources with docker stats, limit it live without recreating it and show the other services’ response times recover.
- An API slowed by its own limit: Prove from cgroup counters that an API container with jittery response times is being CPU-throttled, tune the limit from measurements and show the throttle counter stops rising.
- A database drowning in its writable layer: Show by measurement that the slow app’s bottleneck is heavy disk writes into the container’s writable layer, move the data to a volume and compare timings under the same load.
- A health check that lies: Rewrite the health check of a container that looks healthy while the app returns errors so it tests the real endpoint, and show it turns unhealthy when the app is broken.
- A proxy still sending to the old address: Show the proxy keeps using the old IP after the backend container is recreated, make the setting that refreshes name resolution and verify by recreating the backend again.
- A slow starter that keeps getting killed: Prove the slow-starting service is killed again and again by its health check, measure its start-up time and set interval, timeout and start period accordingly.
About this course
You learn to find why containers crash, slow down or eat resources, using logs, metrics and events.
What you will learn
- Crashes and failed starts
- Logs and metrics
- Performance
- Wrap-up
Prerequisites
You should be comfortable with the level 101 and 201 topics of the same path.
Lab environment
Lessons run in your own Kali Linux terminal in the browser, connected to a private lab network. Nothing to install.
Frequently asked questions
- Is this course right for my level?
- The course is at level 301; it assumes you know the topics of the lower-level courses.
- Is there anything to install?
- No. Every lesson runs in your own lab environment that opens in the browser.
- How do lessons work?
- Each lesson has 8 to 10 steps and a short quiz. Early steps show the command to type, later steps give less help; if two attempts fail, the solution opens.
- Can I try it for free?
- Yes. The first three lessons of every course are free.