Linux performance tuning: CPU, memory and disk
You find the bottleneck with the USE method, tune the scheduler, memory, swap, I/O and kernel parameters by measurement, and prove the effect of every change.
Level 301 · 18 lessons · 15 practice labs
Lessons
Method
- The USE method: utilisation, saturation, errors
- Measuring properly: repetition, warm-up and noise
- Short task: fill in the USE checklist on a slow server
CPU
- The scheduler: CFS, priority and context switches
- CPU affinity: taskset and NUMA
- Frequency governors and power settings
- Short task: prove two jobs slow each other down and separate them
Memory
- The page cache and reclaim
- swappiness and the real effect of swap
- Huge pages and THP
- The OOM killer: who dies and how to protect
- Short task: stop the database from becoming the OOM victim
Disk and kernel
- I/O schedulers and queue depth
- Measuring disks with fio
- Dirty pages and writeback settings
- Kernel tuning with sysctl and making it stick
- tuned profiles
- Capstone: measure and tune a slow reporting server and prove the improvement
Practice labs
- Check resources in turn with the USE method: Collect utilisation, saturation and error measurements for CPU, memory, disk and network and identify the saturated resource.
- Take a baseline before changing anything: Run the workload three times, record mean and variance and show the measurement is repeatable.
- Prove a setting's effect with a controlled experiment: Change a single setting, measure again under the same load and show with numbers that the difference exceeds measurement noise.
- Break down where CPU time goes: Measure user, system, wait and steal proportions per core and write down which dominates.
- Relieve run queue build-up with affinity: Find threads piling onto one core, spread them with affinity and priority and measure the fall in queue length.
- Find the source of a context-switch storm: Measure the excessive context switching behind high system time per process, identify the service with lock contention and tune its worker count.
- Measure the effect of the page cache: Read the same file with a cold and warm cache, compare the times and show the change in memory output.
- Diagnose and reduce swap thrashing: Confirm thrashing from swap-in and swap-out rates, tune swappiness and the service's memory limit and measure again.
- Remove stalls caused by dirty page writeback: Tie second-long freezes during bulk writes to dirty page thresholds, tune them and compare the latency distribution.
- Measure a disk's baseline with fio: Run random read and sequential write tests on the spare disk and record IOPS and latency.
- Tune I/O scheduler and read-ahead for the workload: Change the scheduler and read-ahead, measure the difference on sequential and random workloads and persist the best setting.
- Lower write latency with mount options: On a workload writing many small files change access-time and journalling options, measuring the gain and documenting the safety trade-offs.
- Find dropped connections from network counters: Read listen queue overflow and retransmission counters and identify the one that rises under load.
- Tune socket buffers for throughput: Measure throughput between two lab servers, raise buffer limits and show the gain by measuring again.
- Tune a server end to end and report: On Ilgıntepe Tekstil's order server find the bottleneck, change at most three settings, measure each one's effect and report with rollback steps.
About this course
You find the bottleneck with the USE method, tune the scheduler, memory, swap, I/O and kernel parameters by measurement, and prove the effect of every change.
What you will learn
- Method
- CPU
- Memory
- Disk and kernel
Prerequisites
You should be comfortable with the level 101 and 201 topics of the same path.
Lab environment
Lessons run in your own Kali Linux terminal that opens in the browser. Nothing to install; the lab starts clean in every lesson.
Frequently asked questions
- Is this course right for my level?
- The course is at level 301; it assumes you know the topics of the lower-level courses.
- Is there anything to install?
- No. Every lesson runs in your own lab environment that opens in the browser.
- How do lessons work?
- Each lesson has 8 to 10 steps and a short quiz. Early steps show the command to type, later steps give less help; if two attempts fail, the solution opens.
- Can I try it for free?
- Yes. The first three lessons of every course are free.