Log analysis: from syslog to web server logs
You filter system, authentication and web server logs with grep, awk and jq, merge different sources onto one clock, and follow an attack with simple SIEM queries.
Level 201 · 18 lessons · 15 practice labs
Lessons
The world of logs
- Why logs are the best witness, and when they lie
- syslog, rsyslog and journald: where logs are kept
- Log formats: plain text, JSON, CEF and EVTX
- Retention, rotation and log integrity
- Short task: inventory every log source on a server
Filtering on the command line
- Selecting and counting fields with grep and awk
- Old compressed logs with zgrep and zcat
- Filtering and summarising JSON logs with jq
- Converting time zones to UTC and merging logs
- Short task: merge three sources into one timeline
Attack traces
- Web access logs: scanning, injection and web shell traces
- Brute-force and password-spraying patterns
- A first look at Windows event logs: 4624, 4625, 4688
- Gaps in logs: spotting deleted or stopped logging
- Short task: find the first request to the web shell
SIEM queries
- What a SIEM does: collection, parsing, correlation
- Searching in a SIEM: filters, statistics and time charts
- Capstone: tell the full story of an account takeover from mixed logs
Practice labs
- Inventory the log sources on a server: List every log file and journald unit on a lab server with their formats.
- Audit rotation and retention: Work out from logrotate settings how many days back each log goes and mark the shortest.
- When a log lies: Find the contradiction between two logs describing the same event and justify which to trust and why.
- Select and count fields with grep and awk: Derive the request count by status code from a web access log.
- Search the compressed old logs: Gather all requests of one address across a month of rotated logs with zgrep.
- Summarise JSON logs with jq: Extract the failed operation count per user and first error time from a JSON application log.
- Convert timestamps to UTC: Convert ten log lines in three different time zones to UTC.
- Merge three sources into one timeline: Bring auth.log, the web access log and the application log to a common format and sort by time.
- Correct the server with clock drift: Calculate a server's clock drift from a shared event and merge its logs with corrected time.
- Recognise the brute force pattern: Separate the patterns targeting one account and many accounts in an authentication log.
- Find the first request to the web shell: Show the scanning, upload and first call requests in order in a prepared access log.
- Notice the gap in the log: Find the missing time range in a log and support with another source whether it was stopped or deleted.
- Count 4624 and 4625 events: Count successful and failed logons per account in exported Windows events.
- Search and time chart in a SIEM: Filter one account's logon events in the lab SIEM, chart them hourly and mark the spike.
- Tell an account takeover from start to finish: From Yelkovan Bankası's mixed prepared logs report an account's attempts, login and later actions in time order.
About this course
You filter system, authentication and web server logs with grep, awk and jq, merge different sources onto one clock, and follow an attack with simple SIEM queries.
What you will learn
- The world of logs
- Filtering on the command line
- Attack traces
- SIEM queries
Prerequisites
Finishing the level 101 courses of the same path, or knowing those topics, is enough.
Lab environment
Lessons run in your own Kali Linux terminal that opens in the browser. Nothing to install; the lab starts clean in every lesson.
Frequently asked questions
- Is this course right for my level?
- The course is at level 201; it assumes you know the topics of the lower-level courses.
- Is there anything to install?
- No. Every lesson runs in your own lab environment that opens in the browser.
- How do lessons work?
- Each lesson has 8 to 10 steps and a short quiz. Early steps show the command to type, later steps give less help; if two attempts fail, the solution opens.
- Can I try it for free?
- Yes. The first three lessons of every course are free.