Log analysis and threat hunting with Python
You read, normalise and query Linux, web and Windows event logs in Python to hunt for suspicious behaviour.
Level 201 · 15 lessons · 12 practice labs
Lessons
Log formats
- Parsing syslog and journald records
- JSON logs and nested fields
- Reading Windows event logs with python-evtx
- Timestamps and time-zone pitfalls
- Short task: convert three different logs into one format
Asking questions
- First statistics with Counter and grouping
- Brute-force detection with a sliding time window
- Catching the same account logging in from far apart
- Unusual parent–child process pairs
- Short task: find a newly added scheduled task in the logs
Enrichment
- Adding location from an offline GeoIP database
- Cutting noise with allowlists
- Short task: enrich alerts with asset and owner details
Wrap-up
- An event timeline from several sources
- Capstone: trace an intrusion through one morning’s logs
Practice labs
- Flatten nested JSON fields: Write a script that turns the nested user and request fields of an app’s JSON logs into flat columns, and output the result as CSV.
- Time zones scramble the incident: Convert three log sources written in local time, UTC and another time zone to UTC, merge them and show where the order of events changes.
- Three different logs, one format: Write a converter that maps syslog, JSON and a Windows event log export into one common schema, reporting unrecognised lines separately without losing them.
- Failed logins per account: Count failed logins per account and source address in the Ekin Co-op login logs with Counter, and show the three most-targeted accounts.
- Brute force with a sliding window: Write a sliding-window script that finds bursts of failed logins above a threshold within a set time, and mark separately the account where a success followed.
- Impossible travel: Find cases where the same account logs in from distant locations in a short time using offline GeoIP data, and filter out false alarms caused by company VPN exits.
- A newly added scheduled task: Find scheduled-task creation events in the given Windows event log export, and report the new task’s name, command and creating account.
- A shell launched from an office app: Write a script that finds unusual parent-child process pairs in process-creation logs, and show the machine where a document editor launched a command shell.
- An intrusion across one morning: Build the incident timeline from initial access to persistence using web, authentication and process logs, and back each step with an evidence line.
- Cut noise with an allowlist: Write a filter that removes expected events from scanning and monitoring servers using an allowlist, and show how much the alert count drops.
- Owner and asset details on alerts: Join the Cinnamon Café alerts with the asset inventory to add owner, location and criticality to each alert, and report machines left without an owner.
- A gap in the timeline: Complete a timeline left incomplete by log rotation by also reading the compressed .gz files, and identify the interval where records were deleted.
About this course
You read, normalise and query Linux, web and Windows event logs in Python to hunt for suspicious behaviour.
What you will learn
- Log formats
- Asking questions
- Enrichment
- Wrap-up
Prerequisites
Finishing the level 101 courses of the same path, or knowing those topics, is enough.
Lab environment
Lessons run in your own Kali Linux terminal that opens in the browser. Nothing to install; the lab starts clean in every lesson.
Frequently asked questions
- Is this course right for my level?
- The course is at level 201; it assumes you know the topics of the lower-level courses.
- Is there anything to install?
- No. Every lesson runs in your own lab environment that opens in the browser.
- How do lessons work?
- Each lesson has 8 to 10 steps and a short quiz. Early steps show the command to type, later steps give less help; if two attempts fail, the solution opens.
- Can I try it for free?
- Yes. The first three lessons of every course are free.