Advanced anomaly detection in security data
You turn user and entity behaviour into features, compare statistical, density and tree-based methods, and build an explainable detection pipeline that reduces alert fatigue.
Level 401 · 15 lessons · 12 practice labs
Lessons
Behaviour modelling
- User and entity behaviour analytics
- Engineering features from security data
- Peer groups: unusual compared with whom
- Quick task: user behaviour profiles
Methods
- Robust statistics: median and MAD
- Density-based methods: LOF
- Tuning Isolation Forest and its limits
- Rare event sequences
- Quick task: compare three detectors on the same data
Evaluation and operation
- Scarce labels and evaluation
- The alert budget: how many alerts per day
- Why unusual: explainable scores
- When behaviour changes: drift and retraining
- Evasion and data poisoning
- Capstone: a detector that finds a data-leaking insider
Practice labs
- A user behaviour summary: Derive each user's active hours, device count and daily volume features from a month of events.
- Oddness relative to a peer group: Compare every user with peers in their own department and find the five who behave most unlike their group.
- Model service accounts separately: Separate human and service accounts by behaviour, build a separate baseline for service accounts and show the drop in false alarms.
- A robust score with median and MAD: Compute a median-and-MAD score for upload volumes whose extremes distort the mean and show how it differs from a z-score.
- Local outliers with LOF: Run LOF on data with two behaviour clusters of different density and find the local outliers a global threshold misses.
- Compare three detectors on one data set: Compare MAD, LOF and Isolation Forest against a few labelled incidents and tabulate the hits in each one's top 50 alerts.
- Rare command sequences: Count consecutive command pairs in admin sessions and list the sequences seen only once.
- Deviation from the normal workflow: Model the usual order of steps in a payments application with transition probabilities and find the sessions that skip the approval step.
- Score low-probability sessions: Compute each session's probability with a transition model, correct for length and rank the ten most unusual sessions.
- A threshold from the alert budget: Choose a score threshold for a team with a capacity of 30 alerts a day and show the weekly alert count stays within budget.
- Make the score explainable: Produce an explanation line for each alert showing the three features that contribute most to the score and their usual values.
- Notice a poisoned baseline: Find a baseline an attacker shifted by raising volume little by little for weeks and prove the drift against a fixed reference period.
About this course
You turn user and entity behaviour into features, compare statistical, density and tree-based methods, and build an explainable detection pipeline that reduces alert fatigue.
What you will learn
- Behaviour modelling
- Methods
- Evaluation and operation
Prerequisites
You should have practised the topics of the same path up to level 301.
Lab environment
Lessons run in your own Kali Linux terminal that opens in the browser. Nothing to install; the lab starts clean in every lesson.
Frequently asked questions
- Is this course right for my level?
- The course is at level 401; it assumes you know the topics of the lower-level courses.
- Is there anything to install?
- No. Every lesson runs in your own lab environment that opens in the browser.
- How do lessons work?
- Each lesson has 8 to 10 steps and a short quiz. Early steps show the command to type, later steps give less help; if two attempts fail, the solution opens.
- Can I try it for free?
- Yes. The first three lessons of every course are free.