Machine learning and anomaly detection with Scikit-learn
You build and evaluate simple models that learn from data and use them to find unusual behaviour in security data.
Level 301 · 20 lessons · 18 practice labs
Lessons
Preparation
- What machine learning is: supervised and unsupervised: You understand how a model learns from examples, how classification differs from regression, and where security uses it.
- Setup and exploring a dataset: You install Scikit-learn, load a built-in dataset and examine its features and labels.
- Preprocessing: scaling and categorical data: You scale numeric columns and convert categorical ones to numbers.
- Train/test split and data leakage: You split data honestly and recognise mistakes where test information leaks into training.
- Quick task: a feature table from login data
First models
- Linear regression: You predict a continuous value, such as server load, with a linear model.
- The nearest-neighbours (KNN) classifier: You classify a record by looking at its most similar neighbours and test the effect of k.
- Two-class decisions with logistic regression: You answer yes/no questions, such as whether an email is malicious, with probabilities.
- Decision trees and random forests: You build models that learn rules as trees and read the ranking of feature importance.
- Quick task: a phishing URL classifier
Evaluation
- Metrics: confusion matrix, precision, recall: You tell false alarms from missed events and see why plain accuracy can mislead.
- Cross-validation: You measure a model across several splits to avoid results that depend on luck.
- Pipelines and hyperparameter tuning: You chain preprocessing and a model into one pipeline and find the best settings with grid search.
- Imbalanced data: not missing rare attacks
- Quick task: choose an alert threshold
Use in security
- Anomaly detection with Isolation Forest: You build a model that flags unusually behaving accounts in unlabelled login data.
- Saving and loading models safely: You save a model with joblib and explain why loading model files from untrusted sources is dangerous.
- Limits: evasion attempts and data privacy: You know models can be fooled, training data can leak personal details, and when human review is needed.
- Clustering: behaviour groups with K-means
- Capstone: a suspicious-login detector: You derive features from a login log, build an anomaly model and report results with proper metrics.
Practice labs
- A feature table from raw events: Derive per-user hour, volume and distinct-folder features from file-access events and create features.csv.
- Scaling and categorical encoding: Scale and encode numeric and categorical columns with a ColumnTransformer to produce a matrix a model can use.
- Catch data leakage: Find the feature that leaks the target and the wrongly split time data in a model with unrealistically high accuracy and report the corrected score.
- Predicting device type with KNN: Predict device type from network behaviour features with KNN and tabulate how the value of k affects accuracy.
- Telling spam apart with logistic regression: Build a logistic regression on e-mail features and explain the five most influential features with their coefficients.
- Decision tree versus random forest: Compare a decision tree and a random forest on malicious-file features and show overfitting through the train-test gap.
- Read a confusion matrix: Compute precision, recall and the false-positive count from a detection model's confusion matrix.
- A threshold that fits analyst capacity: Choose a probability threshold for a team that can review at most 50 alerts a day and report the share of attacks caught.
- Cross-validation on imbalanced data: Improve a model on data where attacks are under one per cent using stratified cross-validation and class weights and show the change in PR-AUC.
- Put preprocessing and model in one pipeline: Combine separately written preprocessing and model steps into one Pipeline object and show it gives the same score.
- Tuning with grid search: Tune two parameters of a random forest with GridSearchCV and report the best values and the score gain.
- Save and load a model safely: Save a trained model with version and hash information and write a loader that checks the hash and rejects an untrusted file.
- Odd sessions with Isolation Forest: Run Isolation Forest on VPN sessions and list the ten oddest sessions and the features that make them odd.
- Behaviour groups with K-means: Build K-means clusters of users by working hours and application use and give each cluster a meaningful name.
- Resisting an evasion attempt: Test the model with samples where the attacker shifts features little by little and write which feature is easily fooled and how to guard it.
- Disk growth with linear regression: Fit a linear regression to a log server's daily disk use and estimate in how many days the disk will be full.
- Measure the prediction error: Compute MAE and RMSE for a ticket resolution-time model and find the ticket type with the largest error.
- Splitting by time correctly: Move a traffic forecasting model from a random split to a time-ordered split and explain the realistic score and why it differs.
About this course
You build and evaluate simple models that learn from data and use them to find unusual behaviour in security data.
What you will learn
- Preparation
- First models
- Evaluation
- Use in security
Prerequisites
You should be comfortable with the level 101 and 201 topics of the same path.
Lab environment
Lessons run in your own Kali Linux terminal that opens in the browser. Nothing to install; the lab starts clean in every lesson.
Frequently asked questions
- Is this course right for my level?
- The course is at level 301; it assumes you know the topics of the lower-level courses.
- Is there anything to install?
- No. Every lesson runs in your own lab environment that opens in the browser.
- How do lessons work?
- Each lesson has 8 to 10 steps and a short quiz. Early steps show the command to type, later steps give less help; if two attempts fail, the solution opens.
- Can I try it for free?
- Yes. The first three lessons of every course are free.