Introduction to data analysis
What data is, how to ask questions of it and how to read a table; you run your first analysis on security records at the command line and turn it into a short report.
Level 101 · 18 lessons · 15 practice labs
Lessons
Data and questions
- What data is: records, fields, values
- Data types: numeric, categorical, time
- Asking good questions of data
- The analysis cycle: collect, clean, explore, tell
- Quick task: draft questions for an incident
Reading tables
- Table structure: rows, columns, headers
- The CSV file: delimiters, quotes, encoding
- JSON records: nested fields
- Where security data comes from: logs, inventories, alerts
- Quick task: write a data set's ID card
First analysis
- Counting and grouping
- Minimum, maximum, average
- Sorting and finding the top N
- Working with time: time zones and intervals
- Quick task: find the busiest hour
Telling the result
- Pitfalls: missing data, sampling, correlation
- Writing a short findings report
- Capstone: a first analysis report from a web server log
Practice labs
- Field types in a delivery record: Label every field in Atlas Cargo's delivery records as numeric, categorical or time and write the result to types.csv.
- Categories that look like numbers: Tell apart number-like fields such as postcodes, ports and status codes and mark, with a reason, which ones must never be summed.
- Mixed-type inventory columns: Find the columns in Lumen Retail's device inventory that hold more than one type and propose the correct type and conversion rule for each.
- From a complaint to a measurable question: Turn the complaint “the site is slow” into three measurable questions that the request log in front of you can answer.
- Which data answers which question: Match Orbis Health's five questions to four data sets and name the question no set can answer along with the missing data.
- Turning a vague manager request into analysis: Turn the request “are our accounts safe?” into an analysis plan with a clear scope, metric and data source and write it to plan.md.
- Read a semicolon-separated CSV correctly: Work out the column count and the total amount of a sales file that uses semicolons as separators and commas as decimal marks.
- Commas and line breaks inside quotes: Find the real record count in a support-ticket export whose description field contains commas and line breaks.
- Flatten nested JSON alerts: Pull the source IP, rule name and tag count out of alert records with nested fields and lists into a single flat table.
- The most used browsers: Count the browser types in a web log and write down the three most common ones with their shares.
- Average and maximum per department: Find the average and maximum pages per department in Velora Energy's print logs and rank the departments.
- Peak hour in a log with mixed time zones: Convert session records written in two time zones to UTC, find the real peak hour and show how wrong the naive answer was.
- Note missing data in a report: Find the days missing from a week of records and add them to a short findings note as a data-gap warning.
- Catch a misleading average: Show why a draft report's conclusion that “average response time is fine” is misleading because of outliers and a missing sample.
- Test a correlation claim: Test the claim that attacks rise on patch days against the data and write a five-sentence finding that explains the role of a third factor.
About this course
What data is, how to ask questions of it and how to read a table; you run your first analysis on security records at the command line and turn it into a short report.
What you will learn
- Data and questions
- Reading tables
- First analysis
- Telling the result
Prerequisites
No prior knowledge is needed; the course starts from zero.
Lab environment
Lessons run in your own Kali Linux terminal that opens in the browser. Nothing to install; the lab starts clean in every lesson.
Frequently asked questions
- Is this course right for my level?
- The course is at beginner level; every term is explained where it first appears.
- Is there anything to install?
- No. Every lesson runs in your own lab environment that opens in the browser.
- How do lessons work?
- Each lesson has 8 to 10 steps and a short quiz. Early steps show the command to type, later steps give less help; if two attempts fail, the solution opens.
- Can I try it for free?
- Yes. The first three lessons of every course are free.