Security analytics with Elastic: ECS, KQL, EQL, dashboards
You ingest security data into the Elastic Stack, map it to the common schema, write complex queries with KQL and EQL, and build detection rules and analyst dashboards.
Level 301 · 16 lessons · 15 practice labs
Lessons
Data
- Elastic architecture: index, document, field, data stream
- Shipping data with Elastic Agent and Beats
- Ingest pipelines: parsing, enrichment, GeoIP
- ECS: mapping different sources to common field names
- Short task: ingest a custom application log with ECS fields
Querying
- Advanced KQL: wildcards, ranges, nested fields
- Aggregations and statistics with ESQL
- EQL sequences: this happened, then that
- Investigating with timelines
- Short task: catch a download-then-execute sequence with EQL
Detection and visualisation
- Detection rules: query, threshold, sequence
- Rule exceptions and severity mapping
- Analyst dashboards: what to show and what not to
- Retention and index lifecycle
- Short task: build a one-page dashboard for authentication events
Capstone
- Capstone: connect a new log source to Elastic end to end and write three detections
Practice labs
- Split a log line into fields with an ingest pipeline: Parse the line with a dissect or grok processor and verify the result with simulate.
- Map custom field names to ECS fields: Rename the application’s user, address and outcome fields to their ECS equivalents and show them in a document.
- Add GeoIP enrichment and count by country: Add a geoip processor to the pipeline and run a query that aggregates logins by country.
- Fix events landing on the wrong day due to a bad timestamp: Find the date format and time zone error, fix the date processor and show the events land at the right time.
- Search with ranges and wildcards in KQL: Filter connections to destination ports above 1024 and to a given domain pattern.
- Produce failed-login statistics per user with ESQL: Aggregate failed logins by user and number of sources and find the account tried from many sources.
- Catch a download-then-run sequence with EQL: Match file creation followed by that file running on the same host within a one-minute window.
- Add an until condition to an EQL sequence: Add a condition that ends the sequence when the session closes and show the false matches disappear.
- Alert on password guessing with a threshold rule: Create a rule for twenty failed logins from one source in five minutes and trigger it with test data.
- Add a rule exception and set severity by asset: Make the scanner server an exception and add a mapping that raises severity for critical servers.
- Find the event this detection rule misses and fix it: Find the field-name mismatch in the rule’s query, fix it and show the missed event is caught in past data.
- Build a one-page dashboard for authentication: Build three visuals for failed logins, new source countries and locked accounts and point out one anomaly.
- Order one host’s events on a timeline: Add the suspicious host’s process, network and file events to a timeline and mark the first one.
- Draw a chart of failed logins per hour: Show failed logins from the authentication index as an hourly bar chart and note the busiest hour.
- Catch a three-step chain with one EQL sequence: Write an EQL sequence that looks for a document opening, a script interpreter and an outbound connection on the same host within ten minutes, and show it does not match clean data.
About this course
You ingest security data into the Elastic Stack, map it to the common schema, write complex queries with KQL and EQL, and build detection rules and analyst dashboards.
What you will learn
- Data
- Querying
- Detection and visualisation
- Capstone
Prerequisites
You should be comfortable with the level 101 and 201 topics of the same path.
Lab environment
Lessons run on your own Kali Linux desktop in the browser, with the tools ready in their windowed interfaces. Nothing to install; a computer screen is recommended.
Frequently asked questions
- Is this course right for my level?
- The course is at level 301; it assumes you know the topics of the lower-level courses.
- Is there anything to install?
- No. Every lesson runs in your own lab environment that opens in the browser.
- How do lessons work?
- Each lesson has 8 to 10 steps and a short quiz. Early steps show the command to type, later steps give less help; if two attempts fail, the solution opens.
- Can I try it for free?
- Yes. The first three lessons of every course are free.