PCAP analysis with Python: Scapy and pyshark
With Scapy and pyshark you read captures in code, craft test packets for your own lab network and produce repeatable analysis reports.
Level 301 · 15 lessons · 12 practice labs
Lessons
Reading captures
- Accessing packet layers and fields with Scapy
- rdpcap and PcapReader: reading large files memory-friendly
- Using Wireshark dissectors from Python with pyshark
- Processing tshark JSON output in Python
- Short task: write every DNS query and answer in a capture to CSV
Analysis scripts
- Grouping sessions and per-flow statistics
- Computing a regularity score from connection intervals
- Traffic summaries and charts with pandas
- Short task: write a script that finds the three most regular beaconing hosts
Crafting test packets
- Rules for crafting packets: your own lab network only
- Crafting custom packets to test rules and writing them to PCAP
- Replaying captures on the lab network with tcpreplay
- Automatically testing an IDS rule with generated traffic
- Short task: produce two PCAPs, one triggering a Suricata rule and one not
- Capstone: build a tool that produces an incident report from a PCAP
Practice labs
- Reach the layers with Scapy: Print the IP and TCP fields of the first ten packets of a capture with Scapy.
- Write every DNS query to CSV: Write a script dumping every DNS query and answer in a capture to a CSV file with timestamps.
- Read a large file memory-friendly: Process a 1 GB capture packet by packet with PcapReader and compare memory use with rdpcap.
- List HTTP requests with pyshark: Print the method and URI of the HTTP requests in a capture using pyshark.
- Process tshark JSON output: Write a script reading tshark's JSON output with Python and counting TLS server names.
- Compare two libraries on speed and accuracy: Do the same count with Scapy and pyshark, time both and explain why the results can differ.
- Group sessions by five-tuple: Write a script grouping packets into flows and printing each flow's packet and byte totals.
- The three most regular hosts: Write a script computing a regularity score from the deviation of connection intervals and listing the top three hosts.
- A traffic summary and chart with pandas: Load the capture into a pandas table, draw a per-minute volume chart and find the source of the spike.
- Craft a test packet and write it to a PCAP: Craft a single packet carrying a specific payload with Scapy and write it to a file.
- Two PCAPs: one that fires the rule, one that does not: Produce two test captures, one firing a Suricata rule and one that must not, and verify the outcome.
- Replay a capture on the lab network: Replay a capture on the isolated lab interface with tcpreplay and test an IDS rule automatically with a script.
About this course
With Scapy and pyshark you read captures in code, craft test packets for your own lab network and produce repeatable analysis reports.
What you will learn
- Reading captures
- Analysis scripts
- Crafting test packets
Prerequisites
You should be comfortable with the level 101 and 201 topics of the same path.
Lab environment
Lessons run in your own Kali Linux terminal in the browser, connected to a private lab network. Nothing to install.
Frequently asked questions
- Is this course right for my level?
- The course is at level 301; it assumes you know the topics of the lower-level courses.
- Is there anything to install?
- No. Every lesson runs in your own lab environment that opens in the browser.
- How do lessons work?
- Each lesson has 8 to 10 steps and a short quiz. Early steps show the command to type, later steps give less help; if two attempts fail, the solution opens.
- Can I try it for free?
- Yes. The first three lessons of every course are free.