Mordor PCAPs 📡 — Part 1: Capturing Network Packets from Windows Endpoints with Network Shell (Netsh) ⚔️ and Azure Network Watcher 🌩

Published in

Open Threat Research

10 min readJul 27, 2020

On April 21st, 2020, the ATT&CK evals team released the results of their APT29 evaluation , the emulation plan, all payloads used for Day 1 and Day 2 , and a Do-It-Yourself Caldera plugin. On the same day 😆, I decided to organize a detection hackathon and used the official emulation plan to generate the data we would use to develop detection rules.

All that data was eventually uploaded to the Mordor project and it was the first time that I was sharing packet capture (PCAP) files along with endpoint logs for a large dataset such as the APT29 scenario.

After releasing all that data, I was asked by a good friend of mine Jason Trost if I had considered re-playing those PCAP files against Suricata with the open emerging threat rules. The result of that was summarized in this tweet

I ❤️ how practical and easy it was to collect PCAPs, share them with the community, get feedback, and validate open detection rules 💥!

Well, You Can Do It Too!

This post is part of a three-part series where Nate “neu5ron” Guagenti and I share a few ways to collect PCAP files from Windows endpoints while simulating a few lateral movement techniques and how to run them against tools such as Zeek to develop and validate rule-based detections 🍻.

Also, all the data generated in this series will be available in the the Mordor project for anyone to download. One of the main goals for this post is to encourage collaboration and collect more PCAPs as a community 🌎.

The second and third parts of this series can be found in the following links:

Mordor PCAPs 📡 — Part 2: Capturing Network Packets while Simulating a few Techniques for Lateral Movement
Mordor PCAPs 📡 — Part 3:

In this first post, I will show you how I capture network packets from Windows endpoints with Network Shell (Netsh) and Azure Network Watcher.

I do NOT just want to share a PCAP file. I want to show you how to create one, learn together and contribute back to the community 🍻!

First, What is the Mordor project?

The Mordor project provides pre-recorded security events generated by simulated adversarial techniques in the form of JavaScript Object Notation (JSON) for easy consumption. The pre-recorded data is categorized by platforms, adversary groups, tactics and techniques defined by the Mitre ATT&CK Framework. As part of our efforts to share diverse datasets with the community, we are now also sharing network packet capture (PCAP) files. Follow us @Mordor_Project 😈

What is Packet Capture?

You might already know this, but for anyone that is maybe just starting in the field or has been focused on endpoint events only for a long time (Like Me! 😆), this might be helpful.

Packet capture is the process of intercepting and logging traffic. As data streams flow across the network, the sniffer captures each packet and, if needed, decodes the packet’s raw data, showing the values of various fields in the packet, and analyzes its content according to the appropriate RFC or other specifications

Why Not Collect Network Packets Without Interacting with the Endpoint?

That’s a great question! Even though there are other ways to collect network packets without interacting with the endpoint (i.e Network traffic intercepted by a network sensor), I wanted to learn the following concepts for research purposes and share what actually worked for me for recent projects:

Leverage built-in Event Tracing for Windows (ETW) mechanisms to log and filter network events and packets.
Utilize Azure cloud native services to enable endpoint packet capture capabilities on every Windows VM in my lab, automate and expedite the packet capture process for very specific endpoints, and have the flexibility to trigger a packet capture programmatically (PowerShell, Azure CLI, etc)

1) Network Shell (Netsh)

Netsh is a command-line scripting utility that allows you to display or modify the network configuration of a computer that is currently running.

Based on MS docs, Netsh interacts with other operating system components by using dynamic-link library (DLL) files. Each netsh helper DLL provides an extensive set of features called a context, which is a group of commands specific to a networking server role or feature.

If you want learn more about what contexts are available through netsh, run the following command on a Windows endpoint via the command-prompt or PowerShell (My Windows box was a Win10 1909)

C:\> netsh ?

What Context do I use to Capture Network Traffic?

We need to use the commands available through the Network Trace one. They extend the Network Diagnostic Framework (NDF) and leverage the Event Tracing for Windows (ETW) framework to log network events and packets in a single Event Trace Log (ETL) file.

What is Event Tracing For Windows (ETW)

Event Tracing for Windows (ETW) is an efficient kernel-level tracing facility that lets you log kernel or application-defined events to a log file

You can read more about it in the MS docs or this blog post I wrote about it: “Threat Hunting with ETW events and HELK — Part 1: Installing SilkETW”

How Does a Network Trace Work?

https://docs.microsoft.com/en-us/windows/win32/ndf/network-tracing-in-windows-7-architecture

The Netsh trace context acts as an ETW controller and is capable to start and stop event trace sessions and enable event providers.
Components such as the ones from the network stack, as shown in the image above, register as ETW trace providers and emit events related to network activity to event trace sessions started by the netsh trace context.
For packet capture, Windows leverages the Microsoft-Windows-NDIS-PacketCapture (NDISCAP) provider as an ETW provider.
Traces are then collected and processed to an Event Trace Log (ETL) file.
ETL files can then be converted to other formats such as EVTX or PCAP.

Start a Network Trace Session

First, I recommend to run the following in your Windows box to get more familiarized with the available commands for a network trace:

C:\> netsh trace ?

It is obvious the we can start a trace with netsh trace start . However, there is still a long list of additional commands you can use to customize your trace. Simply run netsh trace start ? to learn more about them.

I learned my first netsh trace commands from Nate “neu5ron” Guagenti 🙏, and this is the one that I always use to start a quick network trace:

C:\>  netsh trace start capture=yes fileMode=single maxSize=0 traceFile=C:\users\pgustavo\Desktop\test.etl

capture: Specifies whether packet capture is enabled.
fileMode: File mode applied when tracing output is generated.
maxSize: Maximum size in MB for saved trace file. 0=No Max.
traceFile: Location and name of the .etl output file.

Run the command in a Windows box, generate some network traffic and then stop the trace with netsh trace stop .

How to Consume a Network ETL File?

Files generated by this type of network trace can be opened by the Microsoft Message Analyzer. However, it was retired/deprecated at the beginning of 2020. You might be asking yourself “What about Wireshark?” . Unfortunately, they cannot be opened by Wireshark. You would have to convert it to a format such as the PCAP Next Generation Dump File (or pcapng for short).

Enter Etl2Pcapng

This is a Microsoft Github project that allows you to convert network etl files to a pcapng format. It is very easy to use and is what I currently use 😉.

Simply download the latest release copy of it from Github and install the latest supported Visual C++ redistributable package (runtime).

Next, run the following command to go from an etl file to a pcapng one

etcl2pcapng.exe test.etl test.pcapng

You can then take the pcapng file and open it with tools such as Wireshark 🍻

I have used this approach for a few projects already and it is very practical and easy to use. I started using it with Azure virtual machines, but I found out that there was a cloud native way via a virtual machine extension provided by the Azure Network Watcher monitoring service.

2) Azure Network Watcher

Azure Network Watcher is a network performance monitoring, diagnostic, and analytics service that allows monitoring of Azure networks. The Network Watcher Agent virtual machine extension is a requirement for capturing network traffic on demand, and other advanced functionality on Azure virtual machines.

Pre Requirements:

Packet capture requires the following outbound TCP connectivity:

to the chosen storage account over port 443
to 169.254.169.254 over port 80
to 168.63.129.16 over port 8037

Install VM Extension (PowerShell AZ Module)

You can use the Set-AzVMExtension command to deploy the Network Watcher Agent virtual machine extension to an existing virtual machine:

Set-AzVMExtension `
  -ResourceGroupName "myResourceGroup1" `
  -Location "WestUS" `
  -VMName "myVM1" `
  -Name "networkWatcherAgent" `
  -Publisher "Microsoft.Azure.NetworkWatcher" `
  -Type "NetworkWatcherAgentWindows" `
  -TypeHandlerVersion "1.4"

Install VM Extension (ARM Templates)

I like this approach better because I can just add it to the Azure Resource Manager (ARM) templates I use to deploy research Windows environments.

Resource type: Microsoft.Compute/virtualMachines/extensions
Extension type: NetworkWatcherAgentWindows

{
    "type": "extensions",
    "name": "Microsoft.Azure.NetworkWatcher",
    "apiVersion": "[variables('apiVersion')]",
    "location": "[resourceGroup().location]",
    "dependsOn": [
        "[concat('Microsoft.Compute/virtualMachines/', variables('vmName'))]"
    ],
    "properties": {
        "publisher": "Microsoft.Azure.NetworkWatcher",
        "type": "NetworkWatcherAgentWindows",
        "typeHandlerVersion": "1.4",
        "autoUpgradeMinorVersion": true
    }
}

Start a Packet Capture (Azure CLI)

There are a few ways to start a packet capture:

For current projects, I have used the Azure CLI way. For example, This is the script I used to start packet capture sessions in the environment I built to simulate APT29 from the ATT&CK evals 😉

The main part of the script is the following Azure CLI function that one could use to start a packet capture session on a specific virtual machine in a specific resource group with a specific filter (filter all the noise!).

az network watcher packet-capture create --resource-group ${RESOURCE_GROUP} --vm ${COMPUTER} --name "${COMPUTER}_PCAP" --storage-account ${STORAGE_ACCOUNT} --filters "                           [
 {
   \"localIPAddress\":\"10.0.0.0-10.0.1.9\",
   \"remoteIPAddress\":\"10.0.0.0-10.0.1.9\"
 },
 { 
   \"localIPAddress\":\"10.0.0.0-10.0.1.9\",
   \"remoteIPAddress\":\"192.168.0.0-192.168.0.10\"
 }                           
]
"

Filtering the noise is great for Azure cloud environments, because a lot of the extensions installed on the endpoints generate a lot of network connections with information about subscriptions, resource groups, etc (Private stuff).

🚨 If you want to share your PCAPs with the community, you need to filter that out!

Once you start a packet capture session, you can see it in your Azure Portal > Network Watcher > Packet Capture view

Stop Packet Capture (Azure CLI)

I also created the following script for the APT29 ATT&CK evals simulation to stop and potentially delete the current packet capture sessions in an network environment. This is the main command used in the script.

az network watcher packet-capture stop --name "${COMPUTER}_PCAP" --location ${LOCATION}az network watcher packet-capture delete --name "${COMPUTER}_PCAP" --location ${LOCATION}

Once you stop a packet capture session, the network packets are automatically saved in the storage account that you pass as a parameter to the Azure CLI commands used to start the packet capture session in the first place.

This is very convenient because everything is stored in a central location. I do not have to RDP or script a way to retrieve all those PCAPs. Now you can download, drag and drop the packet capture files in Wireshark 😉. Also, you can share the PCAP with the community through the Mordor project 😈

Also, in case you were wondering, there is an extension for Linux too.

That’s it! I hope you found these two tools and notes useful for some of your research. I wanted to share some of the ways that have worked for me and recently used in a few recent projects.

Once again, there are several other ways to capture network packets. You might not want to interact with the endpoint to generate PCAP files to keep the creation of additional forensic artifacts to the minimum, and I understand that 💯. I believe it depends on your research scenario, and it is always good to have alternatives 😉 🎊

Some of the alternative ways that Nate “neu5ron” Guagenti suggests 🙏 to consider too are the following:

VirtualBox

VBoxManage modifyvm "windows_10vm01" --nictrace1 on --nictracefile1 windows_10vm01.pcapng.pcapVirtualBox -startvm "windows_10vm01"

VMWare ESXI or ESX

VMWare Workstation Linux

sudo /usr/bin/vmnet-sniffer -e -w windows_10vm01.pcapng /dev/vmnet0

VMWare Workstation Windows

C:\Program Files (x86)\VMware\VMware Workstation vnetsniffer.exe /w "windows_10vm01.pcapng" VMnet0

VMWare Workstation OSX

sudo /Applications/VMware\ Fusion.app/Contents/Library/vmnet-sniffer -e -w windows_10vm01.pcapng vmnet0

In the next post, I will show you how I used some of the main tools showcased in this post to create PCAP files wile simulating a few lateral movement techniques. All those PCAP files will be contributed to the Mordor Project after the post release. Thank you again and enjoy the rest of your week 🍻!