Arkime — Large-Scale Full Packet Capture and Network Traffic Analysis

Introduction

Arkime is a large-scale full packet capture and indexed network history system. Developed originally at AOL, it captures every packet traversing a network link, stores the raw PCAP data on disk, and indexes session metadata in Elasticsearch or OpenSearch for fast querying through a purpose-built web interface.

What Arkime Does

• Captures full network packets at multi-gigabit rates and writes raw PCAP to disk
• Indexes session metadata (IPs, ports, protocols, HTTP headers, TLS certs) in Elasticsearch
• Provides a web UI for searching, filtering, and drilling into individual sessions and packets
• Exports selected sessions as PCAP files for analysis in Wireshark or other tools
• Supports SPI (Session Profile Information) views with decoded protocol details

Architecture Overview

Arkime has three main components: the capture daemon, the viewer web application, and an Elasticsearch/OpenSearch cluster. The capture daemon reads packets from network interfaces using libpcap or af_packet, writes them to PCAP files on local storage, and sends session metadata to Elasticsearch. The viewer is a Node.js application that queries Elasticsearch and serves a web UI. A wise (WISE) service enriches sessions with external threat intelligence during capture.

Self-Hosting & Configuration

• Requires an Elasticsearch 8.x or OpenSearch 2.x cluster for session metadata indexing
• Configure capture interfaces and BPF filters in config.ini
• Allocate fast local storage (SSD or NVMe) for PCAP files; plan for data retention policies
• Use the WISE plugin framework to enrich sessions with threat feeds and IP reputation data
• Deploy multiple capture nodes behind a load balancer for distributed multi-site capture

Key Features

• Handles sustained multi-gigabit capture rates with optimized disk I/O
• Session search with Arkime query language supporting IP, port, protocol, country, and header fields
• SPI graph and map views for visual traffic analysis and anomaly detection
• Automatic PCAP file rotation and expiration based on disk usage thresholds
• Hunt feature for retroactive regex searching across stored packet payloads

Comparison with Similar Tools

• Wireshark — interactive single-file packet analyzer; Arkime captures continuously and indexes for search across terabytes
• Zeek — produces structured logs from traffic; Arkime stores full packets for complete payload inspection
• Security Onion — integrates Arkime as its packet capture component alongside Suricata and Zeek
• ntopng — real-time traffic monitoring and flow analysis; Arkime retains full packets for forensic investigation

FAQ

Q: How much storage do I need? A: It depends on traffic volume. A 1 Gbps link at 50% utilization generates roughly 5 TB per day. Configure PCAP expiration to manage disk usage automatically.

Q: Can I use OpenSearch instead of Elasticsearch? A: Yes. Arkime supports both Elasticsearch 8.x and OpenSearch 2.x as the metadata backend.

Q: Does Arkime decrypt TLS traffic? A: Not by default. You can configure it to use TLS session keys (via SSLKEYLOGFILE) for decryption when available.

Q: Can I integrate threat intelligence feeds? A: Yes. The WISE (With Intelligence See Everything) service enriches sessions with data from threat feeds, file hashes, and custom lookup sources during capture.

Sources

• https://github.com/arkime/arkime
• https://arkime.com/learn