CyberScan
Vulnerability assessment and automated pentest. Continuous asset discovery, AI risk prioritisation, integrated NIS2 compliance manager.
Discover CyberScan →
Cyber Security
CISO-as-a-service consulting: posture, remediation roadmap, ongoing support.
Discover →Scalable full packet capture
Full packet capture (fPCAP) — complete recording of network traffic — is the gold standard for network forensics: it enables retrospective analysis of any incident. But handling terabytes of PCAP per day, with fast search and usable UI, is a non-trivial engineering problem.
In 2012 Andy Wick and Elyse Rinne, engineers at AOL (later merged into Verizon Media / Yahoo), start the Moloch project: indexed full packet capture system, with web UI and Elasticsearch-backed search.
In July 2021, to resolve a trademark issue linked to the name (Moloch is also a Marvel character and a historical deity), the project is renamed Arkime. The Arkime 3.0 release of 27 July 2021 coincides with the rename. The licence remains Apache 2.0.
Architecture
Three main components:
- capture — C process that captures packets from interfaces (AF_PACKET, DPDK, PF_RING), writes PCAP to disk and sends session metadata to Elasticsearch/OpenSearch
- viewer — web UI (Node.js) for search, session visualisation, PCAP download, charts
- Elasticsearch/OpenSearch — session metadata index
A session represents a flow (TCP/UDP) with metadata: src/dst IP, ports, bytes, duration, detected application protocol, extracted HTTP/TLS/DNS/SMB fields.
Distributed capture
Arkime scales horizontally: multiple capture nodes feed a shared Elasticsearch/OpenSearch cluster. The viewer exposes a unified view proxying PCAPs from respective capture nodes on demand. This design enables multi-site deployments (tens of Gbps aggregate, weeks or months of retention).
Protocol parsers
Arkime extracts application metadata via built-in parsers:
- HTTP — URI, host, user-agent, response code
- TLS — SNI, certificates (issuer, subject, fingerprint)
- DNS — query, response, type
- SMB, RDP, SSH, SMTP, FTP, IRC, Socks, MySQL, PostgreSQL, RADIUS, Quic
Extracted fields are indexed and searchable in the UI via SPI (Session Profile Information).
WISE: Wise Intelligence Support Engine
WISE is the enrichment subsystem: it allows tagging sessions in real time with threat intelligence (MISP feeds, custom lists, GeoIP, ASN, reputation). Supports file, Redis, HTTP URL, ElasticSearch sources.
Integration with Suricata and Zeek
Arkime can import Suricata alerts and associate them with corresponding sessions in its index, enabling one-click transition from an IDS alert to the full flow PCAP. Analogous integration for Zeek (formerly Bro) via Zeek logs.
Cryptography and Y-stream
Arkime supports rotating capture keys and encrypted PCAP storage. The Y-stream is an extension to handle high traffic with deduplication and chunking.
Adoption
- National CERTs and CSIRTs — long-term packet retention
- University research networks — retrospective analysis
- Large enterprise and cloud providers — network visibility in multi-tenant infrastructures
- Threat hunting teams — retrospective TTP search
In the Italian context
Arkime is used in:
- ISP and data centre backbones — retroactive forensic visibility
- Internal SOCs of large enterprises — integration with Suricata + ELK
- Universities and research centres — PCAP archives for teaching and theses
- Public administration with retention requirements — regulated incident traceability
Among open source full packet capture tools it is the most mature for scale and usability, with a stable ecosystem inherited from AOL/Verizon experience.
References: Arkime (previously Moloch). Original authors Andy Wick and Elyse Rinne, AOL (later Verizon Media/Yahoo), 2012. Rename and Arkime 3.0 release on 27 July 2021 due to trademark issue with the Moloch name. Apache 2.0 licence. Indexing on Elasticsearch/OpenSearch. Site: https://arkime.com.