Tuning Capture - NormB/sipnab GitHub Wiki

sipnab defaults to a busy production link rather than a laptop demo. This page is what to change when they are still not enough, in the order worth changing them.

Start here, always: find out whether you are actually dropping packets. Every other decision on this page depends on that number, and sipnab reports it.

sipnab polls libpcap's kernel counters once a second and reports two numbers.

PACKETS ARE BEING DROPPED on 'eth0' (kernel buffer: 18432, interface/driver: 0).
The analysis for this run is INCOMPLETE — dialogs may be missing messages and
RTP loss figures will overstate what was on the wire.

You get that warning the moment the first drop happens, and a summary at the end of the capture:

Live capture on 'eth0' finished: 4821003 packets captured, but 18432 dropped by
the kernel buffer and 0 by the interface — THIS ANALYSIS IS INCOMPLETE

A clean run says so explicitly, so silence is never ambiguous:

Live capture on 'eth0' finished: 4821003 packets, no drops

That distinction is the single most useful thing on this page. Operators routinely respond to any drop by raising -B, which does nothing at all for interface drops and wastes memory while the real problem goes unaddressed.

sipnab polls pcap_stats on each open capture handle — the socket this run opened — and libpcap keeps those counters per handle. A handle is one capture socket. An interface is the NIC every process on the host shares. The difference changes what a zero means and what a total covers.

• Zero drops means this run missed nothing. It says nothing about the interface. Another sniffer on the same NIC keeps its own counters, and neither run can read the other's.
• One run can hold several handles. --multi-device opens one per named interface, and --cores N on a live device opens N sockets on one interface (section 8, "On a live device it means capture sockets, not workers"). sipnab adds each handle's increment to one process-wide pair, so the totals cover the whole run rather than any single socket. The first-drop warning names the device that dropped, which is what tells you where to act.
• The packet count beside them is sipnab's own tally, not a libpcap counter. It counts frames sipnab read. The drop pair is what says how many more the wire carried.

sipnab reports both counters on four surfaces and keeps them apart on every one of them, so whichever you are looking at can answer "which kind of drop is this?"

Two separate Prometheus counters rather than one sipnab_capture_dropped_total with a label is deliberate: a dashboard that sums them produces a number whose remedy is ambiguous, and an alert on that sum pages someone to raise a buffer that cannot help. Alert on them separately, or the alert cannot say what to do.

The REST API and MCP report the same pair as kernel_dropped_packets and interface_dropped_packets on the capture-quality object, beside the invalid_timestamps, undecodable_frames and snapped-frame counts that make up the rest of a capture's quality — see GET /v1/stats.

Why a drop is not just "missing packets". A dropped SIP message means a dialog reconstructs wrong — a missing BYE leaves a call that never ends, a missing 200 OK leaves one that never answered. A dropped RTP packet is counted as network loss that never happened, so MOS and loss figures read worse than the call actually was. A lossy capture does not produce a smaller answer; it produces a wrong one.

Default: 64 MiB per capture device.

This is the ring libpcap fills and sipnab drains. It absorbs the difference between when packets arrive (bursty, driven by your traffic) and when sipnab gets scheduled to read them (jittery, driven by your kernel). It is the setting that matters most.

Busy trunk — give it room for a bigger burst:

sudo sipnab -N -d eth0 -B 256

Small or embedded host — cap the memory instead:

sudo sipnab -N -d eth0 -B 8

-B alone may buy less than you think. libpcap divides the ring into fixed-size slots whose size comes from --snaplen, so how many packets 64 MiB holds depends on the snapshot length and on whether NIC offloads are on — anywhere from ~1,000 to ~41,000 — and on whether you named an interface at all. Read section 4, "Snapshot length" and section 5, "Which interfaces: any versus named" before concluding that a bigger -B did not help; the three settings multiply.

Rules of thumb:

• Raise it when kernel buffer drops are non-zero and CPU is not pinned. Dropping with idle CPU means bursts, and bursts are exactly what a ring absorbs.
• Do not raise it when the drops persist and a core is at 100%. A bigger ring buys a longer burst, not more throughput — you are not keeping up on average, and no buffer size fixes that. Go to section 3, "Capture less: BPF filters".
• Lower it on --multi-device runs. The cost is per device: eight interfaces at the default reserve half a gigabyte of kernel memory.

Asking the kernel for a large ring can fail with ENOMEM on a small or loaded host. sipnab does not treat that as fatal — it halves the request and retries, down to a 2 MiB floor, and tells you when it settled for less:

'eth0': the kernel refused a 64 MiB capture buffer; capturing with 16 MiB
instead. This host will tolerate a smaller burst before dropping — watch the
drop counters, and set -B/--buffer explicitly to pin a size.

sipnab honors an explicit small -B exactly and never promotes it upward: -B 1 on a constrained box means 1 MiB.

pcap_set_buffer_size takes a C int, so 2047 MiB is the last whole MiB whose byte count fits. sipnab clamps anything larger and says so:

-B/--buffer 5000 MiB exceeds the 2047 MiB ceiling (pcap_set_buffer_size takes a
C int); capturing with 2047 MiB instead.

The clamp matters because the arithmetic underneath used to wrap: --buffer 2148 once handed libpcap a negative byte count. Read the warning as a real limit rather than as advice — no -B above 2047 buys anything.

Two settings carry the word, and they sit on opposite sides of the kernel boundary. -B sizes the kernel ring. --buffer-budget sizes the in-process queue between the capture thread and the processing loop, in MiB, and it defaults to 64. sipnab turns that budget into a packet count at an assumed 2 KiB average, then clamps the result to between 10,000 and 5,000,000 packets.

Raise it when processing lags behind a bursty capture the ring is already absorbing. It does nothing for a ring that overflows: the kernel discards those packets before this queue ever sees them.

The cheapest packet is the one the kernel never gives you. A BPF filter runs in the kernel, before the ring, so filtered traffic costs no buffer space, no copy, and no parse.

A live capture already has one. Give sipnab no expression of your own and it compiles one from --portrange (default 5060-5061), then adds an arm that reaches SIP inside one VLAN tag, QinQ, a PPPoE Session header or an MPLS label stack. It logs the expression it installed, so -v shows you exactly what the kernel is running. The baseline is therefore signaling-only already, and this section is about narrowing further or about letting media back in.

Your expression replaces that one entirely. sipnab never edits what you typed, so an expression of your own drops the encapsulation arm and makes --capture-tunnels inert. Two warnings cover that: one when a port-based expression shows no sign of handling encapsulation, and one naming --capture-tunnels as ignored when you passed it beside a filter of your own.

Careful with the flag name: --filter is sipnab's own matching language, applied to messages after capture. The BPF expression is the trailing argument, or a file named by --bpf-file.

Only SIP signaling, and nothing tagged:

sudo sipnab -N -d eth0 "port 5060 or port 5061"

Signaling plus one media range:

sudo sipnab -N -d eth0 "port 5060 or (udp portrange 10000-20000)"

One customer's traffic:

sudo sipnab -N -d eth0 "host 203.0.113.10"

This is the correct first response to sustained drops with a busy CPU. Halving the traffic that reaches userspace is worth more than any buffer size.

Careful with RTP. Filtering to port 5060 alone gives you signaling with no media, so every stream turns orphan and every MOS figure disappears. If you want quality metrics, the filter must admit the negotiated media ports too.

Default: 65535 — the whole frame.

The obvious reading of --snaplen is "how many bytes of each packet get copied", and on that reading it looks unimportant: a 1500-byte frame costs 1500 bytes whether the cap is 1600 or 65535. That reading is incomplete, and on a busy server it is the expensive kind of incomplete.

On Linux, snaplen also determines how many packets your ring can hold — on the TPACKET_V2 ring. Which ring you get depends on the run mode, and the subsection at the end of this section is the rule. Read it before applying the arithmetic here to a headless capture. libpcap's create_ring() sizes each slot in the V2 ring from the snapshot length:

frame_size = handle->snapshot;
/* ... clamped for Ethernet ... */
req.tp_frame_size = TPACKET_ALIGN(macoff + frame_size);
req.tp_frame_nr   = (handle->opt.buffer_size + req.tp_frame_size - 1)
                    / req.tp_frame_size;

The slots are fixed size, so -B buys you buffer_size / frame_size packets, not bytes of useful queue. There is a clamp that can rescue you — but read its guard carefully:

if (handle->linktype == DLT_EN10MB) {
        ...
        if (offload)
                max_frame_len = MAX(mtu, 65535);
        else
                max_frame_len = mtu;
        max_frame_len += 18;
        if (frame_size > max_frame_len)
                frame_size = max_frame_len;
}

The clamp only applies to DLT_EN10MB — real Ethernet. sipnab's default capture device on Linux is any (src/capture/device.rs:38-40, chosen because SIP servers often listen on loopback), and any is DLT_LINUX_SLL2, not DLT_EN10MB. So on the default configuration no clamp runs at all and the slot stays at the full snaplen:

So out of the box — default device, default snaplen — a 64 MiB ring holds about a thousand packets, milliseconds of slack and roughly forty times less than the same memory would buy at a smaller snaplen. Before this raised the default from 2 MiB, that same arithmetic gave 31 slots.

Note what this means: naming an interface explicitly and disabling offloads is worth far more than either alone, because only that combination reaches the clamp. Section 5, "Which interfaces: any versus named" is the decision guide for the device half of that — what leaving any gains you, what it costs, and how to check you did not drop a call leg on the way.

# Signaling-focused capture: ~36,000 slots in the same 64 MiB
sudo sipnab -N -d eth0 --snaplen 1600

Three ways out, and they compose:

1. Cap the snaplen (above) — immediate, no root beyond capture.
2. Name the interface (section 5, "Which interfaces: any versus named") — the only way to reach DLT_EN10MB at all, and the prerequisite for the clamp below. It is a coverage trade, so read section 5 before making it.
3. Turn the offloads off (section 7, "Interface and driver drops") — which is independently correct for capture fidelity, because GRO/LRO hand you reassembled super-frames that were never on the wire.

--capture-profile picks the number for you. signaling asks for a snaplen of 1500 — one MTU, which keeps a whole INVITE carrying a long Record-Route set, ISUP encapsulation or a fat SDP offer, while dropping the bulk of every RTP packet. full is 65535, the default. An explicit --snaplen wins over the profile, because someone who typed a number has already answered the question the profile asks.

sudo sipnab -N -d eth0 --capture-profile signaling

Truncation is lossy, and not everything survives it. A small --snaplen breaks audio reconstruction — the TUI's WAV save and the MCP export_audio tool both need whole RTP payloads — and it degrades -O capture re-emit to truncated frames. sipnab tracks captured versus original length per packet, so truncation is visible rather than inferred — but choose the value deliberately, not reflexively.

To limit only how much sipnab parses without truncating what it captures, use -S/--limitlen instead. That is a parser bound, not a capture bound.

Modern libpcap on Linux can use the block-based TPACKET_V3 ring, which sizes its blocks independently of the snapshot length and so does not have the capacity cliff described above. One flag decides whether you get it, and sipnab decides that flag for you.

The rule is in libpcap's prepare_tpacket_socket():

/*
 * The only mode in which buffering is done on PF_PACKET
 * sockets, so that packets might not be delivered
 * immediately, is TPACKET_V3 mode.
 *
 * The buffering cannot be disabled in that mode, so
 * if the user has requested immediate mode, we don't
 * use TPACKET_V3.
 */
if (!handle->opt.immediate) {
        ret = init_tpacket(handle, TPACKET_V3, "TPACKET_V3");

So immediate mode reads like a latency preference and is really a ring-format choice. sipnab answers it by asking who consumes the packets (immediate_mode_for() in src/app/bootstrap.rs):

V3 is not free of its own trap, and sipnab pays for it explicitly rather than inheriting it: libpcap copies the read timeout into req.tp_retire_blk_tov and then polls with -1, so the timeout becomes added delivery latency rather than a poll bound. The interactive 100 ms would have meant up to 100 ms before a block retires. The batched path therefore uses its own BATCHED_READ_TIMEOUT_MS = 5 (src/capture/live.rs). Shutdown responsiveness (--duration, Ctrl-C) never depended on either: the handle is non-blocking, an empty ring returns TimeoutExpired, and the wait is sipnab's own bounded wait_readable().

So the snaplen arithmetic above binds the TUI, not a headless capture. On a headless run --snaplen and the offload settings still matter for capture fidelity and for copy cost, but they are not what decides how many packets the ring holds.

Unverified on hardware. Whether the kernel selects V3, and what that is worth, rest on reasoned from libpcap's source and not measured. strace -e trace=setsockopt for PACKET_VERSION, and KERNEL_DROPPED under load against a V2 baseline, are the two checks that would settle it.

Default on Linux: any — every interface at once, loopback included.

That default is deliberate, and it is a correctness choice, not a performance one. find_default_device() returns "any" on Linux (src/capture/device.rs:35-40), for the reason written beside it:

// On Linux, "any" captures all interfaces, which is what we want.
// SIP servers often listen on loopback, so capturing only eth0 misses traffic.

That is a real hazard, not a hypothetical one. A B2BUA talking to a registrar over 127.0.0.1, a containerized stack bridging SIP across docker0, a proxy handing calls to a media server over a veth pair — capture eth0 alone and those legs are simply absent. A dialog missing one leg does not come back smaller, it comes back wrong, exactly as section 1, "Are you dropping packets?" describes for dropped packets. any is the setting that does not lose calls.

It is also the slowest and least capable device sipnab can open, on four counts. Three are performance. One is a second correctness cost that runs the other way.

1. It forfeits about 40x of your ring capacity — on the V2 ring, so on the TUI. Section 4, "Snapshot length" has the mechanism and the run-mode rule that scopes it: libpcap's create_ring() sizes each TPACKET_V2 slot from the snaplen, and the clamp that cuts a slot down to MTU+18 sits behind if (handle->linktype == DLT_EN10MB). any reports DLT_LINUX_SLL2, so that clamp never runs. Every slot is the full 65535-byte snaplen regardless of interface MTU, and no ethtool setting can reach it — the guard tests the link type, not the offloads. At the 64 MiB default that is ~1,000 slots against ~41,000 for a named Ethernet interface with offloads off. Before the default buffer still defaulted to 2 MiB, the same arithmetic gave any just 31 slots.

2. It cannot go promiscuous. The capture loop in src/capture/live.rs computes let use_promisc = config.promisc && device != "any" — the pseudo-device does not support promiscuous mode, so sipnab does not ask for it. Promisc is on by default for a named device and --no-promisc turns it off. On any there is nothing to turn off. The consequence is that any misses traffic not addressed to the host, which matters on precisely the deployment where you would want it: a SPAN port or tap feeding mirrored calls the capture host is not a party to. That is a correctness cost, and it points the opposite way from the loopback argument — any sees every interface but only the host's own traffic on them.

3. It runs one capture thread. Naming devices unlocks --multi-device, which spawns one coordinator thread plus one capture thread per interface (start_multi_capture() in src/capture/native.rs), each with its own ring and its own drain loop. any is one device, so it is one thread and one ring no matter how many interfaces the traffic actually arrives on. --cores N (section 8, "On a live device it means capture sockets, not workers") is the other way to get more than one socket, and it works on a single device rather than on a list.

4. It sweeps interfaces you never wanted. any also picks up loopback, docker0, veth pairs, tunnels and management interfaces. Every one of those packets costs a BPF evaluation and, if it passes, a copy into the same ring the traffic you do want is competing for.

Stay on any when any of these hold:

• You are diagnosing rather than monitoring.
• You do not yet know which interface carries the traffic.
• SIP genuinely crosses loopback or container bridges and you have not enumerated those interfaces.
• The drop counters from section 1, "Are you dropping packets?" read no drops. If it is not dropping, it is not costing you anything worth this trade.

Name your interfaces when any of these hold:

• Section 1, "Are you dropping packets?" shows sustained kernel buffer drops.
• You are running a long-lived headless capture on a known topology.
• You need promiscuous mode because the switch mirrors the traffic to you rather than addressing it to you.
• You have several busy interfaces and one thread cannot drain them all.

Name what you need, open them in parallel, and turn the offloads off so the clamp can run:

First disable the offloads, once per interface you intend to capture:

sudo ethtool -K eth0 gro off lro off gso off tso off

Then name those interfaces and open them in parallel:

sudo sipnab -N -d eth0,eth1 --multi-device -B 256 "port 5060 or port 5061"

Naming the devices buys all four counts at once: the slots are now clamped to MTU+18 so the same 64 MiB holds tens of thousands of packets instead of ~1,000, promiscuous mode is back, there is one capture thread per interface, and nothing goes to docker0. Neither half works alone — any cannot reach the clamp however you set ethtool, and a named device with offloads on is still stuck at ~65 KB slots (section 4, "Snapshot length").

Verify you did not lose a leg. This is a deliberate trade of coverage for throughput, and the failure mode is silent — calls do not error, they just stop being complete. Before you keep it, confirm the interface list is right:

ip -brief address

If any SIP endpoint answers on 127.0.0.1 or a container bridge, add lo or docker0 to the -d list rather than accepting the gap. --multi-device costs -B per device (section 2, "The kernel capture buffer"), so eight interfaces at the default reserve half a gigabyte — name what you need and no more.

• --no-rtp — skip RTP/RTCP entirely when you only care about signaling. RTP is ~93% of carrier traffic by packet count, so this is the largest single reduction available short of a BPF filter.
• --no-dialog — skip dialog reconstruction.
• -N / --no-tui — do not render a TUI you are not watching.
• --quiet-bad-parse — silence per-packet parse notices on dirty links.
• Avoid per-message output (--json, --text-dump) on a high-rate live capture unless you are consuming it. Formatting and writing every message is real work on the capture path.

If interface/driver is non-zero, the packets never reached libpcap. -B is irrelevant. Look outside sipnab:

• NIC ring buffers — ethtool -g eth0 to inspect, ethtool -G eth0 rx 4096 to raise. This is the NIC's own ring, distinct from libpcap's.
• Offloads — GRO/LRO/GSO make the kernel hand you reassembled super-frames that no longer match what was on the wire. For accurate capture: ethtool -K eth0 gro off lro off gso off tso off.
• IRQ affinity and RPS — a single core servicing all NIC interrupts is a common ceiling on a busy server.
• A tap or SPAN port that is already oversubscribed — if the mirror source is dropping, nothing on the capture host can recover it.

--cores N names a different resource on each source, so read the one that matches your run.

Reading files (-I), it is N parallel reconstruction workers, sharded by host pair, each with its own dialog and RTP-stream stores. That is the case this section measures.

sipnab -N -I /var/captures/ --cores 4 --report

Measured on the reference corpus (benchmarks): 1.13M pkts/s at one core, 2.17M at two, 3.23M at four and 3.24M at eight. The curve flattens after four rather than declining, so four cores buys nearly all of it and eight buys almost nothing more — the gap between them is inside the noise of a single sweep.

Read the flattening as the shape it is. Four cores is where the returns stop, not a ceiling the eighth core falls off. Asking for eight costs the extra workers' memory for a throughput figure that does not move.

The workers compute the frame-provenance digest rather than the sequential reader, so that cost scales with the core count instead of capping every count at once.

--cores N on -d <device> asks the kernel to spread that interface across N AF_PACKET sockets through PACKET_FANOUT. Each socket gets its own ring and its own drain loop, which is the thing a ring that keeps overflowing actually needs — a bigger -B buys a deeper ring and still leaves one thread emptying it. Linux only: elsewhere sipnab captures on one socket and logs the reason.

Three things to know before reaching for it.

• It widens capture, not analysis. Every socket feeds the same channel and the same processing loop, so --cores 8 is not eight cores of reconstruction.
• -B is per socket. N sockets ask the kernel for N rings of that size, so --cores 8 at the 64 MiB default reserves half a gigabyte. sipnab states the total out loud when it opens the group.
• sipnab probes first, then commits. A kernel that refuses PACKET_FANOUT refuses it for every socket, so sipnab tests one throwaway handle, warns, and falls back to a single socket rather than letting N threads each discover it.

sudo sipnab -N -d eth0 --cores 4 -B 32 "port 5060 or port 5061"

Reach for it when section 1, "Are you dropping packets?" shows sustained kernel buffer drops on ONE busy interface and section 2 through section 5 have not cleared them. Several interfaces is --multi-device (section 5, "Which interfaces: any versus named") instead.

sudo sipnab -N -d eth0 \
  -B 256 \
  --snaplen 1600 \
  "port 5060 or port 5061 or (udp portrange 10000-20000)" \
  --report

Note that this already makes the interface choice from section 5, "Which interfaces: any versus named": it names eth0 rather than taking any, so confirm no SIP leg lives on loopback or a container bridge before adopting it.

Then read the drop line at the end. If it says no drops, stop there — and you can walk the settings back to recover fidelity. If it does not, work down section 2 (buffer) → section 3 (BPF) → section 5 (interfaces) → section 6 (less work) → section 7 (interface and driver drops) in that order, and re-measure after each change rather than applying all of them at once.

• CLI reference — every capture flag
• Configuration — the [capture] section
• Benchmarks — measured throughput and method
• Troubleshooting