Capture SIP over TLS - NormB/sipnab GitHub Wiki

You have SIP on port 5061. sipnab shows you nothing useful. You want to see the signaling. This page picks the method for you.

For many production deployments this is not an advanced topic — it is the only topic.

• A hardened SIP server often does not listen on 5060 at all.
• So every call is TLS, and a capture that reads only plaintext reads nothing.
• If that is your situation, start at the table below. Do not start with the plaintext quick-start elsewhere in these docs.

The short version. The wire alone never yields encrypted traffic.

• Modern TLS uses forward secrecy. So even the server's private key does not open a captured session.
• Something on a machine you control has to hand over one of these:
  • the session keys,
  • the plaintext.
• Every method below is one of those two. They differ only in what access each demands.

Answer these in order.

• Each branch ends at one method below.
• The question that separates two branches is always an access question. It is never a protocol question.

flowchart TD
    S["SIP on 5061, and sipnab decodes nothing"] --> Q1{"Can you set SSLKEYLOGFILE<br/>on an endpoint and restart it?"}
    Q1 -->|yes| M1["The endpoint writes a key log: --keylog<br/>full decryption, live or from a pcap"]
    Q1 -->|no| Q2{"Can you get root on the SIP host?"}
    Q2 -->|no| Q6{"Old TLS 1.2 server on RSA key exchange,<br/>and you hold its private key?"}
    Q6 -->|yes| M6["The old RSA case: --tls-key<br/>non-PFS handshakes only"]
    Q6 -->|no| X["Nothing opens it.<br/>Read What does not work, and why"]
    Q2 -->|yes| L["Check what you have: sudo sipnab --uprobe-list<br/>names the TLS libraries this host runs"]
    L --> Q3{"Do you want keys you can keep,<br/>or plaintext right now?"}
    Q3 -->|keys| M5["Lift keys from a daemon: eCapture, then --keylog<br/>daemon untouched, and the pcap stays readable"]
    Q3 -->|plaintext| Q4{"BTF kernel, and a build<br/>carrying the bpf feature?"}
    Q4 -->|yes| M4["Plaintext and the peer address: --uprobe-tls --uprobe-backend bpf<br/>plaintext with the real 5-tuple"]
    Q4 -->|no| M3["No keys at all: --uprobe-tls<br/>plaintext, no peer addresses"]

The table says the same thing in rows. Read down the first column. Stop at the first row you can satisfy.

Most people want --keylog or --uprobe-tls.

• --keylog if you are testing and control an endpoint.
• --uprobe-tls if you are on a production box and cannot restart anything.
  • That is the case sipnab exists for.
  • It is the one people assume is impossible.

This is the ordinary case, and the cheapest. Any TLS library built on OpenSSL, NSS or GnuTLS writes session keys to the file named by SSLKEYLOGFILE.

# Run all of these, in order.
export SSLKEYLOGFILE=/tmp/sip-keys.log
# start your softphone or daemon from this shell, then:
sudo sipnab -d eth0 --keylog /tmp/sip-keys.log --keylog-watch

--keylog-watch re-reads the file as it grows.

• So keys minted after sipnab starts still decrypt.
• Without it you get only the sessions whose keys were already written.

The same flag reads a capture you already recorded:

sipnab -I tls-capture.pcap --keylog /tmp/sip-keys.log

The full recipes are cookbook recipes "Live decryption (UA produces keys, sipnab follows)" to "Decrypt without writing the keys to disk", under recipe "Decrypt SIP over TLS with a key log file". They include:

• exporting a decrypted pcap for Wireshark,
• keeping keys off disk entirely.

sudo sipnab --uprobe-list

This installs nothing. It prints:

• the TLS libraries running on this host,
• which processes use them.

That tells you whether the methods "No keys at all" and "Plaintext and the peer address" are even available. You learn it before you spend time on them.

This is the method the long-standing upstream feature request asks for: TLS visibility with no recompile and no certificate. It is what sipnab does here.

sudo sipnab -N --uprobe-tls

sipnab attaches a uprobe to the TLS library's write function. It reads the bytes before the library encrypts them.

• It recovers no key.
• It decrypts nothing.
• It touches no other machine.

Dialogs carry no addresses and port 0. Their label is uprobe:<process>/<pid>.

• A uprobe sees the bytes an application handed its TLS library.
• It sees nothing about the socket underneath.
• So sipnab names the process rather than inventing a peer.
• If you need the addresses, use the method "Plaintext and the peer address".

If it attaches and reports zero messages, the symbol is almost certainly wrong. OpenSSL 3 applications increasingly call SSL_write_ex rather than SSL_write:

# Run all of these, in order.
nm -D --undefined-only /usr/sbin/opensips | grep -i ssl_write
sudo sipnab -N --uprobe-tls --uprobe-symbol SSL_write_ex

Read the security implications before using this on a production host.

• It reads process memory.
• So anyone who can run it can read every SIP session on that machine, credentials included.

sudo sipnab -N --uprobe-tls --uprobe-backend bpf --portrange 0-65535

This adds a kernel probe on tcp_sendmsg. It pairs the probe with the write by thread. So dialogs carry the real 5-tuple.

It needs two things:

• a kernel with CONFIG_DEBUG_INFO_BTF=y,
• a sipnab that actually holds the kernel programs.
  • --features bpf compiles the loader.
  • The programs themselves need a nightly toolchain and bpf-linker on the build host.

A build missing either still succeeds.

• It produces a binary that lists bpf and refuses at attach time.
• The uprobe walkthrough linked at the foot of this page carries the message. It also carries the build-time switch that turns that gap into a build failure.

sipnab refuses rather than falling back in silence. A silent downgrade would leave you with no peers and no reason given.

Widen --portrange. The port a uprobe reports is whatever the socket used. That is usually ephemeral rather than 5061.

You may rather have keys than plaintext. Keys decrypt a pcap you keep, and plaintext does not.

• eCapture reads them out of a running process.
• sipnab consumes them unchanged:

# Run all of these, in order.
# eCapture picks the TLS library to instrument by looking at curl. Your SIP
# daemon may well map a different one, so name the daemon's explicitly — this
# is the single most common reason a keylog stays empty.
LIBSSL=$(sudo awk '/libssl/ {print $6; exit}' /proc/"$(pgrep -o opensips)"/maps)
sudo ecapture tls -m keylog --libssl="$LIBSSL" --keylogfile=/tmp/keys.log &
sudo sipnab -d eth0 --keylog /tmp/keys.log --keylog-watch

The uprobe attaches to the library, not to one process.

• So a forking daemon needs no --pid.
• It covers every worker that maps that path.
• --pid would restrict you to one of them.

Check the path it chose in eCapture's own startup line (openssl_path=) before trusting the run.

Start the capture before the connections you want to read. This is the single biggest trap on long-lived trunks. It bites differently in each TLS version:

• TLS 1.3 (RFC 8446) numbers each record with a counter both endpoints keep privately.
  • Nothing on the wire carries it, so sipnab searches for it.
  • The search covers about a million records. That is roughly a day of a trunk ticking over at ten records a second.
  • The search costs one AEAD tag check per candidate. It runs once per direction per session.
  • So it is a one-off of about a second, not a per-record cost.
  • Past that, the keys are right and the records still do not open.
  • --tls-lockon-window moves that ceiling.
    • Raising it costs nothing on a connection captured from its handshake. The search widens only as records fail to open.
    • So raise it for a carrier trunk held open for days.
  • A separate per-run trial budget — four full windows — limits one case. It stops one session whose keys belong to some other connection from spending a window on every record it gets offered.
• TLS 1.2 (RFC 5246) is worse.
  • A CLIENT_RANDOM line gives the master secret.
  • The server random and cipher suite that expand it into record keys are in the ServerHello.
  • Miss the handshake and there is no way to use the secret at all, now or later.

sipnab says which of these happened, with counts. The fix is at capture time either way:

• Bounce the connection, or the far end, while capturing.
• So the capture catches the stream from its handshake and its first record.

Cookbook recipe "Decrypt traffic from a daemon you cannot restart" has the full sequence. Recipe "Decrypt without writing the keys to disk" shows feeding keys through a pipe, so they never reach disk.

sipnab -I capture.pcap --tls-key server.key

This works only for TLS 1.2 handshakes that used RSA key exchange.

• Any ECDHE/DHE handshake has forward secrecy. That is every modern configuration, and it is mandatory in TLS 1.3.
• With forward secrecy, the private key does not decrypt a recorded session.
• If this produces nothing, your traffic is almost certainly PFS. You need one of these methods:
  • "The endpoint writes a key log",
  • "No keys at all",
  • "Plaintext and the peer address",
  • "Lift keys from a daemon you cannot restart".

SRTP keys arrive two ways, and sipnab reads both:

• SDES — keys travel in the SDP.
  • So decrypting the signaling decrypts the media with it.
  • There is nothing extra to do.
• DTLS-SRTP — a DTLS handshake carries the keys.
  • Point --dtls-keylog at an NSS-format key log.
  • sipnab runs the RFC 5764 exporter over it to reach the SRTP keys.
  • See cookbook recipe "Decrypt SRTP from a DTLS keylog".

--dtls-keylog is a separate flag from --keylog. The two name different handshakes. If you already hold SRTP master keys rather than a handshake, --srtp-keys takes them directly.

Without a key, SRTP media is still measured.

• Packet counts, loss, jitter and timing come from the RTP header. SRTP leaves that header in the clear.
• The payload is not kept as audio.
• The SDP may declare an SRTP profile (RTP/SAVP, RTP/SAVPF, UDP/TLS/RTP/SAVP or UDP/TLS/RTP/SAVPF). Then sipnab keeps only the payload it decrypted.
  • A payload it could not decrypt is ciphertext, and decodes as noise.
  • A WAV export or a vCon for such a call then carries no audio. It says how many packets sipnab could not decrypt.
  • If sipnab decrypted only some packets, it marks the audio PARTIAL and counts the others.

Any run whose keys arrive while it runs has one race. Such runs are:

• "The endpoint writes a key log" or "Lift keys from a daemon you cannot restart", with --keylog-watch,
• keys fed through a descriptor with --keylog-fd.

"No keys at all" and "Plaintext and the peer address" use no keys. So they do not have it.

The race goes like this:

• You attach the key source to a daemon that is already running.
• A call arrives.
• The first records reach sipnab before the first key does.

On a SIP session the first record is the INVITE.

• So the symptom is not "a few records are unreadable".
• It is a call with no offer in it.
• sipnab then reports that as a media mismatch or a NAT problem. From the dialog's point of view, that is exactly what it looks like.

With --keylog-watch or --keylog-fd, sipnab holds application data it cannot open yet. It retries the data as soon as a key for that session turns up. No separate flag switches this on.

sipnab reads a plain --keylog file, without --keylog-watch, once at startup. It holds nothing then.

• No key arrives later in the run.
• So a record the loaded keys cannot open now never opens.

A run that recovered records says so:

TLS late decrypt: recovered 3 record(s) that arrived before their keys

The hold has limits. Otherwise a peer whose keys never arrive would grow it without end:

Those bounds are the reason to start the key source before the capture wherever you can. Recovery closes a gap measured in packets. It does not close one measured in minutes.

The run also says what the hold could NOT recover. These two are different problems with different fixes:

• "we never had the keys for those records",
• "we had them and had already discarded the ciphertext".

sipnab discarded 3 TLS record(s) held from before a key arrived: the
late-decrypt hold is bounded (4 MiB total, 16 record(s) per direction, 5s),
and it filled before any key matched them. Start the key source before the
capture, or expect the first message of a call to be missing.

That one argues for starting the key source earlier, or for a bigger bound. Records still held when the run ended produce a different line. That line argues for neither:

5 TLS record(s) were still waiting when the run ended: no key ever arrived
for them. Nothing was discarded and starting the capture earlier would not
have helped -- the key source never produced those secrets.

Stated plainly, because time spent here is time people lose:

• A packet capture alone.
  • No amount of sipnab flags decrypts a pcap with no keys.
  • If the handshake had forward secrecy, the information required is not in the capture and never was.
• The server's private key, on modern TLS. See the section on the old RSA case.
• A mirror port or tap, by itself. It gives you the same ciphertext as a local capture. You still need keys or plaintext from an endpoint.
• "No keys at all", "Plaintext and the peer address" or "Lift keys from a daemon you cannot restart", against a machine you do not control. They read process memory on the host they run on. That is the whole boundary.
• Attaching to a long-lived connection and expecting the back catalog.
  • Keys extracted from a running process decrypt records captured from that point on. That holds if the capture catches the stream early enough.
  • sipnab does recover records that reached it in the moments before the key did, as described above.
  • But that closes a gap of packets, not one of history.
  • Traffic that went past before the capture started is not in the capture. Nothing recovers it.

More in Troubleshooting and the uprobe walkthrough.