Internals Domain Primer - NormB/sipnab GitHub Wiki

Written for the Rust engineer who is new to VoIP. Nearly every subtle bug in this tree is a protocol-semantics bug wearing a Rust costume.

• The code compiles.
• The tests pass.
• The number on screen is wrong. The protocol does not mean what it looks like it means.

Each concept names the file that encodes it. So this doubles as an index into the source.

The diagrams here use the same mermaid sequenceDiagram form sipnab itself exports.

• Press E in the Call Flow view to copy a diagram to the clipboard.
• Or press F2 and Tab through the save formats to Mermaid.
• You can regenerate every one of them from a real capture.

A transaction is one request plus its responses.

A dialog is the longer-lived relationship a successful INVITE establishes.

• It spans many transactions (INVITE, re-INVITE, BYE).
• It is what a human means by "a call".

RFC 3261 identifies a dialog by the triple Call-ID + From-tag + To-tag.

• sipnab's DialogStore keys its map on Call-ID alone.
• It keeps from_tag/to_tag as fields on SipDialog.
• That is a deliberate simplification for a capture tool.
  • At capture time the To-tag does not exist yet (it arrives in the first response).
  • So keying on the full triple would mean re-keying every dialog mid-flight.

sipnab still captures the tags. It tells forked calls that share a Call-ID apart downstream, not by the map key.

• to_tag takes the first To tag seen.
• A stronger one replaces it:
  • a 101-199 response to the opening request;
  • then a 2xx to it.
• A 401 or 407 challenge carries a tag of its own and creates no dialog. So the answer to the re-sent request replaces that tag.
• A tag equal to from_tag never becomes to_tag. It is the answer to a request the remote party sent.

Two things must be knowable before a message gets a dialog at all: its Call-ID, and its method.

• The method requirement is the less obvious one.
• It exists because SipDialog::method takes its value once at creation and never corrected.

A response derives it from CSeq.

• A malformed response has Call-ID present and CSeq absent.
• Such a response used to create a dialog under that Call-ID, labeled with an invented method.
• The genuine INVITE arriving afterwards matched that entry instead of creating its own.
• The label then outlived the capture.

Such a message now creates no dialog.

• It is still captured, counted, and searchable.
• The INVITE that follows creates the dialog correctly.

Every request carries CSeq: <number> <METHOD>. Responses echo it. That echo is the only reliable way to know which request a response answers.

This is not pedantry.

• update_timing() records the initial INVITE's CSeq number in invite_cseq.
• It does this so that a re-INVITE's 200 OK cannot overwrite answered_at.
• Without that pin, a call put on hold twenty minutes in reports a twenty-minute setup time. That is a plausible-looking number that is simply false.

A 401 or 407 is the server saying "try again with credentials".

• Nearly every real call starts with one.
• Treating it as a final failure would report most of a healthy carrier's traffic as failed.

SipDialog therefore treats 401/407 as intermediate.

• The reported outcome is the maximum non-challenge final response.
• The challenge only becomes the answer for a call that drew a challenge and never authenticated.

That rule now holds for the dialog state as well, not only the reported code. It did not for a long time.

• Only the REGISTER handler skipped challenges.
• So a challenged INVITE went to Failed.
• The 2xx that followed could not lift it back out. That transition only admits the pre-answer states.
• A captured BYE hid the result by forcing Completed. That is why the defect survived.
• The calls it misreported were:
  • the ones still up;
  • the ones whose BYE never made it into the capture.

The exchange, with what each hop tells the analyzer.

sequenceDiagram
    autonumber
    participant UAC as caller
    participant Proxy
    participant UAS as callee

    UAC->>Proxy: INVITE (CSeq 1 INVITE, From-tag A)
    Proxy-->>UAC: 407 Proxy Authentication Required
    Note over UAC,Proxy: not a failure — outcome stays pending
    UAC->>Proxy: ACK (CSeq 1 ACK)
    UAC->>Proxy: INVITE (CSeq 2 INVITE, Authorization)
    Proxy->>UAS: INVITE
    UAS-->>Proxy: 100 Trying
    UAS-->>Proxy: 180 Ringing (To-tag B)
    Proxy-->>UAC: 180 Ringing
    Note over UAC,UAS: PDD is measured to the 180, not the 100
    UAS-->>Proxy: 200 OK
    Proxy-->>UAC: 200 OK
    UAC->>UAS: ACK (CSeq 2 ACK)
    Note over UAC,UAS: dialog identity complete — Call-ID + both tags

INVITE alone among SIP methods takes a separate ACK transaction to confirm. The code encodes two consequences:

• sipnab records the ACK but makes no state transition. The 200 already moved the dialog to InCall.
• An ACK may carry SDP. See delayed offer below.

Non-INVITE transactions (REGISTER, OPTIONS, MESSAGE) are simple request/response with no ACK at all.

pdd_ms() in timing.rs measures INVITE → first 180/183.

• A 100 Trying does not count.
  • It means "I got your request", not "the callee's phone is ringing".
  • Any proxy emits it immediately.
• Measuring to the 100 would report an excellent PDD for a call the caller experienced as ten seconds of silence.

Normally the request carries the SDP offer and the response carries the answer.

RFC 3261 section 13.2.1 allows an offerless INVITE. Then the roles invert:

• the 200 OK carries the offer;
• the ACK carries the answer.

determine_offer_answer() encodes exactly that.

• An ACK is always an answer.
• A response bearing SDP with no preceding offer in the dialog is itself the offer.
• Label these by message type alone, and every delayed-offer call in the capture is backwards.

sequenceDiagram
    autonumber
    participant UAC as caller
    participant UAS as callee

    UAC->>UAS: INVITE (no SDP body)
    Note over UAC,UAS: offerless — nothing to record in the SDP timeline yet
    UAS-->>UAC: 200 OK + SDP
    Note over UAS: no preceding offer exists, so THIS is the offer
    UAC->>UAS: ACK + SDP
    Note over UAC: an ACK is never an offer — this is the answer

A re-INVITE with a=sendonly puts the far end on hold. a=sendrecv resumes.

• sdp.rs parses sendonly/recvonly/inactive into a direction on the media description.
• The older RFC 2543 convention holds by setting the connection address to c=0.0.0.0.
  • That is not recognized as hold here.
  • Such a call reads as media simply stopping.
• Media stopping mid-call is therefore not automatically a fault. Check the SDP timeline before calling it one-way audio.

sequenceDiagram
    autonumber
    participant A as party A
    participant B as party B

    A->>B: re-INVITE + SDP (a=sendonly)
    B-->>A: 200 OK + SDP (a=recvonly)
    A->>B: ACK
    Note over A,B: RTP from B stops — expected, not a fault
    A->>B: re-INVITE + SDP (a=sendrecv)
    B-->>A: 200 OK + SDP (a=sendrecv)
    A->>B: ACK
    Note over A,B: media resumes — the SDP timeline is the evidence

CANCEL asks to abandon an INVITE with no final response yet.

• The callee's 200 OK may cross it on the wire.
• Then both exist in the capture.
• The naive reading ("last response wins") gives the wrong outcome.

The transition table in dialog_state_machine.rs resolves this by CSeq method.

• A CANCEL request moves the dialog to Canceled.
• So does the 487 on its own.
• Either is sufficient.
  • A CANCEL can travel a different path from the response.
  • A capture can begin mid-dialog.
  • Requiring both once left a canceled call sitting in Ringing forever.
• The 487 is the reported outcome.
• The 200 that merely acknowledged the CANCEL transaction drops out. It belongs to a different CSeq.

The transaction is a coordinate of the table, not a filter the code applies afterwards. This exchange shows why.

• A capture may open on any of these five messages.
• Open on the CANCEL or the 487, and the caller gave up.
• Open on the 200, and nothing yet says how the call ended.
  • It is the same family and the same code that establishes a call one line above.
  • It has the opposite meaning.

The sibling case runs the other way.

• A 2xx answering a BYE is evidence the session ended (RFC 3261 section 15.1.2).
• So a call whose BYE fell outside the capture still leaves InCall.

sequenceDiagram
    autonumber
    participant UAC as caller
    participant UAS as callee

    UAC->>UAS: INVITE (CSeq 1 INVITE)
    UAS-->>UAC: 180 Ringing
    UAC->>UAS: CANCEL (CSeq 1 CANCEL)
    UAS-->>UAC: 200 OK (CSeq 1 CANCEL)
    Note over UAC,UAS: this 200 answers the CANCEL, not the INVITE
    UAS-->>UAC: 487 Request Terminated (CSeq 1 INVITE)
    UAC->>UAS: ACK
    Note over UAC,UAS: outcome is 487 Canceled — filtering by CSeq method is what gets this right

A B2BUA (SBC, PBX) terminates one call and originates another. So one human call is two Call-IDs with no shared identifier by default.

dialog_store.rs correlates legs seven ways. Each has a confidence score and its own reported reason:

The two charging-vector rows are one header and two different claims. That is why they are two reasons rather than one with a sub-field. The argument, and what is still unverified about it, is docs/design/icid-correlation.md.

SdpOrigin compares the whole uniqueness tuple RFC 8866 defines: <username> <sess-id> <nettype> <addrtype> <unicast-address>.

• It never compares sess-id alone.
  • The RFC recommends deriving it from a timestamp.
  • So two unrelated calls from one user agent can share it.
• It excludes sess-version deliberately. So a re-INVITE for hold or a codec change does not break the match.

Two of those scores are 100, and they are not interchangeable.

• SessionId is a standard. Its entire purpose is surviving intermediaries that rewrite everything else.
• XCallId is a vendor convention. It works only when someone configured it.
• Reporting them separately is what lets a reader tell how far to trust a call tree.

The bottom row is the one to be careful with.

• "Same endpoint IP within two seconds" is a guess.
• On a busy SBC, many unrelated calls share an endpoint IP inside that window.
• It exists because most deployments set no correlation header at all.
• See session_id.rs for two things:
  • why the halves of a Session-ID swap across the SBC;
  • why matching is therefore set intersection rather than string equality.

An RTP stream carries its identity in its SSRC, a 32-bit random number. It does not carry it in any SIP field.

• It has no Call-ID and no From.
• Nothing links it to signaling except the IP/port pair the SDP advertised.
  • Only SDP in an offer or an answer links a stream. That is SDP in an INVITE, ACK, PRACK or UPDATE, or in a 1xx or 2xx response to one.
  • SDP in any other message links no stream. An OPTIONS 200 OK, for example, describes capabilities and sets up no session (RFC 6337 section 2.4).
  • sdp_sets_up_media holds this rule for every capture path.

That explains three things:

• why StreamStore is a first-class store rather than a child of the dialog store (D13);
• why sipnab discovers streams heuristically when it never saw their SDP;
• why the --cores merge needs a re-association pass. The media and the signaling can be sharded to different workers.

The 16-bit sequence number wraps roughly every 20 minutes of voice.

• Loss detection compares wrapping_add(1) against the received sequence (stream.rs).
• A plain > comparison reports one enormous loss burst per wrap on every long call.

The RTP timestamp is a media sample counter at the codec's clock rate. It is not a time value. Converting it needs one of these:

• clock_rate_from_pt() for static payload types;
• the a=rtpmap clock rate for dynamic ones.

G.722 is the trap worth knowing.

• RFC 3551 assigns it a 8000 Hz RTP clock despite 16 kHz audio.
• So the obvious "clock rate = sample rate" assumption halves or doubles every derived duration.

RFC 3550 section 6.4.1 defines interarrival jitter as a smoothed mean of the difference in transit time between consecutive packets: J(i) = J(i-1) + (|D(i-1,i)| - J(i-1)) / 16.

• stream.rs computes the transit delta as a signed i32 before taking the absolute value.
• With unsigned arithmetic, a single reordered packet underflows. It reports a jitter spike of about 4.29 billion.

RTCP receiver reports carry a jitter field too, but in RTP timestamp units.

• stream_store.rs converts with the stream's clock rate before storing it.
• So the RTCP-reported and locally measured numbers are comparable.
• MOS gets a millisecond value either way.

The report block is where two of this codebase's historical bugs lived. Both are visible in the same six fields.

sequenceDiagram
    autonumber
    participant S as sender
    participant R as receiver
    participant Cap as sipnab

    S->>R: RTCP SR (NTP + RTP timestamp, packet/octet counts)
    R->>S: RTCP RR report block
    Note over R,S: fraction_lost, cumulative_lost, highest_seq, jitter, last_sr, delay_since_sr
    R->>Cap: same RR observed on the wire
    Cap->>Cap: sign-extend cumulative_lost from 24 bits (negative = net duplicates)
    Cap->>Cap: jitter x 1000 / clock_rate to reach milliseconds
    Note over Cap: skip either step and the numbers are wrong, not missing

estimate_mos_with_delay() is an E-model computation from jitter, loss, codec and one-way path delay.

• It is a model output on the 1.0–4.5 scale.
• It is not an opinion score from a listener.
• Say "estimated MOS" in anything user-facing.
• The distinction is the difference between a tool an engineer trusts and one they re-derive.

The delay term is the one input a passive tap cannot measure. So MosDelay resolves it per stream, and every surface scores through that. The order is:

• what the operator declared;
• then what an endpoint reported in an RTCP XR VoIP-metrics block;
• then what sipnab derives from a receiver report's sender-report echo;
• then a labeled assumption.

Score a stream any other way, and two surfaces report two numbers for one call.

• G.107's delay penalty has a knee at 177.3 ms.
• A call past it reads more than a full MOS point too high on the assumption.

• Ten percent loss in one clump is a dropped word.
• Ten percent scattered is a faint crackle a codec's concealment mostly hides.
• analyze_burst_gap() classifies which it is.
• The loss map view renders it in sequence space.

RFC 4733 telephone-events carry digits as their own payload type, negotiated in SDP.

• extract_dtmf_with_clock() needs both the negotiated PT and its rtpmap clock rate.
• Event duration arrives in clock ticks.
• So decoding a 16 kHz telephone-event with an assumed 8 kHz clock reports every digit as twice its real length.

Endpoints normally send from the same port they receive on ("symmetric RTP").

• Behind NAT, the address in the SDP is the private one.
• The media actually arrives from a translated address.
• So the naive match of SDP address against observed source fails.
• The stream looks unassociated.

diagnose_media() is where sipnab untangles this.

• It infers one-way audio from the directed-endpoint set.
• It infers nat_mismatch from SDP-versus-observed address disagreement.
• The two combine into the diagnosis an operator reads.
• It also checks whether comfort-noise frames explain an asymmetry before flagging it. Silence suppression is not a fault.

sequenceDiagram
    autonumber
    participant A as caller (behind NAT)
    participant NAT
    participant B as callee

    A->>B: SDP offer c=192.168.1.10 port 40000
    Note over B: private address — unroutable from here
    B->>NAT: RTP to the advertised address
    A->>NAT: RTP from 192.168.1.10:40000
    NAT->>B: RTP from 203.0.113.7:52344
    Note over B: source does not match the SDP — nat_mismatch
    Note over A,B: media flows one way — diagnose_media reports both signals together

Every one of these was a real defect in this codebase, found and fixed. They are worth reading as a set.

• None of them is a Rust mistake.
• Each is a place where correct-looking code encoded a wrong protocol assumption.

The lesson generalizes. When a number looks wrong, the bug is usually not in the arithmetic. It is in what someone assumed the field meant.