Codec facts check - NexusTomorrow/server GitHub Wiki
The client facts table and the check that holds every declaration to it: the table's format, how two layouts are compared, how a disagreement is pinned, and where the check stands. Why the codec is checked this way is on Codec.
The client facts table
Rows
One JSON Lines row per wire opcode, 1,119 rows: 366 client to server, 718 server to client and 35 both ways. A row holds the direction, and for each side whether the layout is fixed or computed, its total bits on the empty path, its field widths in order, where its computed parts (strings, arrays, unions, blobs) sit, and the client's full layout in the layout tree notation, each field the client reads as a signed integer, an IEEE float or a packed float marked s, f or h. The widths are taken from the tree, so where the client copies raw bytes they're bytes:
{"op":154,"dir":"c2s","c2s":{"kind":"fixed","bits":81,"widths":[32,16,32,1],"tree":"32 16 32 1"}}
The summary
wire-facts-16042.meta.json records the format, the build and the counts:
| Key | Value |
|---|---|
schema |
2 |
build |
16042 |
rows |
1119 |
sides_without_tree |
3: the client envelopes ClientEnvelope (0x0244), NestedMessage (0x025C) and TaggedMessage (0x038C), checked on the empty path |
float_fields |
417 server to client and 224 client to server: the fields the client reads as IEEE floats |
half_fields |
0 server to client: the fields the client reads as packed floats |
signed_fields |
3 server to client: the fields the client reads as signed integers |
The table is our own numbers, public beside the codec, so public CI checks it with no client data. It holds for both the 64-bit and the 32-bit client of build 16042, whose trees agree on 1,145 of the 1,154 sides. Six, all envelopes, can't be compared between the two, and three differ: in each, the 32-bit client also accepts two selector values far outside the range, and in 0x098B one arm differs as well, settled when its area is read.
The comparison
Same bits on every path
Every declared message's full tree is compared with the client's, node by node. Two layouts agree when they put the same bits on every path. Three pairs of forms are treated as equal before comparing, because the wire can't tell them apart:
| Form | Equal to |
|---|---|
| An array counted by a 1-bit field | A union of nothing or one element |
| Consecutive arrays sharing one count and holding fields of one width | A single array of their elements together |
| Fields or strings every arm of a union ends with | The same reads after the union |
A run of fields split differently agrees if the totals meet, nothing later reads those fields as a count or a selector, and the run crosses no string, array, union or blob: one 14-bit field against an 8-bit and a 6-bit, for example. The same widths in another order, such as 8 then 16 against 16 then 8, are still reported.
The split gate
A message at Fields, or one whose layout is confirmed, may not split a value the client reads whole. The check fails such a declaration that draws one client field as several unless the split is listed as settled. Three are. In two arms of UnitMovement (0x0638), each element of a packed vector list is one 48-bit field to the client, which holds three packed floats, and the declaration holds it as those three 16-bit floats. In CombatLogOtherPlayersSet (0x0248), the declaration takes the one bit with a meaning apart from the 31 the client reads with it. One declared field over several the client reads doesn't trip it. It can't see a split past the first node of a sequence that doesn't line up, which is a reported disagreement anyway, on a side with no client tree, or in an arm only one side has.
A confirmed layout
A message marked [LayoutConfirmed] agrees with the client facts on every side it travels, field for field, with no split of one field into several but a settled one, and no pinned disagreement on it, and the facts give a layout for each of those sides. The marker check fails otherwise, so a confirmed layout can't drift from the table. The marked messages read "layout confirmed" in the Mark column on Messages.
Encodings
A field the client reads as a signed integer, an IEEE float or a packed float has to be read that way in the declaration. The client facts mark those fields, s, f or h in the tree, so a declaration that reads a marked field otherwise disagrees, while an unmarked field says nothing against any encoding the declaration gives it.
What's compared, and what isn't
- For a selector wider than 8 bits, the client facts hold its arms only at 0, 1, 2, 3,
0xFF,0x100, its top bit alone and all ones, so other values aren't compared. - A layout that contains itself is reported, not expanded.
- It catches wrong widths, missing or extra fields, wrong enum widths, opcodes swapped between different layouts, fields swapped when their widths differ, and anywhere in a full layout: an array's element, a missing or extra union arm, a string of the wrong kind, a count or selector read from the wrong field.
- It can't see swaps of equal width and encoding, or meaning. Generated round-trip tests cover every declared message on top of it.
Pins
How a pin works
Each disagreement is named by opcode, side and path, such as 0x013C server to client [1].element[1], and pinned with its reason. The test fails on a disagreement that isn't pinned, and on a pinned one that no longer occurs, so the list only shrinks when a fix lands. Each pin records the exact problem it covers, and the test fails when that problem's text changes, so a pin can't quietly come to cover a different fault in the same place. Several problems at one place are kept together on one pin.
Where it stands
Each run reports the check on its own lines. In the CI run in the header badge:
| Line | Reads |
|---|---|
| The facts check | "1119 declared messages: 1104 agree with the client facts; 18 disagreements pinned." |
| The split gate | "484 messages at fields; 10 layout-confirmed; 3 splits settled." |
Most pins read "the client's full layout differs; settled when its area is read", because the message audit redeclares each message as its feature area is checked. No pin names an encoding: every field the client facts mark is declared the way the client reads it.
Known edges
- The three client envelopes are checked on the empty path only, since the table holds no tree for them.
- Selectors wider than 8 bits are compared at their sampled values only.
Where the code lives
| Piece | Path |
|---|---|
| The client facts table and its summary | Source/NexusTomorrow.Network.Codec/Facts/ |
| The facts check, the comparison and the pins | Tests/NexusTomorrow.Network.Codec.Tests/Facts/ |
| The split gate and its settled list | Tests/NexusTomorrow.Network.Codec.Tests/Facts/FieldsSplits.cs |
| The marker check | Tests/NexusTomorrow.Network.Codec.Tests/Verification/RegistryMarkersTests.cs |