Network Message Protocool - nichnet/netplay GitHub Wiki

Network Message Protocool

The Netplay library implements a custom binary protocol optimized for real-time communication between clients and servers. Each message follows a fixed, easy-to-parse binary format designed for efficient network transmission.

Message Layout

Every message consists of four parts transmitted in this exact order:

  1. Length (2 bytes) - Total size of the message excluding these 2 bytes themselves
  2. Options (1 byte) - Bit flags for message options (compression, encryption, etc.)
  3. Type (2 bytes) - Message type identifier
  4. Payload (variable length) - The actual message data
[               Complete Message              ]
( Length ) ( Options ) (  Type  ) [  Payload  ]
( uint16 ) (  uint8  ) ( uint16 ) [    ...    ]
(   2b   ) (   1b    ) (   2b   ) [    ...    ]

Message Structure Details

Field Size Type Description
Length 2 bytes uint16 Size of Options (1b) + Type (2b) + Payload length
Options 1 byte uint8 Bit flags for message options
Type 2 bytes uint16 Message type identifier (0-65535)
Payload Variable bytes Serialized message data

Options Flags

The options byte uses individual bits to indicate message properties:

Flag Bit Position Description
Compressed 0 Message payload is GZIP compressed
Encrypted 1 Message payload is encrypted
Reserved 2-7 Reserved for future use

Data Serialization

The payload contains serialized data using the following format for each data type:

Primitive Types

Type Size Format
byte 1 byte Raw byte value
boolean 1 byte 0x00 (false) or 0x01 (true)
short 2 bytes Big-endian signed 16-bit integer
int 4 bytes Big-endian signed 32-bit integer
long 8 bytes Big-endian signed 64-bit integer
float 4 bytes Big-endian IEEE 754 single precision

Strings

Strings are encoded as UTF-8 with a length prefix:

[ Length ] [    UTF-8 Data    ]
[ uint16 ] [ variable length  ]
[   2b   ] [       ...        ]
  • Length: Number of bytes (not characters) in the UTF-8 encoding
  • Data: UTF-8 encoded string bytes

Arrays

Arrays are serialized with a length prefix followed by elements:

[ Length ] [  Element 0  ] [  Element 1  ] [ ... ]
[ uint16 ] [     ...     ] [     ...     ] [ ... ]
[   2b   ] [     ...     ] [     ...     ] [ ... ]
  • Length: Number of elements in the array
  • Elements: Each element serialized according to its type

Nested Objects

Nested NetworkSerializable objects are serialized recursively:

[ Has Object ] [ Object Length  ] [  Object Data  ]
[  boolean   ] [     uint16     ] [      ...      ]
[    1b      ] [       2b       ] [      ...      ]

Protocol Properties

  • Byte Order: Big-endian (network byte order)
  • Maximum Message Size: 65,535 bytes (uint16 limit)
  • Default Buffer Size: 1,024 bytes
  • Compression: Optional GZIP compression for payloads
  • String Encoding: UTF-8

Message Examples

Chat Message Example

Chat from "John" with message "Hello, World"

@NetworkMessageHandler(value = 2, compressed = true)
public class NetworkMessageChat extends NetworkSerializable {
    @NetworkSerializableProperty(0)
    public String getSender() { return sender; }
    
    @NetworkSerializableProperty(1) 
    public String getMessage() { return message; }
}

// Example data:
ChatMessage chat = new ChatMessage("John", "Hello, World");

Payload breakdown (before compression):

Part Size (bytes) Description
sender length 2b Length of sender (4b)
sender data 4b UTF-8 bytes for "John"
message length 2b Length of "Hello, World" (12b)
message data 12b UTF-8 bytes for "Hello, World"

Complete wire format:

[                               Complete Message                                  ]
[ (    length     ) (options) ( type ) [                  data                  ] ]
[ (      23b      ) (   -   ) ( CHAT ) [ ( 4 ) { John } ( 12 ) { Hello, World } ] ]
[ ( 1b + 2b + 20b ) (  1b   ) (  2b  ) [  2b  +   4b   +  2b  +       12b       ] ]

Hex representation (uncompressed payload):

00 17 01 00 02 00 04 4A 6F 68 6E 00 0C 48 65 6C 6C 6F 2C 20 57 6F 72 6C 64

Array Example

List of friends: Alice, Bob, Charlie

// Example data:
String[] friends = new String[] {
    "Alice",
    "Bob", 
    "Charlie"
};

Payload breakdown:

Part Size (bytes) Description
array length 2b Number of usernames (3)
array[0] length 2b Length of "Alice" (5b)
array[0] data 5b UTF-8 bytes for "Alice"
array[1] length 2b Length of "Bob" (3b)
array[1] data 3b UTF-8 bytes for "Bob"
array[2] length 2b Length of "Charlie" (7b)
array[2] data 7b UTF-8 bytes for "Charlie"

Complete wire format:

[                                     Complete Message                                 ]
[ (    length     ) (options) (    type     ) [                  data                ] ]
[ (      26b      ) ( NONE  ) ( FRIEND_LIST ) [ (3) { "Alice",   "Bob",  "Charlie" } ] ]
[ ( 1b + 2b + 23b ) (  1b   ) (     2b      ) [ 2b +  (2b+5b) + (2b+3b) + (2b+7b)    ] ]

Hex representation:

00 1A 00 00 05 00 03 00 05 41 6C 69 63 65 00 03 42 6F 62 00 07 43 68 61 72 6C 69 65

Complex Object Example

Player health information

class PlayerInfo extends NetworkSerializable {
    @NetworkSerializableProperty(0)
    public int getId() { return id; }
    
    @NetworkSerializableProperty(1) 
    public String getName() { return name; }
    
    @NetworkSerializableProperty(2)
    public float getHealth() { return health; }
}

// Example data:
PlayerInfo p = new PlayerInfo(42, "John", 87.5f);

Payload breakdown:

Part Size Description
id 4b Integer: 42
name length 2b Length of "John" (4b)
name data 4b UTF-8 bytes of "John"
health 4b Float value 87.5

Complete wire format:

[                              Complete Message                         ]
[ (    length     ) (options) (    type     ) [          data         ] ]
[ (      17b      ) ( NONE  ) ( PLAYER_INFO ) [ { 42,  "John", 87.5 } ] ]
[ ( 1b + 2b + 14b ) (  1b   ) (     2b      ) [   4b + (2b+4b) + 4b   ] ]

Hex representation:

00 11 00 00 03 00 00 00 2A 00 04 4A 6F 68 6E 42 AF 00 00

Implementation Notes

  1. Property Order: Serialization follows the order specified by @NetworkSerializableProperty annotations
  2. Compression: Applied automatically when @NetworkMessageHandler(compressed = true) is set
  3. Type Safety: Message types are validated during registry initialization
  4. Error Handling: Invalid messages or unknown types throw exceptions during deserialization

Compression Guidelines

Based on the compression overhead, compression is most effective for:

  • 0-50 bytes: Compression increases size by ~20% - avoid compression
  • 50-80 bytes: Compression is roughly size-neutral - optional
  • 80+ bytes: Compression reduces size by 20-60% - recommended
  • 500+ bytes: Compression reduces size by 50-80% - highly recommended

The library automatically handles compression/decompression when the compressed flag is set in the message handler annotation.