Custom protocol using BSON - trevex/golem GitHub Wiki

The underlying protocol of golem is interchangeable, but there are there are certain requirements the custom protocol needs to fulfill:

• extracting the name of the event from the incoming data
• and afterwards unmarshalling the leftover data / interstage data into the type associated with the event
• when sending data it needs to be able to marshal it and pack the event name into it

The default JSON-based protocol does this in a very simple manner as described on this wiki page.

The requirements are visual through the interface the custom protocol needs to fulfill:

type Protocol interface {
	Unpack([]byte) (string, interface{}, error)
	Unmarshal(interface{}, interface{}) error
	MarshalAndPack(string, interface{}) ([]byte, error)
	GetReadMode() int
	GetWriteMode() int
}

• Unpack takes the incoming data and returns the extracted string, the interstage product and an error if unsuccessful.
• Unmarshal takes the interstage product and a pointer to a value of the desired type and returns an error if unsuccessful.
• MarshalAndPack takes the event name and value pointer to marshall and pack it for sending, it returns the outgoing byte array and an error, if unsuccessful.
• GetReadMode and GetWriteMode determine in what WebSocket-mode data is received or send (text-based or binary).

In this tutorial we will create a BSON-based custom protocol using mgo's bson library, so make sure you go get it.

As protocol-base we use this simple data structure, i.e.:

{ "e": "eventname", "d": { ... } }

To unpack the event name we need to partially unmarshal, so we need a type representing our data.

type RawIncomingBSONMessage struct {
	Event string   `bson:"e"`
	Data  bson.Raw `bson:"d,omitempty"`
}

The bson.Raw type allows us to omit this part and unmarshal it later when we know the definitive type of the event. Our custom protocol needs a type we can add the required methods to:

type BSONProtocol struct{}

The first method is unpacking, where we need to extract the event name using our RawIncomingBSONMessage structure:

func (_ *BSONProtocol) Unpack(data []byte) (string, interface{}, error) {
	rawMsg := &RawIncomingBSONMessage{}
	err := bson.Unmarshal(data, rawMsg)
	if err != nil {
		return "", nil, err
	}
	return rawMsg.Event, &rawMsg.Data, nil
}

If we are not able to unmarshal we simply return the error. If successful we return the event name and a pointer to our bson.Raw interstage product. This interstage product will not be used by golem, but merely forwarded to Unmarshal if an event handler was found and the type was retrieved. This means we can typecast the interstage parameter to the type we returned from Unpack. (NOTE: the interface parameters for the interstage product allow flexibility for custom format, e.g. the default JSON-based protocol uses []byte as interstage product.

func (_ *BSONProtocol) Unmarshal(data interface{}, typePtr interface{}) error {
	return data.(*bson.Raw).Unmarshal(typePtr)
}

For outgoing messages we don't need an in-between step to determine the unmarshal-type, so only a single function is necessary. To represent the outgoing message we therefore need a simple structure again:

type OutgoingBSONMessage struct {
	Event string      `bson:"e"`
	Data  interface{} `bson:"d,omitempty"`
}

This structure can be used to marshal and pack the data:

func (_ *BSONProtocol) MarshalAndPack(name string, structPtr interface{}) ([]byte, error) {
	outMsg := &OutgoingBSONMessage{
		Event: name,
		Data:  structPtr,
	}
	return bson.Marshal(outMsg)
}

The last requirement of the interface is simply to define the read and write modes of the WebSockets using this protocol. Since BSON is binary encoded the sockets should use the binary operation code for sending and receiving:

func (_ *BSONProtocol) GetReadMode() int {
	return golem.BinaryMode
}
func (_ *BSONProtocol) GetWriteMode() int {
	return golem.BinaryMode
}

If the protocol should be used it can be simply set to a router by exposing an instance having all the required methods:

protocol := &BSONProtocol{}
myrouter.SetProtocol(protocol)

Now the server-side successfully uses the new protocol, now the client-side needs to use it as well. To use BSON in JavaScript the library provided by MongoDB is used. The client-side API follows the same patterns as the server-side except being JavaScript idiomatic and therefore function start with a lower case:

    var conn = new golem.Connection("127.0.0.1:8080/ws", true);
    var BSON = bson().BSON,
    	BSONProtocol = {
        	unpack: function(data) {
            	var doc = BSON.deserialize(new Uint8Array(data));
            	return [doc.e, doc.d];
        	},
        	unmarshal: function(data) {
        		return data; // just return data (it is automatically unmarshalled during unpack step)
        	},
        	marshalAndPack: function(name, data) {
        		return BSON.serialize({ e: name, d: data }, false, true, false).buffer;
        	}
    };
    conn.enableBinary();    // Too use BSON efficiently binaryType arraybuffer should be used.
    conn.setProtocol(BSONProtocol);

(Note: unpack returns an array since multiple return values are not supported by JavaScript)

The main difference of the API is the need to call enableBinary to use arraybuffers for incoming messages. The presented approach to implement a custom protocol could now be used to extend the getting started example. The full source of the working client and server of the presented protocol can be found in the example repository.