Getting started with WFC

WFC is a compiler that generates C++ code for given protocol specifications. It has been inspired by Protocol Buffers, and is backward compatible to it, but provides additional features and is optimized for use on embedded systems.

A .wfc file specifies the format of messages for which classes are created that provides functions for setting and getting values of the individual fields of the message and for serializing and deserializlizing the messages into a binary data stream that can be transmitted over a network or stored on non-volatile storage.

Data types

For the specification of message fields WFC provides several built-in data types. Furthermore, messages can also be used as field data types, and options exist to modify the exact semantics of specific fields. The data types provided can be categorized as follows:

• integer types
• floating point types
• strings and byte arrays
• structured data types (messages)

Message Fields

Message consist of a set of fields, where every field has a basic data type and can be set to occur just once or multiple times. Further options provide means to limit the number of occurances, to set the default field value, and many other aspects. These aspects are described in the documentation of the relevant options.

Additionally, every field requires a message specific unique field identifier (positive integer value) for data serialization. This unique ID makes it possible to transmit data between different versions of the generated code, as long as the ID does not change and care is take to modify the type only in a compatible way.

Example message

message Letter {
	string sender = 1;
	string receiver = 2;
	unsigned postage = 3	[ unset = 0 ];
	bool express_delivery = 4;
	string content = 5;
	repeated bytes attachments = 6;
}

As can be seen in this example, a message field always has at least a data-type, a name, and a field identifier integer for data serialization. Additionally, there may be specified further options in square brackets, and a keyword for determining if the field appears always in a message (required), is optionally included (no keyword or optional), or may appear multiple times (repeated).

Generated code

The generated code will consist of a .cpp and a .h file for implementing the message specification, and optionally files for the core functionality if WFC is set to generate the relevant parts as a library.

Every message will generate a class that has functions for handling the individual fields. E.g. the above Letter message provides following functions for handling the field sender:

	// string sender, id 1
	bool has_sender() const;
	void clear_sender();
	const std::string &sender() const;
	void set_sender(const void *data, size_t s);
	void set_sender(const std::string & v);
	void set_sender(const char *);
	std::string *mutable_sender();

Additionally, functions are generated for handling the message as a whole for serializing and derserializing or generating a JSON representation or so. Per default the following functions are created for the message Letter above:

	bool operator != (const Letter &r) const;
	bool operator == (const Letter &r) const;
	
	//! function for resetting all members to their default values
	void clear();
	
	//! Calculate the required number of bytes for serializing this object.
	//! If the data of the object change, the number ob bytes needed for
	//! serialization, may change, too.
	size_t calcSize() const;
	
	//! Function for parsing memory with serialized data to append and update this object.
	//! @param b buffer of serialized data
	//! @param s number of bytes available in the buffer
	//! @return number of bytes parsed or negative value if an error occured
	ssize_t fromMemory(const void *b, ssize_t s);
	
	//! Serialize the object to memory.
	//! param b buffer the data should be written to
	//! param s number of bytes available in the buffer
	//! return number of bytes written
	ssize_t toMemory(uint8_t *, ssize_t) const;
	
	//! Serialize the object using a function for transmitting individual bytes.
	//! @param put function to put individual bytes for transmission on the wire
	void toWire(void (*put)(uint8_t)) const;
	
	//! Function for serializing the object to a string.
	void toString(std::string &put) const;
	
	//! Function for writing the JSON representation of this object to a stream.
	void toJSON(std::ostream &json, unsigned indLvl = 0) const;
	
	//! Function for writing an ASCII representation of this object to a stream.
	void toASCII(std::ostream &o, size_t indent = 0) const;
	
	//! Function for determining the maximum size that the object may need for
	//! its serialized representation
	static size_t getMaxSize();
	
	//! Function for setting a parameter by its ASCII name using an ASCII representation of value.
	//! @param param parameter name
	//! @param value ASCII representation of the value
	//! @return number of bytes parsed from value or negative value if an error occurs
	int setByName(const char *param, const char *value);

The defaults include the whole feature set of functions, but it can be reduced to the project relevant functions using options to omit unneeded functions.

Finally, at the end of the class specification, the member variables are specified including the valid bits for optional fields that have no in-band unset value.

	protected:
	// string sender, id 1
	std::string m_sender;
	// string receiver, id 2
	std::string m_receiver;
	// unsigned postage, id 3
	uint64_t m_postage;
	// bool express_delivery, id 4
	bool m_express_delivery;
	// string content, id 5
	std::string m_content;
	// bytes attachments, id 6
	std::vector<std::string> m_attachments;
	
	private:
	uint8_t p_validbits;

Sending a message

To send a message, there are different ways of serializing its data. The easiest way to get started serializing, is to use the toString function. Therefore, you simply pass a std::string as an argument to this function and the class data will be serialized as binary data in the string. After that you can send the binary data or store it in non-volatile storage, the way you need.

	// now the message 'letter' is filled with data
	Letter l;
	l.set_sender(from);
	l.set_receiver(to);
	l.set_content(text);
	l.set_postage(postage);

	// serialize to string
	string envelope;
	l.toString(envelope);

Another way, if you do not want to use std::string on your embedded system, you can query the number of bytes required for serializing the data and provide the memory directly to the serialization toMemory:

	size_t s = l.calcSize();
	uint8_t *buffer = (uint8_t *) malloc(s);
	ssize_t n = l.toMemory(buffer,s);

So, at the end of that example the binary data are stored in the string envelope or in the manually allocated buffer.

Receiving a message

To receive a message, you need to deserialize the binary data using the fromMemory function.

	std::string data;
	// fill data from network or disk
	Letter l;
	ssize_t p = l.fromMemory(data.data(),data.size());

This little example triggers the deserialization of data from the string data. The return value yields the number of bytes successfully parsed or a negative value as a return code that can be used to find out the reason of the deserialization error by searching for the negative value in the generated code.

A small, but more complete and fully compilable hello-world example is also included in the sources of WFC. This can be used for getting started with WFC.