Implementing Custom Serializer - liuli-neko/NekoProtoTools GitHub Wiki

The library provides a flexible and extensible framework for serialization and deserialization. To implement a custom serializer, you need to define the following components:

Input Serializer: Responsible for deserializing data from a specific format into C++ objects.
Output Serializer: Responsible for serializing C++ objects into a specific format.
Format-Specific Traits: Define how the serializer interacts with the data format (e.g., formatting options, buffer types, etc.).
Prologue and Epilogue Functions: Handle pre- and post-processing for serialization and deserialization.

The library uses templates and traits to ensure type safety and extensibility.

Steps to Implement a Custom Serializer

1. Define Format-Specific Traits

Create traits to define how the serializer interacts with the data format. These traits specify buffer types, writer types, and other format-specific details.

if you want to support multiple output buffer type, you can define a trait like this:

template <typename T, class Enable = void>
struct custom_output_buffer_type {
    using output_buffer_type = void;
};

template <>
struct custom_output_buffer_type<std::vector<char>, void> {
    using output_buffer_type = std::vector<char>;
    using wrapper_type       = CustomBufferWrapper; // Replace with your buffer wrapper
    using writer_type        = CustomWriter<CustomBufferWrapper>; // Replace with your writer
    using char_type          = char;
};

Similarly, define traits for input buffers:

template <typename T, class Enable = void>
struct custom_input_buffer_type : std::false_type {
    using input_buffer_type = void;
};

template <typename T>
struct custom_input_buffer_type<T, typename std::enable_if<std::is_base_of<std::istream, T>::value>::type>
    : std::true_type {
    using input_buffer_type = T;
};

2. Implement the Output Serializer

The output serializer is responsible for converting C++ objects into the desired format. It should inherit from the OutputSerializer base class provided by the library.

class CustomOutputSerializer : public detail::OutputSerializer<CustomOutputSerializer> {
public:
    explicit CustomOutputSerializer(std::vector<char>& buffer);
    bool saveValue(const uint8_t value) {} // Implement serialization logic for uint8_t
    bool saveValue(const int8_t value) {} // Implement serialization logic for uint8_t
    bool saveValue(const uint16_t value) {} // Implement serialization logic for uint16_t
    bool saveValue(const int16_t value) {} // Implement serialization logic for int16_t
    bool saveValue(const uint32_t value) {} // Implement serialization logic for uint32_t
    bool saveValue(const int32_t value) {} // Implement serialization logic for int32_t
    bool saveValue(const uint64_t value) {} // Implement serialization logic for uint64_t
    bool saveValue(const int64_t value) {} // Implement serialization logic for int64_t
    bool saveValue(const float value) {} // Implement serialization logic for float
    bool saveValue(const double value) {} // Implement serialization logic for double
    bool saveValue(const bool value) {} // Implement serialization logic for bool
    bool saveValue(const std::string& value) {} // Implement serialization logic for string
    bool saveValue(const char* value) {} // Implement serialization logic for char*
    bool saveValue(const std::string_view value) {} // Implement serialization logic for string_view, if c++17 or later
    template <typename T>
    bool saveValue(const SizeTag<T>& size) {} // Implement serialization logic for SizeTag, The container type will use this class to mark and get the size.
    template <typename T>
    bool saveValue(const NameValuePair<T>& nameValue) {} // Implement serialization logic for NameValuePair, a value with a name, like a key-value pair.

    bool startArray(const std::size_t) {} // Implement serialization logic for starting an array
    bool endArray() {} // Implement serialization logic for ending an array
    bool startObject(const std::size_t = -1) {} // Implement serialization logic for starting an object
    bool endObject() {} // Implement serialization logic for ending an object
    bool end() {} // Implement serialization logic for flush the stream
};

3. Implement the Input Serializer

The input serializer is responsible for parsing data from the format and populating C++ objects. It should inherit from the InputSerializer base class.

class CustomInputSerializer : public detail::InputSerializer<CustomInputSerializer> {
public:
    explicit CustomInputSerializer(const char * data, std::size_t size) {}
    bool loadValue(uint8_t& value) {} // Implement deserialization logic for uint8_t
    bool loadValue(int8_t& value) {} // Implement deserialization logic for int8_t
    bool loadValue(uint16_t& value) {} // Implement deserialization logic for uint16_t
    bool loadValue(int16_t& value) {} // Implement deserialization logic for int16_t
    bool loadValue(uint32_t& value) {} // Implement deserialization logic for uint32_t
    bool loadValue(int32_t& value) {} // Implement deserialization logic for int32_t
    bool loadValue(uint64_t& value) {} // Implement deserialization logic for uint64_t
    bool loadValue(int64_t& value) {} // Implement deserialization logic for int64_t
    bool loadValue(float& value) {} // Implement deserialization logic for float
    bool loadValue(double& value) {} // Implement deserialization logic for double
    bool loadValue(bool& value) {} // Implement deserialization logic for bool
    bool loadValue(std::string& value) {} // Implement deserialization logic for string

    template <typename T>
    bool loadValue(const NameValuePair<T>& value) {} // Implement deserialization logic for NameValuePair, a value with a name, like a key-value pair.
    template <typename T>
    bool loadValue(const SizeTag<T>& value) {} // Implement deserialization logic for SizeTag, The container type will use this class to mark and get the size.
    
    bool startNode() {} // Implement deserialization logic for starting a node
    bool finishNode() {} // Implement deserialization logic for ending a node

    // Optional: only if you want to use it in jsonrpc module
    void rollbackItem() {} // Implement deserialization logic for rollback an item
    bool isArray() {}
    bool isObject() {}
};

4. Define Prologue and Epilogue Functions

Prologue and epilogue functions handle pre- and post-processing for serialization and deserialization. These functions are optional but can be used to customize behavior for specific types.

template <typename T, typename BufferT>
inline bool prologue(CustomInputSerializer<BufferT>& serializer, const T& value) {
    return serializer.startNode(); // Replace with your format's start node logic
}

template <typename T, typename BufferT>
inline bool epilogue(CustomInputSerializer<BufferT>& serializer, const T& value) {
    return serializer.finishNode(); // Replace with your format's finish node logic
}

5. Register the Serializer

Finally, define a type alias for your custom serializer to integrate it into the library.

struct CustomSerializer {
    using OutputSerializer = CustomOutputSerializer;
    using InputSerializer  = CustomInputSerializer;
};

Example Usage

Once your custom serializer is implemented, you can use it as follows:

std::vector<char> buffer;
CustomSerializer::OutputSerializer serializer(buffer);

int value = 42;
serializer.saveValue(value);

std::string str = "Hello, World!";
serializer.saveValue(str);

For deserialization:

std::istringstream inputStream("{\"key\": 42}");
CustomSerializer::InputSerializer deserializer(inputStream);

int value;
deserializer.loadValue(value);

std::string str;
deserializer.loadValue(str);

Notes

Ensure that your custom serializer adheres to the same interface as the provided JSON serializer.
Use traits and templates to handle type-specific logic.
Follow the library's conventions for error handling and logging.

By following these steps, you can implement a custom serializer for any data format while maintaining consistency with the library's design.