Protocol Puffers

A little, highly templated, and protobuf-compatible serialization/deserialization library written in C++20

📕 Documentation

Requirements

• a compiler and a standard library implementation with C++20 support
  • GCC 11 or above, or
  • Clang 12 or above, or
  • MSVC 14.29 (Visual Studio 2019 Version 16.9) or above
  • Apple Clang 17 (Xcode 16.4) or above
• CMake 3
• GoogleTest (optional, for unit tests)
• vcpkg (optional, vcpkg install protopuf to install)

Features

• Data structures are described using type in C++ instead of DSLs like the Protocol Buffer Language (.proto)
• Fully compatible with encoding of the Protocol Buffers, capable of mutual serialization/deserialization
• Extensive compile-time operations aimed to improving run-time performance

An Example

For the following data structure described using .proto:

message Student {
    uint32 id = 1;
    string name = 3;
}

message Class {
    string name = 8;
    repeated Student students = 3;
}

We can use protopuf to describe it as C++ types:

using namespace pp;

using Student = message<
    uint32_field<"id", 1>,
    string_field<"name", 3>
>;

using Class = message<
    string_field<"name", 8>,
    message_field<"students", 3, Student, repeated>
>;

Subsequently, both serialization and deserialization become so easy to do:

// serialization
Student twice {123, "twice"}, tom{456, "tom"}, jerry{123456, "jerry"};
Class myClass {"class 101", {tom, jerry}};
myClass["students"_f].push_back(twice);

array<byte, 64> buffer{};
auto result = message_coder<Class>::encode(myClass, buffer);
assert(result.has_value());
const auto& bufferEnd = *result;
assert(begin_diff(bufferEnd, buffer) == 45);

// deserialization
auto result2 = message_coder<Class>::decode(buffer);
assert(result2.has_value());
const auto& [yourClass, bufferEnd2] = *result2;
assert(yourClass["name"_f] == "class 101");
assert(yourClass["students"_f][2]["name"_f] == "twice");
assert(yourClass["students"_f][2]["id"_f] == 123);
assert(yourClass["students"_f][1] == (Student{123456, "jerry"}));
assert(yourClass == myClass);
assert(begin_diff(bufferEnd2, bufferEnd) == 0);

More examples can be found in our test cases (test/message.cpp).

Supported Field Types

Category	Supported Types
Varint	int32, int64, uint32, uint64, sint32, sint64, bool, enum
64-bit	fixed64, sfixed64, double
Length-delimited	string, bytes, embedded messages, packed repeated fields
32-bit	fixed32, sfixed32, float

Known issues

• There is a known bug in Visual Studio 2019 Version 16.8 related to template parameter lists of lambda expressions, which can produce a wrong compilation error while compiling protopuf
• Although class type in NTTP (P0732R2) is implemented in GCC 10, there is a CTAD bug (PR96331, exists until GCC 10.2) to reject valid NTTP usage, which prevent protopuf to compile successfully

Acknowledgement

• to @dngrudin for the contribution of out-of-bounds checking and unsafe/safe mode
• to JetBrains for its Open Source License of All Products Pack