https://tristangermain.com/tags/binary-encoding/Recent content in Binary Encoding on Tristan GermainHugo -- gohugo.ioen-us© Tristan GermainSat, 21 Dec 2024 18:04:25 +0200https://tristangermain.com/posts/binaries-encoding/4-conclusion/Sat, 21 Dec 2024 18:04:25 +0200https://tristangermain.com/posts/binaries-encoding/4-conclusion/Avro Avro is a really performant technology with a great feature set. The payload size performance is top notch and its feature set is the best for enforcing data consistency between services: With the ability to serialize properly complex types By forcing each service to deserialize the payload with the schema version it was serialized with By helping with schema migration But it also has major cons: The learning curve is steep because of its ecosystem and its intrinsic complexity The setup, maintenance and coordination cost between services’ teams is important when using single object encoding Avro suits really well in environments where either consistency between services is a major constraint, event message size is critical, or events have such a size that the extra bytes of embedding the schema within the payload is negligible.https://tristangermain.com/posts/binaries-encoding/3-protobuf/Thu, 07 Nov 2024 18:04:25 +0200https://tristangermain.com/posts/binaries-encoding/3-protobuf/Third post of the binary encoding technologies series. It presents how Protobuf can be used to encode messages stored in an event bus. As a reminder each technology is presented according to the following plan: The key differentiation factors The available tooling by showing the implementation of a hypothetical blog post comment creation event: { "id": "de2df598-9948-4988-b00a-a41c0e287398", "message": "I’m sorry, Dave. I’m afraid I can’t do that.", "locale": "en_US", "author": { "id": "78fc52a3-9f94-43c6-8eb5-9591e80b87e1", "nickname": "HAL 9000", }, "blog_id": "b4e05776-fca3-485e-be48-b1758cedd792", "blog_title": "Binary encoding" } The ecosystem: quality of documentation and tooling, available resources etc.https://tristangermain.com/posts/binaries-encoding/2-json/Tue, 01 Oct 2024 18:04:25 +0200https://tristangermain.com/posts/binaries-encoding/2-json/Second post of the binary encoding technologies series. It presents how JSON, MessagePack and JSON schema can be used together to encode messages stored in an event bus. As a reminder each technology is presented according to the following plan: The key differentiation factors The available tooling by showing the implementation of a hypothetical blog post comment creation event: { "id": "de2df598-9948-4988-b00a-a41c0e287398", "message": "I’m sorry, Dave. I’m afraid I can’t do that.https://tristangermain.com/posts/binaries-encoding/1-avro/Sat, 07 Sep 2024 18:04:25 +0200https://tristangermain.com/posts/binaries-encoding/1-avro/This is the first post in a series comparing different binary encoding technologies for messages stored in an event bus. After some research the technologies usable in a polyglot production environment are: Avro JSON with Message Pack and JSON schema. Protobuf The first three posts present the following for each technology: The key differentiation factors The available tooling by showing the implementation of a hypothetical blog post comment creation event: { "id": "de2df598-9948-4988-b00a-a41c0e287398", "message": "I’m sorry, Dave.