Benchmarking Protocol Buffers and JSON - Information for Architects

You may be wondering what the performance differences between JSON and Protocol Buffers (Protobuf) really are.  You are likely using JSON and have heard about Protobuf and wondering if it is worth using.  We’ll discuss more about the differences and considerations when deciding whether or not to use protobuf but since most likely you want to get straight to the benchmarks I’ll put them at the start of the article.  

Note: when referring to serialisation it is always serialisation and deserialisation.

The Benchmarks 

• Java and PHP are the languages used as they represent fast compiled languages and fast dynamic languages - Internal PHP functions are C functions and very fast.
• A Mock “Trip” payload of varying sizes has been used to see the effect of size on serialisation.  Each "Trip" comprises a set of X "transport" nodes and each "transport" has a set of X "package" nodes where X is the count of nodes.  For each size test X was set as:

• The proto file (schema of the protobuf) is equivalent to the DTOs used in PHP except for DateTimes.  For Protobuf it is a Timestamp which does not have as developer friendly methods available when de-serialised. 
• PHP has 4 variants test variants:

• Java has 2 variants.  Shaped and Protobuf. Shaped uses JsonB because of its modern API. Jackson would have been about 50% faster.
• Each test is run against 100 payloads - each is unique. The times are shown as the sum and the average.

The results can be found at: https://docs.google.com/spreadsheets/d/18P15ntbiUSJ8Q8cDVtPC9t6Jn03J2uivGkC3E-FuDRY/edit?usp=sharing

Source code at: https://github.com/andrew-belac/blog-jsonprotobuf

Results

Please do use the google document link to view the benchmarks in a more friendly format.

Small (5.5 Kb)

Medium (36 Kb)

Medium Large (76.1Kb)

Large (1.3 Mb)

Considering JSON and Protobuf

JSON is the standard for data serialization for web applications and services.  It took over from XML which was harder to work with in web browsers, slower, larger and less friendly for developers and technical users.  But probably more importantly because it is Javascript’s native serialisation format and the web became central to most application development and always had a client server architecture.

The rise of “hyperscale” businesses, microservices saw the increased use of binary formats like thrift and Protobuf.  These formats traded human friendly human readable text formats for binary formats offering more performance and smaller size. 

As such you may have heard that JSON is slow and that you should use Protobuf - Google created it - and that 10% of your compute is used for serialisation if you are using JSON. 

But is faster the only relevant property?  And is it materially faster?  Let us consider the context.

Context

When considering whether or not to use JSON or Protobuf consider the context:

• Who will be using the API?  Is it public or internal?  Is it for partners and customers in B2B scenarios.  Is it for service to service operations?  
• What does your internal architecture look like?  Is it a monolith of any type?  Is it micro service based and if so how much do they need to consume each others’ APIs?  

A note on Monolith architectures and Microservice architectures.

Monoliths/Moduliths vs Microservices

The diagram illustrates the difference between monolith service to service calls vs microservice based ones.  Monoliths can participate in transactions and are in process / shared memory and therefore DO NOT require serialisation for service calls.  Microservices require an HTTP connection and the serialisation is required for each service request.  The chattier the microservices the higher the cost.

Closing

A typical API request within the context of a Monolithic application will typically only require a single deserialization for the request and a single serialisation for the response.  Given the microsecond and low millisecond time it takes to serialise and deserialise for any normal request/response body the performance improvements Protobuf are NOT material.

If your API requires many service to service calls in a micro service architecture then at some point those internal services would benefit i.e. there would be a material improvement in response times, and the efficiency of the infrastructure you have by moving to Protobuf or another binary format.

Public APIs or APIs consumed by partners, customers, etc. should likely stay as JSON APIs as it may be challenging for each of these partners to accommodate the use of Protobuf as opposed to the universal nature and human friendly properties of JSON.