Keeping System.Text.Json lean

The System.Text.Json serializer has an unexpected performance penalty when used with options

Björn Weström

10 minute read

TL; DR

Serialization with System.Text.Json has an unexpected performance penalty when used with options, such as setting the PropertyNamingPolicy to CamelCase. For small objects, serialization is ~200x slower! To avoid this issue, store the options object in a class member and pass that member to JsonSerializer.Serialize.

UPDATE 2020-09-15

I posted an issue about this on Github, and the dotnet maintainers were quick to respond. This behavior is understood and the recommendation is to use a static (or shared) options object to avoid it. The root cause is:

The serializer undergoes a warm-up phase during the first (de)serialization of every type in the object graph when a new options instance is passed to it. This warm-up includes creating a cache of metadata it needs to perform (de)serialization: funcs to property getters, setters, ctor arguments, specified attributes etc. This metadata caches is stored in the options instance. This process is not cheap, and it is recommended to cache options instances for reuse on subsequent calls to the serializer to avoid unnecessarily undergoing the warm-up repeatedly

There are related issues #40072 and #38982.

How it all began

I was working on upgrading several of our applications from .Net Core 2.1 to .Net Core 3.1. One significant change was the switch from Newtonsoft.Json to System.Text.Json for serialization. Since the System.Text.Json package has been positively received due to the improved performance, I decided to give it a go.

Making the transition wasn’t particularly difficult, since we didn’t use any advanced features of Newtonsoft.Json. The main snag was handling reference loops, which Newtonsoft.Json conveniently could sort out for you, but even that has been fixed with a few well placed [JsonIgnore] attributes.

To keep the transition as smooth as possible, I decided to configure System.Text.Json to use camelCase for property names. Many of the APIs has a angular based frontend as its main consumer, and it just didn’t make sense to me to start sending PascalCase JSON. (This is also the default configuration for the serializer built into .Net Core MVC)

I was just finishing up a Web API, and everything looked promising. But there was a particular operation that seemed unusually slow. The operation fetches a list of objects, in my tests I was retrieving 80 objects. Stepping through the code in the debugger, I found that the Automapper conversion from entity model to contract model took ~150 ms! The objects are simple enough, about 10 properties. However, they include a Hash property, which is calculated from a serialization of the entity object.

In a trial and error attempt to find the root cause of the delay, I removed the options parameter to the JsonSerializer.Serialize call. And sure enough, the same Automapper conversion now took only 3 ms! I found it hard to believe that using camelCase would make the serialization orders of magnitude slower - looking at available benchmarks online the performance impact should be hardly noticeable. This piqued my interest and I decided to do some benchmarks of my own!

Different methods of passing the options

Some more trial and error revealed that putting the JsonSerializationOptions in a static class member and passing that member to JsonSerialize.Serialize reached similar performance as without passing any options at all. Hence I put together this benchmark with different methods of passing in options, with and without camelCase, and comparing that to using no options.

Benchmark

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.17763.1397 (1809/October2018Update/Redstone5)
Intel Core i7-8700 CPU 3.20GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
.NET Core SDK=3.1.101
  [Host]     : .NET Core 3.1.5 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.27001), X64 RyuJIT
  DefaultJob : .NET Core 3.1.5 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.27001), X64 RyuJIT
Method Mean Error StdDev
Serialize 4.479 μs 0.0123 μs 0.0103 μs
Serialize_InlineOptions_Default 839.179 μs 7.5675 μs 7.0786 μs
Serialize_InlineOptions_CamelCase 858.304 μs 11.1388 μs 9.3014 μs
Serialize_LocalOptions_Default 842.628 μs 4.0955 μs 3.4199 μs
Serialize_LocalOptions_CamelCase 839.716 μs 6.8355 μs 6.3939 μs
Serialize_StaticMemberOptions_Default 4.464 μs 0.0144 μs 0.0120 μs
Serialize_StaticMemberOptions_CamelCase 4.428 μs 0.0588 μs 0.0699 μs
Serialize_MemberOptions_Default 4.499 μs 0.0311 μs 0.0259 μs
Serialize_MemberOptions_CamelCase 4.578 μs 0.0228 μs 0.0202 μs

The difference between creating the options on the fly (inline or locally) and providing them from a class member is daunting! It adds nearly 1 ms to the serialization time. Since creating the options on the fly means creating them for each call to JsonSerializer.Serialize it was expected that these should perform slightly worse. Could the instantiation of the options object explain this difference? Lets find out!

Benchmark code

Type of function call

The middle part of the name of each benchmark identifies the type of call to JsonSerializer.Serialize.

  • InlineOptions - Options are instantiated inline
JsonSerializer.Serialize(myObject, new JsonSerializerOptions());
  • LocalOptions - Options are instantiated as a local variable
var options = new JsonSerializerOptions();
JsonSerializer.Serialize(myObject, options);
  • StaticMemberOptions - Options are instantiated as a static class member
private static JsonSerializerOptions jsonStaticDefaultOptions = new JsonSerializerOptions();
...
JsonSerializer.Serialize(myObject, jsonStaticDefaultOptions);
  • MemberOption - Options are instantiated as a class member
private JsonSerializerOptions jsonDefaultOptions;
...
this.jsonDefaultOptions = new JsonSerializerOptions();
...
JsonSerializer.Serialize(jsonDefaultOptions);

Type of options

The suffix of the name of each benchmark identifies the type of options used.

  • Default - Default options
new JsonSerializerOptions();
  • CamelCase - Options set for camelCase
new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase };

Options construction

If creating the options when they are needed slows down the serialization, could it be that the options object requires some heavy lifting to be instantiated? Unlikely but worth a benchmark.

Benchmark

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.17763.1397 (1809/October2018Update/Redstone5)
Intel Core i7-8700 CPU 3.20GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
.NET Core SDK=3.1.101
  [Host]     : .NET Core 3.1.5 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.27001), X64 RyuJIT
  DefaultJob : .NET Core 3.1.5 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.27001), X64 RyuJIT
Method Mean Error StdDev
NoOp 0.0163 ns 0.0014 ns 0.0011 ns
CreateMyObject 4.7044 ns 0.0101 ns 0.0089 ns
CreateOptions 337.1717 ns 3.2470 ns 2.5351 ns
CreateOptions_Camel 341.6551 ns 3.1801 ns 2.8190 ns

The results show that the options object is indeed a rather large one (compare to the test object MyObject used for the serialization). But it still takes only a fraction of a μs to instantiate - this cannot explain the large performance gap in the previous benchmark.

Benchmark code

  • NoOp is simply an empty function, added for reference
  • CreateMyObject instantiates an object of the simple test class MyObject
  • CreateOptions / CreateOptions_Camel instantiates a JsonSerializerOptions object in a similar way as in the previous benchmark.

Serialization of a list

In the previous benchmarks a really small object was serialized. How does this performance gap scale if we serialize something larger, like a list of the small objects?

Benchmark

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.17763.1397 (1809/October2018Update/Redstone5)
Intel Core i7-8700 CPU 3.20GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
.NET Core SDK=3.1.101
  [Host]     : .NET Core 3.1.5 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.27001), X64 RyuJIT
  DefaultJob : .NET Core 3.1.5 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.27001), X64 RyuJIT
Method N Mean Error StdDev
Serialize 10 45.92 μs 0.913 μs 1.250 μs
Serialize_InlineOptions_Default 10 944.99 μs 9.046 μs 8.019 μs
Serialize_InlineOptions_CamelCase 10 938.90 μs 17.853 μs 19.844 μs
Serialize_StaticOptions_Default 10 43.30 μs 0.106 μs 0.089 μs
Serialize_StaticOptions_CamelCase 10 44.24 μs 0.040 μs 0.037 μs
Serialize 100 528.36 μs 1.481 μs 1.313 μs
Serialize_InlineOptions_Default 100 1,429.01 μs 10.067 μs 8.924 μs
Serialize_InlineOptions_CamelCase 100 1,434.56 μs 4.027 μs 3.570 μs
Serialize_StaticOptions_Default 100 510.18 μs 2.280 μs 2.133 μs
Serialize_StaticOptions_CamelCase 100 517.13 μs 2.558 μs 2.268 μs
Serialize 1000 4,852.29 μs 25.266 μs 23.634 μs
Serialize_InlineOptions_Default 1000 5,727.67 μs 81.384 μs 72.145 μs
Serialize_InlineOptions_CamelCase 1000 5,713.51 μs 84.481 μs 70.545 μs
Serialize_StaticOptions_Default 1000 4,829.04 μs 25.773 μs 24.108 μs
Serialize_StaticOptions_CamelCase 1000 4,939.82 μs 22.851 μs 21.374 μs

Where N is the number of objects in the list. Ok, so it seems that the on the fly options add about 900 μs to the serialization regardless of object size. That’s good news at least.

Serialization in a loop

How about serializing those objects one by one?

Benchmark

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.17763.1397 (1809/October2018Update/Redstone5)
Intel Core i7-8700 CPU 3.20GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
.NET Core SDK=3.1.101
  [Host]     : .NET Core 3.1.5 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.27001), X64 RyuJIT
  DefaultJob : .NET Core 3.1.5 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.27001), X64 RyuJIT
Method N Mean Error StdDev
Serialize 10 44.59 μs 0.038 μs 0.033 μs
Serialize_InlineOptions_Default 10 8,364.56 μs 80.834 μs 75.612 μs
Serialize_InlineOptions_CamelCase 10 8,438.08 μs 57.459 μs 53.747 μs
Serialize_StaticOptions_Default 10 45.02 μs 0.142 μs 0.133 μs
Serialize_StaticOptions_CamelCase 10 44.94 μs 0.126 μs 0.112 μs
Serialize 100 463.57 μs 1.972 μs 1.748 μs
Serialize_InlineOptions_Default 100 84,244.24 μs 594.933 μs 556.501 μs
Serialize_InlineOptions_CamelCase 100 88,661.04 μs 1,743.013 μs 1,711.872 μs
Serialize_StaticOptions_Default 100 503.88 μs 9.914 μs 14.532 μs
Serialize_StaticOptions_CamelCase 100 504.59 μs 10.077 μs 20.810 μs
Serialize 1000 5,070.30 μs 101.357 μs 224.600 μs
Serialize_InlineOptions_Default 1000 898,815.78 μs 16,983.724 μs 17,441.035 μs
Serialize_InlineOptions_CamelCase 1000 900,245.68 μs 17,592.917 μs 25,231.237 μs
Serialize_StaticOptions_Default 1000 4,902.03 μs 74.758 μs 58.366 μs
Serialize_StaticOptions_CamelCase 1000 5,213.93 μs 103.911 μs 282.699 μs

Where N is the number of objects to serialize. All objects are created before the benchmark, then they are serialized one by one (one call to JsonSerializer.Serialize per object). This is similar to the results from serialization of the list - on the fly instantiation of options adds 800-900 μs per call.

Alternate overloads

Finally I decided to check if there are other overloads of JsonSerializer.Serialize that would perform better with on the fly options.

Benchmark

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.17763.1397 (1809/October2018Update/Redstone5)
Intel Core i7-8700 CPU 3.20GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
.NET Core SDK=3.1.101
  [Host]     : .NET Core 3.1.5 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.27001), X64 RyuJIT
  DefaultJob : .NET Core 3.1.5 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.27001), X64 RyuJIT
Method Mean Error StdDev
Serialize 4.470 μs 0.0237 μs 0.0185 μs
Serialize_InlineOptions_Default 848.252 μs 5.4895 μs 5.1349 μs
Serialize_AltInlineOptions_Default 836.154 μs 3.9665 μs 3.5162 μs
Serialize_Alt2InlineOptions_Default 846.886 μs 8.8031 μs 8.2345 μs

The results are clear, all overloads for passing options to JsonSerializer.Serialize shares the same weakness.

Benchmark code

  • InlineOptions - This is the method used in the previous benchmarks

  • AltInlineOptions - Overload that accepts a type argument

JsonSerializer.Serialize(myObject, typeof(MyObject), new JsonSerializerOptions());
  • Alt2InlineOptions - Generic overload that accepts a type argument
JsonSerializer.Serialize<MyObject>(myObject, new JsonSerializerOptions());

Discussion

Some will claim that 900 μs of additional delay for serialization isn’t a big deal. While this is correct in some scenarios, I think that anyone striving to keep their request pipeline lean won’t agree. In our applications we are approaching 10 ms response time for the less complex requests (not a record for sure, but this includes a lot of enterprise stuff like logging, audit logging, AD authorization). Adding 900 μs makes or apps 9% slower!

Secondly, I would presume that there are many others who perform multiple serializations within a single request. In our case, we do this for each item in a list to calculate a hash value for each item. When returning 100 items, you are suddenly accumulating ~100 ms of delay, which is pretty bad - in particular when it can be easily avoided.

What is worse is that the official documentation is full of examples that instantiate the options on the fly! Example of how to use camelCase. I agree fully that it makes the example code much more compact when presented this way, but there are no notes about the performance impact. A senior developer would probably move the options to a class member if that member is used multiple times within a class, but when the options object is used only once it makes the code more clean to simply do it inline.

The burning question is why does this performance gap exist? If instantiating objects inline and passing them to functions is really this terrible, we should avoid that everywhere. But that cannot be true? My guess is that the options object is used quite heavily when initiating the serialization. We saw from the object creation benchmark that the options object is quite large, hence there are plenty of options that must be processed. If accessing properties of a class member is slightly faster than accessing properties of a locally scoped object, then this difference would accumulate.

Still I think it is a mystery that the difference is so huge. How can serializing a small object be 200x slower due to how options are passed?

Source code

The full source used in the benchmarks can be found here.

Tools