The main culprit of slow deserialization is the serialization library used by Bolt called FullSerializer (FS). Back in 2016, it was the most powerful open-source library available and the only one capable of handling Bolt’s complex nested graph structure and type freedom.

Unfortunately, FullSerializer is also one of the slowest serialization libraries available. Its creator explicitly said he favored accuracy over speed, which was important in the early days of Bolt, but is now limiting its users with large projects.

Another part of the deserialization problem is that Unity will deserialize every asset (including every Macro) in the project in the first frame, regardless of whether it is used in the current scene. This means the deserialization slowdown is directly proportional to the amount of macros a given project contains.

Thankfully, the geniuses at Sirenix have created a modern, powerful and most importantly super-fast serialization library called Odin Serializer (OS), used to power their best-selling Unity asset Odin Inspector. Recently, Odin Serializer was open sourced, and we’ve been in contact with the developers at Sirenix ever since to see how we could migrate from FS to OS.

This alone could, according to their benchmarks, speed up deserialization by a factor of 3.

The proposed changes are:

  1. Fork OS to a new repository or submodule
  2. Move all types to a Ludiq.Dependencies.Odin namespace to avoid conflicts
  3. Include OS in the build process to generate a DLL
  4. Add this DLL to the Ludiq.Core references
  5. Implement an OS serialization policy that matches the current FS policy
  6. Extend the SerializationData class to include OS data alongside FS data
  7. Rework the Serialization utility class to:
    1. Serialize to both OS and FS*
    2. Deserialize with OS data when available, fallback to FS data

*7a is a trade-off we have to consider. On the one hand, serializing to FS is necessary to ensure backwards compatibility from graphs created in Bolt 2 to be imported in Bolt 1. However, this is not yet considered a core constraint of the upgrade. On the other hand, the advantage of not serializing to FS is performance: according to the same benchmarks, it would be 6x slower to serialize to both OS and FS than to just serialize to OS. But considering serialization is not the current major issue (deserialization is), it may not be too bad of a sacrifice.

Optimizing the serialization library will have far ranging positive side effects, because a lot of Ludiq and Bolt subsystems depend on it. For example, Project Settings, Editor Windows and Unit Options all currently use FS and would load faster by switching to OS. If we decide to not implement dual-serialization, unit database generation (dubbed Extraction in Bolt 2) will also be greatly sped up, as a large part of the time it takes comes from serializing the units.