Problem

We currently rely on an antiquated and out of date in both the feature set and time version of AssemblyScript to compile our cross-platform bucketing shared codebase.

This has a few problems that are causing us friction:

• AssemblyScript does not follow the WASM spec; and has intentionally avoided it in some cases to our detriment. Hence sticking at a legacy version; which we forked.
• We don’t have any new features from WASM to take advantage of in our SDKs or pipeline
  • Threading
  • Memory Performance
  • Exception handling
  • Security/Dependancy updates
• AssemblyScript tries to be JS; but isn’t - causing weird problems with serialization
• Limited support for non-JSON data

This has caused us to not implement features that could have been possible (multi-memory access for config/etc)

Proposal

The proposal here is to re-write our core bucketing engine in Rust. This has a few core benefits;

• Performance
  • We can make use of the most recent WASM codebase and spec for features.
  • We can make use of non-JSON spec transit types
• Security
  • Using up-to-date packages and features
• Ease of development
  • Documentation; options; language native performance and testing tooling
  • Support for other systems and test features beyond basic unit testing
  • Commonly used language and codebase
• Portability
  • We can generate a C based dylib - allowing us to import this library into C based/supported languages; or WASM - depending on the performance requirements for that language.
  • WASM support is a core tenant of the Rust language now; as a native build output
  • Write once; deploy everywhere

There are definitely a few drawbacks here:

• Lack of familiarity
  • I (Jamie) have been working on learning rust and rewriting the bucketing logic in Rust slowly myself as a learning project - and I’ve been able to make good progress; but the nature of the codebase and how the Rust language works is very unforgiving as a developer. Even for me as a more systems level/low level language dev it’s not easy.
  • We are mostly JS - this would mean that engineers need to learn another language from scratch to properly implement everything safely