Dark as of today (4/18/2020) does not support changing Datastore schemas. Therefore, you need to do a migration if you want to make schema changes.
Migrations in Dark are similar to migrations in traditional backend systems. The following steps outline a double-write strategy, which allows you to move data from Datastore A → B without taking down the service or losing data. If these are not concerns, you can use a REPL to move data from A → B and incrementally update callsites. However, you may want to use the following strategy regardless to have a less stressful migration 😬.
We'll use the simple example of adding an email field to a
Create the new Datastore. We'll call it
Users2. Our goal is to end up in a place where we are running on
Users can be deleted. Note that Datastores cannot be renamed after they hold data (the next step).
Update all your
DB::set Users callsites to also write to
Note that if your newer schema contains data not in the older schema (in this example the
If you have a lot of
DB::set callsites, you may consider extracting logic into a function like
updateUser which performs both
Once all your
DB::set callsites are writing to both Datastores, we're ready to do the big migration. Create a REPL that iterates through
Users and writes them to
At this point, both your Datastores should be in a usable state. Now's a good time to use a REPL to double check that both Datastores contain what you expect.
Update your readers to read from the new Datastore. This is a good place to use feature flags!
Confirm that your APIs are still working as expected. In this example, we'd expect the
You may want to stay in this state for a little bit to build confidence. Once we continue, we can't easily undo the previous steps because it will be missing data.
We want to get a state where the old Datastore has no more dependents.
Remove the writes to the old Datastore.
Once your old Datastore has no more dependents, you can clear it with a REPL.
Last but not least, delete your old Datastore!
Some migrations may be trickier, such as ones that migrate multiple Datastores at the same time. However, the same mental model applies of getting to a state where you have two usable up-to-date Datastores and then flipping the switch to read from the new Datastore.