Closed Early Access Cost - Monthly

In the closed early access stage, we assume a maximum user count of 100, with each storing a maximum of 20 models and deploying 5 models at a time. The maximum model size would be ~500 MB. This means that the total disk/object storage required which is a maximum of 100 * 20 * 0.5 = 1000 GB (1TB). The maximum memory (RAM) required for a single replica of this deployments is 100 * 5 * 0.5 = 250 GB.

CPU Cost

The most efficient on-demand compute server option is the e2-highmem-4 offered by Google. To reach a single replication of the maximum capacity, we would need 250/32 ~= 8 of these nodes.

8 x e2-highmem-4: 8 * 130 = $1040

Object Storage Cost

Assuming the models are converted to binary format (gzip?) of a similar file size and each stored in object storage, we would need (10020.5) 1000 GB, and about 10020100 (assuming ~100 operations per model on average) write operations. Google's objects storage is the cheapest here.

1,000 GB of object storage: $20

200,000 write ops: $1

Disk Storage

Since this is a clustered environment, it is more efficient to have an NFS than attached block storage on each node. (While block storage is cheaper than NFS storage, the cost is multiplied by however many nodes we provision. If there is some redundancy in the data we store, or if the storage space is not fully utilized at all times by all of the nodes, then having shared storage can be more economic.)

This estimation is the most difficult, since we don't quite know the scope of the application and can't predict how much disk space will be necessary to shuffle around the models or cache them locally. If we assume that we will have all of the "deployed" models cached on disk rather than in object storage, then we would need 250 GB of NFS. To provide a breathing room, we provision an extra 50 GB to be safe.

GCP and AWS offer the same price for networked file system, so we go with GCP to go with our previous 2 selections.

300 GB NFS: $90