Indexers operate Graph Nodes, indexing on-chain data and processing queries upon Dapps' request (Consumers). The indexing work is a heavy task and for highly used subgraphs, query time and performance are paramount for Indexers. These network participants must constantly find the right balance between infrastructure costs and performance, having in mind potential rewards to be collected from query fees. In order to achieve a stable network with almost zero downtime and an efficient market, Indexers must properly plan ahead for operational costs.
As such, The Graph Foundation is looking for an individual/team to design and build an advanced Cloud Indexing Cost Calculator capable of predicting the costs involved with indexing a particular set of subgraph(s) and respective networks, and how profitable it may be based on properly sized cloud infrastructure. The tool should provide a breakdown of involved costs, and an estimate on ROI (Return on Investment) or break-even point when estimating indexing rewards and query fees as income.
- Must expose an API, as it can, in the future, be integrated with existing or new UIs.
- Must work with different subgraphs from different supported chains.
- Must take into account compute and store needs for different RPC nodes or Firehose-based endpoints for specific chains.
- Must properly estimate load and resource consumption at indexing and querying time.
- Input sources to be taken into account:
qlog (https://github.com/graphprotocol/qlog/), network-available data or any other data altruistically shared by Indexers based on experience.
- Optionally, a dedicated database could be built, collecting metrics from different Indexers. Related, separate services could be built capable of replicating traffic against a known environment to collect information on resources consumption for a particular query volume and data set.
- Must consume other Cloud Providers’ API to derive pricing information for a particular environment (CPU Cores, RAM, Disk, redundancy, etc.).
Note: The scope of the RFP can be extended to include other value add tools to compute a subgraph’s complexity and it’s on load for a given query volume. These tools can be static analyzers or others capable of determining code complexity by inspecting a subgraph mapping load on CPU and IO. The Graph Foundation encourages applicants to work on such separate tools as well, regardless if applying to the RFP in scope or not.
- Document #1: Tech stack and User Stories
- Chosen tech stack and architecture, paired with user stories detailing high-level expected functionalities.
- Functional MVP
- A complete functional demo showcasing the calculated costs for at least one subgraph for a particular query volume and dataset, computed against Cloud Providers’ publicly available information on hardware pricing
- This MVP should make use of the underlying solution presented in #1 as much as possible. The demo can either be a live demo or a recording privately shared with The Graph Foundation.
- Final solution: all source code publicly shared in a GitHub repo, with accompanying documentation on usage, expected input, and how to make sense of resulting output. If exposing a RESTful API, an OpenAPI specification should be provided.