IPNI Weekly 03/20

:network-indexer: InterPlanetary Network Indexer week 10

Community engagement:

IPNI Weekly 03/13

:network-indexer: InterPlanetary Network Indexer week 10

Community engagement:

Team prepared a video presentation about the importance of the IPNI and why storage providers should care for the web3 Big Data Summit in China on 04/07 will post here once it’s aired!

IPNI team reviewing the CAR indexing wg scope to support their efforts and identify any possible overlap that either reduces their efforts or identifies dependencies. Present perspective is that there is little overlap, but identifying IPNI members as stakeholders on their technical design plan is seen as valuable.

IPNI team is cultivating a set of talks in preparation for the IPFS Thing.

Double Hashing for reader privacy

Our ingestion process(we are ingesting the entire value store of the network indexer and double hashing it so that there is a parallel private index that can start serving production double hashed queries) is already back up to ~58% ingested!

We expect the effort to be completed completed by the end of this week and @Ivan Schasny is going to help us end the quarter with an analysis of the findings of our data we gained from this process.

@Ivan Schasny implemented dhfind, a service that cascades non-doublehashed lookups onto the double hashed index records by double hashing the queried multihash on behalf of the clients. The service then gathers coverage metrics to help us infer what percentage of query traffic could be satisfied via the reader-privacy enabled path which help validate duble-hashing coverage and pro-actively mitigate risks associated with fully switching queries to use double-hashed lookup path. The service is expected to be deployed and operational next week.

Indexer Scalability Analysis of the first failed ingestion process illuminated that there were several areas of efficiency improvements possible which will ultimately greatly improve the scalability of the indexer as a service. @Andrew Gillis identified several seperations of duties for different indexer workstreams he is planning to tackle(things like chain ingestion, processing the chain, processing car files).