2022-12-13 Content Routing WG #2

Agenda

Important reference documents:

Purpose of Meeting 5m:

To agree on technical implementation of double hashing being proposed, and feedback any questions or concerns regarding future implementations of ambient content routing.

Topics to cover 30m:

Items from last meeting:

HTTP Delegated Routing rollout to Kubo and gateways:
- Spec: https://github.com/ipfs/specs/pull/337
- Rollout
  - https://github.com/ipfs/go-delegated-routing/issues/63
    - cid.contact - Support for HTTP delegated routing is now deployed to production in cid.contact
    - Hydra CPU utilization took a hit testing and implementing today
    - Kubo - cid.contact will be enabled next to the DHT by default in 0.18.0-rc1 (Routing.Type=auto)
  - https://github.com/ipfs/kubo/issues/9150 and https://github.com/protocol/bifrost-infra/issues/2183
    - ipfs.io gateway
    - @mario Camou owns this on the bifrost team, plans to complete the effort by next Monday, may be additional dependencies outside of the currently recognized effort. Bifrost rollout with delegated routing.
    - Point to focus on leveraging HTTP delegated routing as opposed to reframe.
CID.contact server side metrics for the HTTP delegated routing endpoint that was added?
- Looking for a class of metrics(call volume/performance details)
  - Translation on ingress pathway means that current metrics apply - we have the metrics to support this.
Kubo will send /PUTS, how do we track that? cid.contact only supports /GET

https://github.com/ipni/indexstar/issues/54
Commitment to double-hashing design pathway.
- Reader Privacy Specification Github
- https://github.com/ipni/specs/pull/5
- PoC demonstrates 1.5x(isolate to read operation) perf hit over baseline(down from 10x estimated) testing NON production.
  - Sub topic data migration of “single hashed CID’s” Big migration/In situ read migration + simultaneous back-processing of un-retrieved, but indexed CID’s.
    - We may start a new indexer with the extra hashing on and wait for it to-reingest history from scratch. backlog may need extra CPU for the hashing/encryption
    - Also we could approach this by just starting a dedicated node and get the entire re-ingestion over with over the course of a week and stomach the impact to metrics.
  - 2 modes of operation
    - Serving normal traffic to unencrypted CID’s
    - Enabling lookups in private mode
- Currently we have 1 round trip. This now mode will result in a second round trip(that’s avoidable under specific circumstances)
  - The benefit of indexers is to reduce round trips(minimize speed of light impacts) we need to highlight that this is an impact.
  - Is there an upgrade path in the future we need to keep in mind here?
    
    We’ve left specific encryption out of the discussion but propose the same encryption DHT is potentially using. If we want to upgrade the protocol/spec if we only use SHA-256 we may be limiting our future potential.
  @Steve (biglep) : I think we should add more details about the caching mechanisms and upgrade path. If we’re doing the same as the IPFS DHT lets reference/link to that.
  - Any change to how content providers push records?
    - No specifically focusing on the readers part. Clients send request with hash/multihash of the CID, client decrypts peerID using original multihash, indexer replies with provider records encrypted with original peerID. Reply includes full peerID.
  - Request k-anonymity/plausible deniability entity provider record content access obfuscation. Small network overhead but big privacy benefit.
  - Reader privacy of the DHT/Ambient Routing/Reader privacy of the Indexer are simultaneous. Can we speed this process up by parallelizing the efforts?
    - Bedrock proposes to focus on reader privacy ahead of ambient discovery.
Next Steps:

Changing write privacy will be a follow on iteration. We’re presently focused entirely on reader privacy.

Key point, reader privacy alone will make it expensive to crawl and read the network, but writer privacy implementation will drawdown the systemic cost.

Performance measurements MVP roll-out.

Data migration pathway

Easiest for us best impact outcomes for customer.

End of Jan/Feb is the timeline for full read implementation and migration completed.

Additionally implement a client library to do these interactions.

Does Kubo need to write it’s own go code for this API? Bedrock will provide the client go library that can be used in Kubo. Go libipfs repo has the monorelated utilitites/packages so versioning between the tools is in place. Possible place we can house this.

We are aligned with the Chainsafe implementation.
Reader privacy from the DHT side of things?
- Chainsafe is implementing and attempting to leverage test ground for the testing. There is a small concern in the design that’s being remediated presently. They can migrate what we have now and iterate an improvement later for privacy by default, but we’d rather minimize the DHT migrations. All providers need to publish on both old and new DHT during the migration which we’d like to avoid.
- They can ship readerprivacy for the indexers and focus on working on the more comprehensive solution as a second iteration.
- No IPIP yet for the DHT migration.
These changes are being made independent of Kubo.
Clarification: Kubo integration and double-hashing implementation are fully decoupled and no longer present cross team dependencies?
- Yes this is the case. Specification for HTTP we don’t have yet, but we will have it by the end of January. There aren’t obvious problems. So long as we have the API we’ll be fine.
Ambient Routing, Will is working on the technical design spec. More on timeline post completion.
- Reviews still outstanding do we require more details to approve?
- @Will Scott ‘s initial thoughts on MVP scope are here, but maybe this changes with cid.contact already hard-coded. What is the next iteration point we want to aim for?