The story so far

Graphsync’s protocol refactor began as a small change to switch request IDs to UUIDs. As we got started, we realized since breaking protocol changes are painful, we wanted to get as much done as possible. This led to the protocol refactor becoming much larger – we changed the serialization format to CBOR and made the protocol officially an “IPLD” based protocol with a schema. We made several other long standing desired changes. We’ve begun to lay the foundation for the features that would remove the need for go-data-transfer’s libp2p protocol entirely. This would enable go-data-transfer to operate entirely on top of GraphSync, without the complexity of having its own libp2p library.

However, we need to land this soon, and we need to figure out the path to landing.

Where things stand

  1. Graphsync’s new 2.0 protocol is implemented to feature parity to the 1.0 protocol, with important upgrades to the format (UUIDs + CBOR + request types + built in metadata) that unlocks many improvements, some of which are already implemented.
  2. Go-graphsync is implemented and tested to support both 1.0 & 2.0 protocols for the time being – it’s backwards compatible no matter who it talks to.
  3. The new version of go-graphsync is NOT integrated into go-data-transfer, markets, or up to Lotus
  4. There is a Lotus feature release with a code freeze on March 8th we’d like to get the protocol into.
    1. This would be followed by a consensus release later in the month.
    2. Once the protocol ships with a consensus release, it’s safe to assume miners support it, which enables us to move towards dropping support for the 1.0 protocol in the future.
  5. There are a few remaining outstanding protocol improvements we need to make if we want to eliminate go-data-transfer’s libp2p protocol.

Scoping integration work

(warning these estimates are rough)

  1. The basics (can’t ship to Lotus without these)
    1. Merge the protocol work into master, along with current outstanding PRs - 1-2 programmer days
    2. Upgrade go-data-transfer, go-fil-markets, and lotus to use the new version of go-graphsync. Deal with a small set of breaking changes in library interfaces - 1 programmer week
    3. Do some benchmarking and testing prior to getting in the release candidate (there will be more testing in the release candidate phase) - 3-4 programmer days
  2. Protocol completeness (these are the remaining features we need to support in order for the Graphsync protocol to avoid upgrades in the near future if we decide to eliminate the libp2p go-data-transfer protocoll)
    1. New pausing protocol - https://github.com/ipfs/go-graphsync/issues/347 - 3-4 programmer days
    2. Push requests - https://github.com/ipfs/go-graphsync/issues/346 - 2-3 programmer days
    3. Restart requests – tbd - 4-5 programmer days
    4. Completing scope of https://github.com/ipfs/go-graphsync/issues/335 - 4-5 programmer days
  3. Shipping refactored go-data-transfer that can operate entirely on GraphSync as its libp2p protocol for supporting clients
    1. Switch go-data-transfer to use UUIDs - 2-3 programmers weeks, and release will need to run a migration
    2. Implement new interfaces for Transport to work with graphsync new features and operation without go-data-transfer protocol - 2 programmer weeks
    3. Implement backwards compatibility for older graphsync clients to switch between graphsync 1.0 + data transfer libp2p & graphsync 2.0 only - 3-4 programmer weeks
    4. Possible other changes to simplify stack - 2 programmer weeks

Suggested prioritization

March 8th is 3 weeks away. With 2 programmers on the project, that means we can get done about 6 programmer weeks of worth of work if we line up workstreams so we can make independent progress.

Realistically, think it’s better to say 5 programmer weeks.

In my mind the prioritization for landing should look like this: