- Working on auto retrieval feature right now. Bridge from bitswap client to retrieve things on Filecoin
- Connects to bitswap and starts serving requests
- Estuary has root CIDs and branch CIDs - uses Estuary’s database
- Concurrency limit - some have it set super low (2 or 1)
- Could be a bottleneck
- Highest that he saw was 30
- minerX3: 10, 5, 6, 2, 1
- Wait for information to start coming in - ttfb is very high, doesn’t seem super responsive
- As a user downloading, it ends up taking a while (1 min or so) to start getting data in
- With auto retrieve, when we send a request from bitswap, our server isn’t able to respond with a minute, b/c it is waiting for data to come in
- Overall responsiveness is not good
- If miner has 2 retrievals going at the same time, need to queue to wait for the previous retrievals to finish processing
- Your retrieval will block until it’s ready
- Is there a way to deprioritize connections?
- If a download isn’t progressing from a miner, it doesn’t get cancelled, unless a miner automatically cancels it..
- Retrievals could get abandoned
- Couldn't figure out any way through the code that you’re not going to do the retrieval anymore (through go-data-transfer)
- Automatically manage dropped connections