Nice — I’ll walk you through a detailed, under-the-hood roadmap: how every component behaves, the data flows, exact building steps, algorithms, failure modes, tests, and concrete checkpoints so you (and a judge) can verify correctness. I’ll keep it implementation-focused and modular so you can pick Node.js or Go pieces as you build.
High-level design recap (one line)
Client chunks → optional FEC parity → prioritized WebTransport streams → server verifies chunks (SHA-256 → Merkle) → reassembles file → relay (optional) stores/forwards → dashboard shows realtime state.
1. Core design concepts (how it works under the hood)
1.1 Session & Streams
- A WebTransport session (over HTTP/3/QUIC) is established between client and server.
- Within the session:
- Control bidirectional stream: JSON control messages (metadata, ACK/NACK, bitmap, stats).
- Priority unidirectional streams: one or more streams per priority class (high/normal/low). Streams carry chunk frames (binary).
- Optional telemetry stream: for dashboard updates if you prefer dedicated channel.
1.2 Chunking & Stripe formation
- File is split into fixed-size chunks (default 64 KB, last chunk smaller).
- Chunks are grouped into stripes of
k data shards and m parity shards for Reed-Solomon FEC.
- Example: k=10, m=4 → 14 shards per stripe. Each data shard is a single chunk (or padded to chunk size).
- Encoding steps:
- Collect
k chunks.
- Run Reed-Solomon encoder → produce
m parity shards.
- Send all
k data shards; parity shards go on low-priority stream (or after data shards).
1.3 Integrity & Merkle root
- For each chunk compute
SHA-256(chunk).
- Keep an array of chunk hashes in chunk index order. After all chunk hashes are known compute a Merkle tree; publish Merkle root in the file metadata at session start.
- Server validates each chunk hash on arrival and at finalization recomputes Merkle root to match metadata.
1.4 Resume & Bitmap
- Both client and server maintain a persistent bitmap: one bit per chunk (0 = missing, 1 = received).
- On reconnect:
- Server sends its bitmap for
file_id.
- Client computes difference and sends only missing chunks (and parity as required).
1.5 Prioritization