Originate v2 source routes and wire fragmentIdFilter targeted resync (#1378)

* Originate v2 source routes and wire fragmentIdFilter targeted resync

Part A — source-route origination policy:
- Gate route application (BLESourceRouteOriginationPolicy): only packets we
  author, directed at a single peer, with TTL headroom, whose recipient is
  not directly connected. Relays no longer attach routes to (and re-sign)
  packets they merely forward.
- Version-gate paths: MeshTopologyTracker records the highest protocol
  version observed per peer; BFS routes require every intermediate hop and
  the recipient to be v2-observed, capped at 4 intermediate hops.
- Degrade on failure: BLESourceRouteFailureCache marks a routed send that
  sees no inbound traffic from the recipient within 10s as failed and floods
  for 60s before retrying routes.

Part B — REQUEST_SYNC fragmentIdFilter (TLV 0x06):
- Requester: BLEFragmentAssemblyBuffer reports stalled broadcast
  reassemblies (no new fragment for 5s, retried at most every 10s); the
  maintenance pass sends a types=fragment REQUEST_SYNC naming the stalled
  8-byte fragment stream IDs to each connected peer.
- Responder: GossipSyncManager restricts the fragment diff to exactly the
  named streams, bypassing the since-cursor while the GCS filter still
  excludes pieces the requester holds; RSR/TTL-0/rate-limit semantics
  unchanged and REQUEST_SYNC stays link-local.
- Bounds: at most 60 IDs per request (60*17-1 = 1019 bytes <= the 1024-byte
  decoder cap); oversized 0x06 values are ignored, not fatal.

Docs: SOURCE_ROUTING.md gains the iOS origination policy (§8);
REQUEST_SYNC_MANAGER.md documents 0x05/0x06 as implemented.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Fix stall-clock refresh on duplicates and overflow suppression in fragment resync

Two fixes to stalledBroadcastFragmentIDs bookkeeping in
BLEFragmentAssemblyBuffer:

- Duplicate fragments no longer reset the stall clock. Fragment packets
  bypass the packet deduplicator, so relayed duplicates of an
  already-held index arriving every few seconds kept lastFragmentAt
  fresh and suppressed the targeted REQUEST_SYNC indefinitely. Now
  lastFragmentAt only updates when the index is new (actual progress).

- Only the streams that will actually be encoded on the wire are
  rate-limited. Previously every stalled candidate got
  lastResyncRequestAt set, but encodeFragmentIdFilter serializes at most
  RequestSyncPacket.maxFragmentIdFilterCount (60) IDs, so overflow
  streams were suppressed for retryAfter without ever being requested.
  Selection now caps at that shared constant, oldest stall first, so
  overflow stays eligible and rotates fairly on the next pass.

Tests: duplicates arriving periodically still trigger the stall report;
70 stalled streams yield the 60 oldest on the first pass and the
remaining 10 on the next.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: jack <jackjackbits@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
jack
2026-07-07 14:42:53 +02:00
committed by GitHub
co-authored by jack Claude Fable 5
parent cee2bcd535
commit 70229f0be1
17 changed files with 1049 additions and 43 deletions
+5 -3
View File
@@ -20,9 +20,11 @@ The new implementation introduces a **RequestSyncManager** to track outgoing syn
### Request Sync Payload
The `REQUEST_SYNC` packet payload (TLV encoded) has been updated to include:
* **Future Filters**:
* `sinceTimestamp` (Type 0x05): To request packets since a certain time (UInt64 big-endian).
* `fragmentIdFilter` (Type 0x06): To request specific fragments (UTF-8 string).
* `sinceTimestamp` (Type 0x05): filter-coverage cursor (UInt64 big-endian). The requester's GCS filter only covers packets at or after this timestamp; the responder skips older packets instead of re-sending them every round.
* `fragmentIdFilter` (Type 0x06): targeted fragment resync (UTF-8 string). Comma-separated 16-hex-char (8-byte) fragment **stream IDs** — the ID that prefixes every fragment payload.
* **Requester**: when a broadcast reassembly stalls (no new fragment for 5 s), the fragment assembler reports the stream ID and a `REQUEST_SYNC` with `types = fragment` and this filter goes to each connected peer (re-requested at most every 10 s per stream). Directed reassemblies are excluded — peers only archive broadcast fragments for sync.
* **Responder**: when the filter is present, the fragment diff is restricted to exactly the named streams and the `sinceTimestamp` cursor is bypassed for them; the GCS filter still excludes pieces the requester already holds. Responses keep RSR marking, TTL 0, per-peer response rate limiting (8/30 s), and `REQUEST_SYNC` itself remains link-local (TTL 0, never relayed).
* **Bounds**: at most 60 IDs per request. Each ID encodes as 16 hex chars plus a comma separator, so the largest value is 60 × 17 1 = 1019 bytes, within the decoder's 1024-byte acceptance cap; oversized filter values are ignored (the rest of the request still decodes).
## Architecture
+42 -1
View File
@@ -2,7 +2,7 @@
This document specifies the Source-Based Routing extension (v2) for the BitChat protocol. This upgrade enables efficient unicast routing across the mesh by allowing senders to specify an explicit path of intermediate relays.
**Status:** Implemented in Android and iOS. Backward compatible (v1 clients ignore routing data).
**Status:** Implemented in Android and iOS: both decode routed packets, forward along routes, and originate routes. iOS origination is policy-gated (see §8). Backward compatible (v1 clients never receive routed frames from iOS: routes are only originated when every node on the path has been observed speaking v2).
---
@@ -144,3 +144,44 @@ When a node receives a packet **not** addressed to itself:
* **Fallback:** If the Next Hop is unreachable, **fall back to broadcast/flood** to ensure delivery.
3. **If NO (Standard):**
* Flood the packet to all connected neighbors (subject to TTL and probability rules).
---
## 8. iOS Origination Policy
iOS attaches a route (upgrading the packet to v2 and re-signing it) only when
**all** of the following hold at send time (`BLESourceRouteOriginationPolicy`):
1. **Authored locally.** The packet's `SenderID` is our own peer ID. Relays
never rewrite someone else's packet — adding a route would force a
re-sign under the wrong key. Relays only *follow* existing routes
(`BLERouteForwardingPolicy`).
2. **Directed.** The packet has a single-peer `RecipientID` (not the
broadcast ID). In practice this covers Noise-encrypted private traffic,
private file transfers, and their fragments (fragments inherit the
parent's route and version, per §5).
3. **TTL headroom.** `TTL > 1`. Link-local packets (e.g. `REQUEST_SYNC`,
always TTL 0) never carry routes.
4. **Recipient not directly connected.** A direct write already delivers in
one hop; a route would only add bytes.
5. **Complete v2 path exists.** BFS over the confirmed-edge mesh graph
(`MeshTopologyTracker`, built from verified announce `directNeighbors`
claims, entries expiring after 60 s) finds a path with **at most 4
intermediate hops** where every intermediate hop **and the recipient**
has been observed originating or relaying a v2 packet. Nodes never seen
speaking v2 are assumed v1-only and are excluded — a v1 client cannot
decode a v2 frame, so routing through it would silently drop the packet.
6. **No recent route failure.** See below.
If any gate fails, behavior is exactly the pre-routing flood/direct-write
path — v1 peers observe no change.
### Failure Fallback
A routed unicast rides one path; a broken hop loses the packet where a flood
would heal around it. iOS keeps a small per-recipient health cache
(`BLESourceRouteFailureCache`): a routed send that sees no inbound packet
authored by the recipient within 10 s counts as a route failure, and directed
sends to that recipient fall back to flooding for the next 60 s before
routing is attempted again. Retransmission of the payload itself stays where
it always was (MessageRouter and higher layers).