Originate v2 source routes and wire fragmentIdFilter targeted resync (#1378)

* Originate v2 source routes and wire fragmentIdFilter targeted resync

Part A — source-route origination policy:
- Gate route application (BLESourceRouteOriginationPolicy): only packets we
  author, directed at a single peer, with TTL headroom, whose recipient is
  not directly connected. Relays no longer attach routes to (and re-sign)
  packets they merely forward.
- Version-gate paths: MeshTopologyTracker records the highest protocol
  version observed per peer; BFS routes require every intermediate hop and
  the recipient to be v2-observed, capped at 4 intermediate hops.
- Degrade on failure: BLESourceRouteFailureCache marks a routed send that
  sees no inbound traffic from the recipient within 10s as failed and floods
  for 60s before retrying routes.

Part B — REQUEST_SYNC fragmentIdFilter (TLV 0x06):
- Requester: BLEFragmentAssemblyBuffer reports stalled broadcast
  reassemblies (no new fragment for 5s, retried at most every 10s); the
  maintenance pass sends a types=fragment REQUEST_SYNC naming the stalled
  8-byte fragment stream IDs to each connected peer.
- Responder: GossipSyncManager restricts the fragment diff to exactly the
  named streams, bypassing the since-cursor while the GCS filter still
  excludes pieces the requester holds; RSR/TTL-0/rate-limit semantics
  unchanged and REQUEST_SYNC stays link-local.
- Bounds: at most 60 IDs per request (60*17-1 = 1019 bytes <= the 1024-byte
  decoder cap); oversized 0x06 values are ignored, not fatal.

Docs: SOURCE_ROUTING.md gains the iOS origination policy (§8);
REQUEST_SYNC_MANAGER.md documents 0x05/0x06 as implemented.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Fix stall-clock refresh on duplicates and overflow suppression in fragment resync

Two fixes to stalledBroadcastFragmentIDs bookkeeping in
BLEFragmentAssemblyBuffer:

- Duplicate fragments no longer reset the stall clock. Fragment packets
  bypass the packet deduplicator, so relayed duplicates of an
  already-held index arriving every few seconds kept lastFragmentAt
  fresh and suppressed the targeted REQUEST_SYNC indefinitely. Now
  lastFragmentAt only updates when the index is new (actual progress).

- Only the streams that will actually be encoded on the wire are
  rate-limited. Previously every stalled candidate got
  lastResyncRequestAt set, but encodeFragmentIdFilter serializes at most
  RequestSyncPacket.maxFragmentIdFilterCount (60) IDs, so overflow
  streams were suppressed for retryAfter without ever being requested.
  Selection now caps at that shared constant, oldest stall first, so
  overflow stays eligible and rotates fairly on the next pass.

Tests: duplicates arriving periodically still trigger the stall report;
70 stalled streams yield the 60 oldest on the first pass and the
remaining 10 on the next.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: jack <jackjackbits@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

bitchat/Services/BLE/BLEFragmentAssemblyBuffer.swift

@@ -64,6 +64,9 @@ struct BLEFragmentAssemblyBuffer {
         let type: UInt8
         let total: Int
         let timestamp: Date
+        let isBroadcast: Bool
+        var lastFragmentAt: Date
+        var lastResyncRequestAt: Date?
     }
 
     private var fragmentsByKey: [BLEFragmentKey: [Int: Data]] = [:]
@@ -105,7 +108,15 @@ struct BLEFragmentAssemblyBuffer {
             return .oversized(header: header, projectedSize: projectedSize, limit: limit, started: started)
         }
 
+        // Only actual progress resets the stall clock: fragment packets
+        // bypass the packet deduplicator, so relayed duplicates of an
+        // already-held index must not keep suppressing the targeted
+        // REQUEST_SYNC for a stalled stream.
+        let isNewIndex = fragmentsByKey[header.key]?[header.index] == nil
         fragmentsByKey[header.key]?[header.index] = header.fragmentData
+        if isNewIndex {
+            metadataByKey[header.key]?.lastFragmentAt = now
+        }
 
         guard let fragments = fragmentsByKey[header.key],
               fragments.count == header.total else {
@@ -138,10 +149,59 @@ struct BLEFragmentAssemblyBuffer {
         }
 
         fragmentsByKey[header.key] = [:]
-        metadataByKey[header.key] = Metadata(type: header.originalType, total: header.total, timestamp: now)
+        metadataByKey[header.key] = Metadata(
+            type: header.originalType,
+            total: header.total,
+            timestamp: now,
+            isBroadcast: header.isBroadcastFragment,
+            lastFragmentAt: now
+        )
         return true
     }
 
+    /// Fragment stream IDs (8-byte, big-endian) of incomplete broadcast
+    /// reassemblies that have not seen a new fragment for `stalledAfter`
+    /// seconds — candidates for a targeted REQUEST_SYNC. Each returned
+    /// stream is marked so it is not re-requested within `retryAfter`.
+    /// At most `RequestSyncPacket.maxFragmentIdFilterCount` streams are
+    /// returned per pass — the wire filter cannot carry more — selected
+    /// oldest-stall first; overflow streams stay unmarked and eligible for
+    /// the next pass. Directed reassemblies are excluded: peers only archive
+    /// broadcast fragments for gossip sync, so a targeted request cannot
+    /// recover them.
+    mutating func stalledBroadcastFragmentIDs(
+        stalledAfter: TimeInterval,
+        retryAfter: TimeInterval,
+        now: Date = Date()
+    ) -> [Data] {
+        var candidates: [(key: BLEFragmentKey, lastFragmentAt: Date)] = []
+        for (key, metadata) in metadataByKey {
+            guard metadata.isBroadcast,
+                  let fragments = fragmentsByKey[key],
+                  fragments.count < metadata.total,
+                  now.timeIntervalSince(metadata.lastFragmentAt) >= stalledAfter else { continue }
+            if let lastRequest = metadata.lastResyncRequestAt,
+               now.timeIntervalSince(lastRequest) < retryAfter { continue }
+            candidates.append((key: key, lastFragmentAt: metadata.lastFragmentAt))
+        }
+
+        // Mark only the streams that will actually go on the wire, so the
+        // overflow is not silently suppressed for `retryAfter`.
+        let selected = candidates
+            .sorted {
+                if $0.lastFragmentAt != $1.lastFragmentAt {
+                    return $0.lastFragmentAt < $1.lastFragmentAt
+                }
+                return ($0.key.sender, $0.key.id) < ($1.key.sender, $1.key.id)
+            }
+            .prefix(RequestSyncPacket.maxFragmentIdFilterCount)
+
+        return selected.map { candidate in
+            metadataByKey[candidate.key]?.lastResyncRequestAt = now
+            return withUnsafeBytes(of: candidate.key.id.bigEndian) { Data($0) }
+        }
+    }
+
     private static func assemblyLimit(for originalType: UInt8) -> Int {
         if originalType == MessageType.fileTransfer.rawValue {
             // Allow headroom for TLV metadata and binary framing overhead.

bitchat/Services/BLE/BLEService.swift

@@ -69,7 +69,9 @@ final class BLEService: NSObject {
     #endif
     private var selfBroadcastTracker = BLESelfBroadcastTracker()
     private let meshTopology = MeshTopologyTracker()
-    
+    // Route health for originated source routes; guarded by collectionsQueue.
+    private var sourceRouteFailures = BLESourceRouteFailureCache()
+
     // 5. Fragment Reassembly (necessary for messages > MTU)
     private var fragmentAssemblyBuffer = BLEFragmentAssemblyBuffer()
     private var outboundFragmentTransfers = BLEOutboundFragmentTransferScheduler()
@@ -586,6 +588,7 @@ final class BLEService: NSObject {
             let entries = outboundFragmentTransfers.removeAll().map { ($0.id, $0.workItems) }
             peerRegistry.removeAll()
             fragmentAssemblyBuffer.removeAll()
+            sourceRouteFailures = BLESourceRouteFailureCache()
             // Also clear pending message queues to avoid stale state across sessions
             pendingNoiseSessionQueues.removeAll()
             pendingDirectedRelays.removeAll()
@@ -2392,13 +2395,31 @@ extension BLEService {
     }
 
     private func computeRoute(to peerID: PeerID) -> [Data]? {
-        meshTopology.computeRoute(from: myPeerIDData, to: routingData(for: peerID))
+        // Version-gated: every hop and the recipient must have been observed
+        // speaking v2, since a v1-only node drops v2 frames on decode.
+        meshTopology.computeRoute(
+            from: myPeerIDData,
+            to: routingData(for: peerID),
+            maxHops: TransportConfig.bleSourceRouteMaxIntermediateHops,
+            requiringVersion: 2
+        )
     }
 
     private func applyRouteIfAvailable(_ packet: BitchatPacket, to recipient: PeerID) -> BitchatPacket {
-        guard let route = computeRoute(to: recipient), route.count >= 1 else {
-            return packet
-        }
+        let now = Date()
+        let route = BLESourceRouteOriginationPolicy.route(
+            for: packet,
+            to: recipient,
+            localPeerIDData: myPeerIDData,
+            isRecipientConnected: { self.isPeerConnected($0) },
+            shouldAttemptRoute: { peer in
+                self.collectionsQueue.sync(flags: .barrier) {
+                    self.sourceRouteFailures.shouldAttemptRoute(to: peer, now: now)
+                }
+            },
+            computeRoute: { self.computeRoute(to: $0) }
+        )
+        guard let route else { return packet }
         // Create new packet with route applied and version upgraded to 2
         let routedPacket = BitchatPacket(
             type: packet.type,
@@ -2416,6 +2437,9 @@ extension BLEService {
             SecureLogger.error("❌ Failed to re-sign packet with route", category: .security)
             return packet // Return original packet if signing fails
         }
+        collectionsQueue.sync(flags: .barrier) {
+            sourceRouteFailures.noteRoutedSend(to: recipient, now: now)
+        }
         return signedPacket
     }
 
@@ -3187,13 +3211,23 @@ extension BLEService {
         // Update peer info without verbose logging - update the peer we received from, not the original sender
         updatePeerLastSeen(peerID)
 
-        // Track recent traffic timestamps for adaptive behavior
+        // Track recent traffic timestamps for adaptive behavior; the same
+        // barrier hop confirms route health for the packet's originator.
         collectionsQueue.async(flags: .barrier) { [weak self] in
             guard let self = self else { return }
             self.recentTrafficTracker.recordPacket(at: Date())
+            self.sourceRouteFailures.noteInboundActivity(from: senderID)
+        }
+
+        // Per-peer protocol version: originated source routes only use hops
+        // observed speaking v2 (a v1-only node cannot decode v2 frames).
+        if packet.version >= 2 {
+            meshTopology.recordObservedVersion(packet.version, for: packet.senderID)
+            if peerID != senderID {
+                meshTopology.recordObservedVersion(packet.version, for: routingData(for: peerID))
+            }
         }
 
-        
         // Process by type
         switch context.messageType {
         case .announce:
@@ -3760,10 +3794,21 @@ extension BLEService {
         // Clean old processed messages efficiently
         messageDeduplicator.cleanup()
         
-        // Clean old fragments (> configured seconds old)
-        collectionsQueue.sync(flags: .barrier) {
+        // Clean old fragments (> configured seconds old), then ask peers for
+        // the specific fragment streams whose reassembly has stalled instead
+        // of waiting for the next periodic GCS fragment round.
+        let stalledFragmentIDs = collectionsQueue.sync(flags: .barrier) { () -> [Data] in
             let cutoff = now.addingTimeInterval(-TransportConfig.bleFragmentLifetimeSeconds)
             fragmentAssemblyBuffer.removeExpired(before: cutoff)
+            sourceRouteFailures.prune(now: now)
+            return fragmentAssemblyBuffer.stalledBroadcastFragmentIDs(
+                stalledAfter: TransportConfig.bleFragmentResyncStallSeconds,
+                retryAfter: TransportConfig.bleFragmentResyncRetrySeconds,
+                now: now
+            )
+        }
+        if !stalledFragmentIDs.isEmpty {
+            gossipSyncManager?.requestMissingFragments(fragmentIDs: stalledFragmentIDs)
         }
 
         // Clean old connection timeout backoff entries (> window)

bitchat/Services/BLE/BLESourceRouteFailureCache.swift

@@ -0,0 +1,95 @@
+import BitFoundation
+import Foundation
+
+/// Tracks whether source-routed sends to a recipient appear to be working.
+///
+/// A routed unicast rides exactly one path, so a broken hop silently loses the
+/// packet where a flood would have healed around it. Rather than building a
+/// retransmission machine (MessageRouter already retries at a higher layer),
+/// this cache degrades: a routed send that sees no inbound traffic from the
+/// recipient within the confirmation window marks the route as failed, and
+/// subsequent sends fall back to flooding until the suppression TTL lapses.
+struct BLESourceRouteFailureCache {
+    struct Config {
+        /// How long a routed send may go unconfirmed before it counts as a
+        /// route failure.
+        var confirmationWindowSeconds: TimeInterval = TransportConfig.bleSourceRouteConfirmationWindowSeconds
+        /// How long to flood instead of routing after a failure.
+        var suppressionSeconds: TimeInterval = TransportConfig.bleSourceRouteSuppressionSeconds
+    }
+
+    private struct State {
+        var pendingSince: Date?
+        var suppressedUntil: Date?
+    }
+
+    private let config: Config
+    private var states: [PeerID: State] = [:]
+
+    init(config: Config = Config()) {
+        self.config = config
+    }
+
+    /// Whether the next directed send to `recipient` may carry a source
+    /// route. Flips the recipient into suppression when the last routed send
+    /// went unconfirmed past the confirmation window.
+    mutating func shouldAttemptRoute(to recipient: PeerID, now: Date = Date()) -> Bool {
+        guard var state = states[recipient] else { return true }
+
+        if let until = state.suppressedUntil {
+            guard now >= until else { return false }
+            state.suppressedUntil = nil
+        }
+
+        if let pending = state.pendingSince,
+           now.timeIntervalSince(pending) > config.confirmationWindowSeconds {
+            // The routed send was never confirmed: treat the route as broken
+            // and flood until the suppression window lapses.
+            state.pendingSince = nil
+            state.suppressedUntil = now.addingTimeInterval(config.suppressionSeconds)
+            states[recipient] = state
+            return false
+        }
+
+        states[recipient] = state
+        return true
+    }
+
+    /// Records that a source-routed packet was sent to `recipient`. Keeps the
+    /// earliest unconfirmed send so back-to-back packets share one deadline.
+    mutating func noteRoutedSend(to recipient: PeerID, now: Date = Date()) {
+        var state = states[recipient] ?? State()
+        if state.pendingSince == nil {
+            state.pendingSince = now
+        }
+        states[recipient] = state
+    }
+
+    /// Any inbound packet authored by `peer` confirms the pending routed send
+    /// (delivery acks and replies arrive this way). Deliberately does not
+    /// lift an active suppression: that traffic may have arrived via flood.
+    mutating func noteInboundActivity(from peer: PeerID) {
+        guard var state = states[peer] else { return }
+        state.pendingSince = nil
+        if state.suppressedUntil == nil {
+            states.removeValue(forKey: peer)
+        } else {
+            states[peer] = state
+        }
+    }
+
+    /// Drops entries that can no longer influence a routing decision. An
+    /// expired-but-unconverted pending entry is kept for as long as the
+    /// suppression it would trigger could still be active.
+    mutating func prune(now: Date = Date()) {
+        let pendingRetention = config.confirmationWindowSeconds + config.suppressionSeconds
+        states = states.filter { _, state in
+            if let until = state.suppressedUntil, now < until { return true }
+            if let pending = state.pendingSince,
+               now.timeIntervalSince(pending) <= pendingRetention {
+                return true
+            }
+            return false
+        }
+    }
+}

bitchat/Services/BLE/BLESourceRouteOriginationPolicy.swift

@@ -0,0 +1,40 @@
+import BitFoundation
+import Foundation
+
+/// Decides whether an outbound directed packet should carry a v2 source
+/// route. Pure gating logic so BLEService's hot send path stays a thin wire.
+enum BLESourceRouteOriginationPolicy {
+    /// Returns the intermediate-hop route to attach, or nil to keep the
+    /// current flood/direct-write behavior unchanged.
+    ///
+    /// Routes are only originated when every gate passes:
+    /// - we authored the packet (relays must not rewrite and re-sign someone
+    ///   else's packet; route-following for in-flight routed packets lives in
+    ///   `BLERouteForwardingPolicy`),
+    /// - the packet is directed at a single peer (not broadcast),
+    /// - the packet has TTL headroom to traverse hops (link-local TTL-0
+    ///   packets like REQUEST_SYNC never route),
+    /// - the recipient is not directly connected (a direct write already
+    ///   delivers in one hop),
+    /// - routing to the recipient is not suppressed by a recent unconfirmed
+    ///   routed send, and
+    /// - the topology yields a complete path.
+    static func route(
+        for packet: BitchatPacket,
+        to recipient: PeerID,
+        localPeerIDData: Data,
+        isRecipientConnected: (PeerID) -> Bool,
+        shouldAttemptRoute: (PeerID) -> Bool,
+        computeRoute: (PeerID) -> [Data]?
+    ) -> [Data]? {
+        guard packet.senderID == localPeerIDData else { return nil }
+        guard let recipientData = packet.recipientID,
+              recipientData.count == 8,
+              !recipientData.allSatisfy({ $0 == 0xFF }) else { return nil }
+        guard packet.ttl > 1 else { return nil }
+        guard !isRecipientConnected(recipient) else { return nil }
+        guard shouldAttemptRoute(recipient) else { return nil }
+        guard let route = computeRoute(recipient), !route.isEmpty else { return nil }
+        return route
+    }
+}