mirror of
https://github.com/permissionlesstech/bitchat.git
synced 2026-07-24 23:45:18 +00:00
f8ab0a7dc0
* Fix restore-path main↔bleQueue deadlock and courier drop amplification A device froze permanently in a two-phone test. Debugger stacks showed an ABBA deadlock: the main actor was in bleQueue.sync (delivery-ack send → broadcastPacket → readLinkState) while bleQueue was in main.sync (captureBluetoothStatus reading backgroundTimeRemaining). The load that lined the two edges up came from a courier-drop amplification storm: drop dedup was in-memory only while the outbox driving 120s re-deposits is persisted, so every relaunch republished the same undelivered DM as a fresh 24h relay drop and every gateway relaunch re-fetched the whole backlog — ~20 copies of one DM delivered in 40ms, each triggering decrypt + delivery + ack + handshake work. Fixes, in rank order: - Edge B (P0): captureBluetoothStatus no longer main.syncs from bleQueue; backgroundTimeRemaining is sampled on main and cached behind a lock. Invariant documented: bleQueue must NEVER sync-dispatch to main. - Edge A (P0, defense in depth): sendDeliveryAck / sendReadReceipt / sendPrivateMessage / sendNoisePayload / triggerHandshake hop to messageQueue like sendMessage, so no main-actor call path reaches readLinkState's bleQueue.sync. - Drop dedup (P1): publishedDropKeys and seenDropEventIDs persist across relaunches (new BridgeDropDedupStore, entries expire with the 24h NIP-40 drop window; wiped on panic) — one drop per message ID per 24h regardless of relaunch count. - Receiver dedup (P1): openCourierEnvelope dedups on the inner private message ID before delivery, so a duplicate copy costs one decrypt and never re-delivers, re-acks, or re-triggers a handshake. - Handshake gating (P2): queued acks initiate a Noise handshake only for reachable peers; mail from absent/rotated identities no longer turns each copy into a mesh-wide handshake flood (the ack stays queued and flushes when a session eventually establishes). - Outbox (P3): re-enqueueing a queued message ID carries over its depositedCourierKeys so resends stop re-burning the same courier slots. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Review fixes: offline-drop durability, gateway handoff retry, restore-log freshness, coalesced persist Adversarial review of the storm/deadlock PR surfaced four issues: - Offline blackhole (must-fix): a deposit made while relays were down persisted its dedup key even though the drop only sat in the in-memory pending queue — app killed before reconnect meant the relaunch lost the drop but the persisted key blocked every re-deposit for 24h. The persisted snapshot now excludes keys still pending; they become durable only when flushPendingDrops actually publishes them. - Gateway handoff: seen-event IDs were consumed before the deliverToPeer handoff; a failed handoff (peer walked away) permanently dropped the event for a single-gateway island. deliverToPeer now reports whether the handoff was attempted, and a failure releases the seen slot so a relaunch or backlog redelivery retries. - Restore-path logs: central/peripheral-restore captures logged the init sentinel bgRemaining=∞. The cache is now seeded in init's main-thread branch and restore captures route through the sampler, which refreshes the cached budget before logging. - Persist cost: the dedup record was a full JSON encode + atomic write on the main actor per mutation (once per event during a backlog re-fetch). Writes now coalesce behind a 1s window, flushed immediately on background/terminate; panic wipe stays immediate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Remove BoundedIDSet.remove, orphaned by the ExpiringIDSet migration The drop-dedup sets that needed slot release moved to ExpiringIDSet; remaining BoundedIDSet users only insert and check. Periphery caught it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: jack <jackjackbits@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Test Harness Guide
This test suite uses an in-memory networking harness to make end-to-end and integration tests deterministic, fast, and race-free without touching production code.
In-Memory Bus
- File:
bitchatTests/Mocks/MockBLEService.swift - Registry/Adjacency: Global
registrymapspeerIDto aMockBLEServiceinstance;adjacencyrecords simulated links between peers. - Setup: Call
MockBLEService.resetTestBus()insetUp()to clear state between tests. - Topology: Use
simulateConnectedPeer(_:)andsimulateDisconnectedPeer(_:)to add/remove links.connectFullMesh()helpers in tests build larger topologies. - Handlers: Tests can observe data via
messageDeliveryHandler(decodedBitchatMessage) andpacketDeliveryHandler(rawBitchatPacket). - De‑duplication: A thread-safe
seenMessageIDsprevents duplicate deliveries during flooding/relays.
Broadcast Flooding
- Flag:
MockBLEService.autoFloodEnabled - Intent: When
true, public broadcasts propagate across the entire connected component (ignores TTL for reach) while still de‑duping to prevent loops. - Usage: Enabled in Integration tests (
setUp) to simulate large-network broadcast; disabled in E2E tests to keep routing explicit and verify TTL behavior (seePublicChatE2ETests.testZeroTTLNotRelayed).
Rehandshake Flow (Noise)
- Why: The legacy NACK recovery path was removed; recovery now relies on Noise session rehandshake after decrypt failure or desync.
- Manager:
NoiseSessionManagermanages per-peer sessions. - Pattern: On decrypt failure, proactively clear the local session and re-initiate a handshake. The peer accepts and replaces their session.
- Test:
IntegrationTests.testRehandshakeAfterDecryptionFailure- Corrupts ciphertext to induce a decrypt error.
- Calls
removeSession(for:)on the initiator’s manager beforeinitiateHandshake(with:)to avoidalreadyEstablished. - Verifies encrypt/decrypt succeeds post-rehandshake.
Tips
- Determinism: Add small async delays only where handler installation/topology changes could race the first send.
- Scoping: Keep
autoFloodEnabledtoggled only within Integration tests; always reset intearDown()to avoid cross-test contamination. - Direct vs Relay: Private messages target a specific peer when adjacent; otherwise they are surfaced to neighbors for relay and, if known, also delivered to the target.
Quick Start
- Create nodes and connect them:
let svc = MockBLEService(); svc.myPeerID = "PEER1"svc.simulateConnectedPeer("PEER2")
- Observe messages:
svc.messageDeliveryHandler = { msg in /* asserts */ }
- Enable broadcast flooding for Integration suites only:
MockBLEService.autoFloodEnabled = true