Fix restore-path main↔bleQueue deadlock and courier drop amplification

A device froze permanently in a two-phone test. Debugger stacks showed an
ABBA deadlock: the main actor was in bleQueue.sync (delivery-ack send →
broadcastPacket → readLinkState) while bleQueue was in main.sync
(captureBluetoothStatus reading backgroundTimeRemaining). The load that
lined the two edges up came from a courier-drop amplification storm:
drop dedup was in-memory only while the outbox driving 120s re-deposits
is persisted, so every relaunch republished the same undelivered DM as a
fresh 24h relay drop and every gateway relaunch re-fetched the whole
backlog — ~20 copies of one DM delivered in 40ms, each triggering
decrypt + delivery + ack + handshake work.

Fixes, in rank order:
- Edge B (P0): captureBluetoothStatus no longer main.syncs from bleQueue;
  backgroundTimeRemaining is sampled on main and cached behind a lock.
  Invariant documented: bleQueue must NEVER sync-dispatch to main.
- Edge A (P0, defense in depth): sendDeliveryAck / sendReadReceipt /
  sendPrivateMessage / sendNoisePayload / triggerHandshake hop to
  messageQueue like sendMessage, so no main-actor call path reaches
  readLinkState's bleQueue.sync.
- Drop dedup (P1): publishedDropKeys and seenDropEventIDs persist across
  relaunches (new BridgeDropDedupStore, entries expire with the 24h
  NIP-40 drop window; wiped on panic) — one drop per message ID per 24h
  regardless of relaunch count.
- Receiver dedup (P1): openCourierEnvelope dedups on the inner private
  message ID before delivery, so a duplicate copy costs one decrypt and
  never re-delivers, re-acks, or re-triggers a handshake.
- Handshake gating (P2): queued acks initiate a Noise handshake only for
  reachable peers; mail from absent/rotated identities no longer turns
  each copy into a mesh-wide handshake flood (the ack stays queued and
  flushes when a session eventually establishes).
- Outbox (P3): re-enqueueing a queued message ID carries over its
  depositedCourierKeys so resends stop re-burning the same courier slots.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
jack
2026-07-09 17:46:59 +02:00
co-authored by Claude Fable 5
parent 304460ee83
commit d905452df2
11 changed files with 593 additions and 38 deletions
@@ -556,6 +556,70 @@ struct CourierEndToEndTests {
#expect(!stored)
}
/// Regression for the relaunch amplification storm: redundant copies of
/// one message arrive as *distinct* envelopes (every seal uses a fresh
/// ephemeral key), so only the inner message ID can dedup them. The first
/// copy delivers; duplicates must stop right after the decrypt no
/// second delivery, no second ack/handshake trigger downstream.
@Test func duplicateCourierCopiesDeliverInnerMessageOnce() async throws {
let alice = makeService()
let bob = makeService()
let bobDelegate = NoiseCaptureDelegate()
bob.delegate = bobDelegate
let bobKey = bob.noiseStaticPublicKeyData()
let first = try #require(alice.sealBridgeCourierEnvelope("once", messageID: "dup-1", recipientNoiseKey: bobKey))
let second = try #require(alice.sealBridgeCourierEnvelope("once", messageID: "dup-1", recipientNoiseKey: bobKey))
// Distinct seals: envelope-level dedup can never catch this pair.
#expect(first.ciphertext != second.ciphertext)
#expect(bob.openBridgedCourierEnvelope(first))
#expect(bob.openBridgedCourierEnvelope(second))
let delivered = await TestHelpers.waitUntil(
{ !bobDelegate.snapshot().isEmpty },
timeout: TestConstants.defaultTimeout
)
#expect(delivered)
// Give a duplicate delivery a chance to surface, then confirm the
// second copy never reached the delegate.
let duplicated = await TestHelpers.waitUntil(
{ bobDelegate.snapshot().count > 1 },
timeout: TestConstants.shortTimeout
)
#expect(!duplicated)
#expect(bobDelegate.snapshot().count == 1)
}
/// Acks for mail from absent senders (the usual couriered/bridged case)
/// queue for a future session instead of initiating a handshake
/// broadcast otherwise every duplicate copy of every drop turns into a
/// mesh-wide handshake flood at an identity that cannot answer.
@Test func deliveryAckForAbsentPeerQueuesWithoutHandshake() async throws {
let ble = makeService()
let outbound = PacketTap()
ble._test_onOutboundPacket = outbound.record
// Nobody by this ID on the mesh (not connected, not reachable).
ble.sendDeliveryAck(for: "msg-1", to: PeerID(str: "00000000000000ee"))
let initiated = await TestHelpers.waitUntil(
{ outbound.count(ofType: .noiseHandshake) > 0 },
timeout: TestConstants.shortTimeout
)
#expect(!initiated)
// Control: a peer that is actually around still gets the handshake.
let present = PeerID(str: "00000000000000ef")
ble._test_seedConnectedPeer(present, nickname: "present")
ble.sendDeliveryAck(for: "msg-2", to: present)
let initiatedForPresent = await TestHelpers.waitUntil(
{ outbound.count(ofType: .noiseHandshake) > 0 },
timeout: TestConstants.defaultTimeout
)
#expect(initiatedForPresent)
}
private func makeUnsignedAnnounce(from service: BLEService) throws -> BitchatPacket {
let announcement = AnnouncementPacket(
nickname: "Unsigned",