bitchat/docs/PUSH-TO-TALK-DESIGN.md

Smart Push-to-Talk (PTT) — Design

Status: Draft for review — no code yet Scope: Live voice bursts over the BLE mesh, in public chat and Noise DMs, with graceful degradation to the existing voice-note pipeline.

1. Goal and core idea

Today, holding the mic button records an AAC .m4a and ships it as a fileTransfer (0x22) after you release — the receiver hears nothing until the whole file arrives. PTT makes the same gesture live: encoded audio frames stream over the mesh while you speak, and nearby receivers hear you with sub-second delay, walkie-talkie style.

The "smart" part is that PTT is not a separate mode the user must choose. It is a delivery strategy, picked automatically per conversation:

Context	Delivery
DM, peer connected/reachable on mesh	Live stream (Noise-encrypted frames) + finalized voice note for reliability
DM, peer only reachable via Nostr	Existing voice-note recording only (no live; media doesn't ride Nostr today)
Public mesh chat	Live broadcast stream, best-effort, no file follow-up
Geohash (Nostr) channels	PTT unavailable (matches existing canSendMediaInCurrentContext media policy)

One gesture (hold mic), one mental model ("talk"), and the system degrades from live → reliable-note → unavailable based on what the transport can actually do.

Bandwidth reality check: the mesh moves ~15 KB/s per link (469 B fragments at 30 ms spacing); our voice codec needs ~2 KB/s. Live voice fits with a wide margin, even relayed.

2. What already exists (reused, not rebuilt)

• Capture UX: ContentComposerView.micButtonView already implements hold-to-record / release-to-send via DragGesture, backed by VoiceRecordingViewModel's state machine and VoiceRecorder (AAC-LC, 16 kHz mono, 16 kbps). Mic permission string is already in Info.plist.
• Reliable delivery: BitchatFilePacket TLV + fileTransfer (0x22) + fragmentation + transfer progress + ChatMediaTransferCoordinator.
• Playback: VoiceNotePlaybackController + VoiceNotePlaybackCoordinator (single active playback), WaveformView, VoiceNoteView.
• Transport: signed public packets, Noise sessions with typed inner payloads, RelayController flood control, MessageDeduplicator, MessageRouter.canDeliverPromptly.

New work is the streaming path: a frame encoder/packetizer, a wire format for bursts, a jitter-buffered receiver/player, relay policy, and the live-bubble UI.

3. Wire protocol

3.1 Burst framing (shared inner format)

A talk burst is a sequence of packets sharing an 8-byte random burstID:

[burstID: 8][seq: UInt16 BE][flags: 1][payload…]

flags bits:

• 0x01 START — payload is a header TLV: codec (1 B enum, 0x01 = AAC-LC/16kHz/mono), frame duration ms, batch size. Sent as seq 0 and re-attached (piggybacked TLV) every ~2 s so mid-burst joiners can sync.
• 0x02 END — payload: total data-packet count (UInt16), duration ms (UInt32). Lets receivers detect tail loss.
• 0x04 CANCELED — sender slid-to-cancel; receivers stop playback, discard buffered audio, and drop the bubble.
• 0x00 (data) — payload is N length-prefixed AAC frames: [len: UInt16 BE][ADTS-less raw AAC frame]…

Audio math: AAC-LC at 16 kHz has 1024-sample frames = 64 ms/frame ≈ 130 B at 16 kbps. A greedy packetizer batches frames up to a 210-byte burst-content budget (pttMaxBurstContentBytes), chosen so the Noise ciphertext (content + 1 inner-type byte + 16-byte tag) stays inside MessagePadding's 256-byte bucket — the whole directed packet is 288 bytes and voice never enters the fragment scheduler (which caps concurrent transfers at 2 and would starve file sends). At 16 kbps that works out to 1 frame/packet, ~15.6 pkt/s, ~5.3 KB/s wire for DMs; public bursts (unpadded, planned for phase 2) can batch 3 frames.

3.2 Public mesh: new MessageType.voiceFrame = 0x29

• Broadcast (no recipient), signed like public messages; unsigned or signature-mismatched frames are dropped on receive.
• No padding (add to BLEOutboundPacketPolicy alongside fileTransfer) — padding to the 512 block would push every packet over MTU into fragmentation.
• TTL: messageTTL (7) at origin; RelayController gets a voiceFrame case mirroring the fragment policy — dense clamp to 5, jitter 8–25 ms — so live audio floods like fragments, not like announces. Per-hop cost is ~10–40 ms; 3 hops stays comfortably inside the jitter buffer.
• Dedup: existing MessageDeduplicator (timestamp ms + seq make each packet unique).
• 0x29 is the next free code after nostrCarrier = 0x28; unknown types are ignored by older clients (iOS and Android), so rollout is compatible — old clients simply don't hear live bursts.

3.3 DM: new NoisePayloadType.voiceFrame = 0x08

• Inner payload = the same burst framing, wrapped in noiseEncrypted (0x11) directed packets — wire-indistinguishable from other DM traffic by size/type (0x04/0x05 are reserved per the comment in BitchatProtocol.swift; 0x08 is the first free slot after groupKeyUpdate = 0x07).
• Directed encrypted packets are already always-relayed by RelayController, so multi-hop DMs work unchanged.
• Requires an established session; PTT-live is only offered when noiseService.hasEstablishedSession (otherwise first hold triggers the normal handshake + falls back to voice-note for that burst).
• Fire-and-forget: no delivery acks, no retransmit for frames. Late audio is worthless; reliability comes from the finalized note (§5).

3.4 Known traffic-analysis tradeoff

A steady ~8 pkt/s cadence for the duration of a burst is a timing fingerprint even under Noise (observers can infer "someone is speaking to someone", not content). This is inherent to live voice; the doc records it as accepted for v1. Block padding is skipped for voiceFrame (size classes would leak little beyond what cadence already does).

4. Sender pipeline

VoiceRecorder (AVAudioRecorder → file) can't stream, so PTT capture uses a parallel path in a new PTTStreamEncoder actor:

AVAudioEngine input tap (native format)
  → PTTInputResampler → 16 kHz mono PCM
  → PTTFrameEncoder (AVAudioConverter) → AAC-LC frames → packetizer → BLEService.sendVoiceFrame
  → the same PCM is simultaneously written to an AVAudioFile .m4a (identical settings
    to VoiceRecorder), so finalization needs no remux step

• On release: emit END, then hand the finalized .m4a to the existing sendVoiceNote flow. The note's fileName embeds the burst ID (voice_<burstID hex>.m4a) — that links note↔burst with zero changes to BitchatFilePacket/Android interop.
• On slide-to-cancel: stop tap, emit CANCELED, delete local file, skip finalization. UX note: unlike voice notes, live audio already played on the far side cannot be unsent — the recording UI must make "you are live" unmistakable (see §7).
• Caps: max burst 120 s (matching the voice-note recorder's cap), exactly one outbound burst at a time.
• Audio session: reuse VoiceRecorder's .playAndRecord config; add .duckOthers.

5. Receiver pipeline

New VoiceBurstAssembler (keyed by sender + burstID) feeding a PTTBurstPlayer:

• Jitter buffer: start playback after 350 ms of buffered audio or 500 ms wall-clock, whichever first. Frames decode via AVAudioConverter → AVAudioPlayerNode.
• Loss handling: gap in seq → insert silence for the missing frames (64 ms each) and keep going. No PLC in v1; at these frame sizes brief dropouts are acceptable.
• Burst end: on END, or 3 s with no frames (talker walked out of range).
• Persistence: frames append to an incoming ADTS .aac file (already an allowed MimeType), so every burst becomes a replayable voice-note bubble containing whatever was captured — even a partial one.
• Dedup with the finalized note (DM): when a fileTransfer arrives whose fileName carries a burstID we already assembled, it silently replaces the partial file behind the existing bubble (no new message row, no second notification). Receivers that heard everything live just get a lossless copy.
• Resource caps: ≤ 8 concurrent assemblies, ≤ 256 KB per burst (60 s × 2 KB/s + slack), 30 s stale cleanup, and drop inbound frames beyond ~2× realtime per sender (spam/flood guard).

6. Playback policy — when does it actually make sound?

Auto-playing strangers' audio is the fastest way to make this feature hated. Rules:

1. Live autoplay only when all hold: app foregrounded, the burst's conversation is the one currently on screen, and the "live voice messages" toggle is on (app-level, in the app-info sheet, default on; per-conversation overrides are a v2 refinement). The same toggle gates live sending — off means voice behaves exactly like classic notes in both directions.
2. Otherwise the burst appears as a live bubble: pulsing waveform + LIVE badge + sender name; tapping it joins playback at the live edge. When the burst ends it becomes a normal voice-note bubble.
3. One voice at a time: route through VoiceNotePlaybackCoordinator. If two people talk simultaneously, the first burst holds the floor; the second shows as a tappable live bubble. No mixing in v1.
4. Notifications: a live burst in a non-focused DM fires the normal message notification once (at START), not per-frame.

Floor courtesy (public mesh): while someone else's burst is live in the current channel, the mic button tints "busy" with the talker's name. Holding it still works — a decentralized mesh has no floor arbiter, and rejecting sends would desync under partitions — but the UI discourages talk-over. Hard floor control (token passing) is explicitly out of scope.

7. UX

• Same gesture: hold mic = talk. When the live path is active, the recording HUD shows a red pulsing LIVE treatment (vs. the current neutral recording UI) so the sender knows audio is leaving in real time, not on release. When live isn't available (Nostr-only peer, no session yet), the HUD looks like today's and behavior is unchanged.
• Slide-to-cancel keeps working in both modes, with the §4 caveat surfaced simply: cancel stops and discards; it can't unplay what was heard.
• VoiceOver: mirror the existing accessibilityAction toggle-record pattern on the mic button.
• Foreground-only in v1: no audio UIBackgroundMode (App Store review + battery implications). If the app backgrounds mid-burst, capture stops cleanly with END. Background listen/talk is a v2 candidate.

8. Latency budget (1 hop, DM)

Stage	Cost
Frame accumulation (1 × 64 ms)	64 ms
Encode + packetize	~5 ms
BLE write + delivery	30–60 ms
Jitter buffer	350 ms
Mouth-to-ear	~470 ms

Each relay hop adds ~10–40 ms jitter + radio time. 2–3 hops stays under ~800 ms — solidly in walkie-talkie territory (commercial PoC apps run 300 ms–1 s+).

9. Security & privacy summary

• Public frames signed with the existing packet signature; verified against the sender's announce before decode. Unsigned → dropped.
• DM frames ride inside Noise; content confidentiality/integrity as any DM.
• Codec input is validated by frame-length prefixes and total-size caps before touching AVAudioConverter (malformed frames dropped, assembly aborted over cap).
• Timing fingerprint accepted per §3.4. No PTT in geohash channels (would put voice on public Nostr relays).
• Mic capture only while the button is held; recording state is always visible.

10. New components & touch points

Piece	Where
PTTStreamEncoder (tap → AAC → packets)	bitchat/Features/voice/
PTTBurstPlayer (jitter buffer → decode → engine)	bitchat/Features/voice/
VoiceBurstFramer / VoiceBurstAssembler (wire encode/decode, caps)	bitchat/Protocols/ + bitchat/Services/PTT/
MessageType.voiceFrame = 0x29	localPackages/BitFoundation/.../MessageType.swift + BLEReceivePipeline / BLEService.handleReceivedPacket
NoisePayloadType.voiceFrame = 0x08	BitchatProtocol.swift + ChatTransportEventCoordinator dispatch
Relay policy case	RelayController (fragment-like clamp/jitter)
No-padding rule	BLEOutboundPacketPolicy
Pacing/cap constants	TransportConfig
Live bubble + LIVE HUD + busy mic tint + settings toggle	VoiceNoteView/MediaMessageView, ContentComposerView, conversation settings
Delivery-mode selection	ChatViewModel / ChatMediaTransferCoordinator (reuse isPeerConnected / canDeliverPromptly / hasEstablishedSession)

11. Phasing

1. Phase 1 — DM live (highest value, lowest blast radius): encoder, framer, Noise inner type, assembler/player, live bubble, finalize-as-note dedup. Single-hop DMs are the dominant real-world case.
2. Phase 2 — public mesh: voiceFrame 0x29, signing/verification, relay policy, floor-courtesy UI, autoplay defaults.
3. Phase 3 (v2 candidates): background audio mode, mid-burst repair requests (piggyback on REQUEST_SYNC), talk-over mixing, AAC-ELD low-delay profile, dedicated walkie-mode screen, media-over-Nostr (unlocks live-ish PTT for internet peers), Android protocol spec sync.

12. Testing

• Unit: framer round-trip (START/data/END/CANCELED, seq gaps, oversize frames), assembler caps + stale cleanup + burstID note dedup, packetizer batch sizing vs. MTU (assert no fragmentation), relay-policy clamps.
• Integration: two-simulator loopback via the existing mesh test harness; loss injection (drop every Nth frame) → verify silence-fill + partial-file persistence.
• Device: two-phone live DM (latency measurement vs. §8 budget), three-phone relay chain, talk-over behavior, cancel semantics, mic-permission-denied path.