mirror of
https://github.com/permissionlesstech/bitchat.git
synced 2026-07-24 22:45:19 +00:00
Live push-to-talk voice for DMs (streams while you talk, voice note as fallback) (#1403)
* Live push-to-talk voice for DMs: stream while you talk, voice note as fallback Holding the mic in a DM now streams AAC frames live over the Noise session (walkie-talkie style, ~0.5s mouth-to-ear at one hop) while recording the same audio as a normal voice note. On release the note ships through the existing fileTransfer pipeline; receivers that heard the live stream absorb it silently into the same bubble (matched by the burst ID embedded in the file name), so reliability comes for free and nobody sees duplicates. Protocol: - NoisePayloadType.voiceFrame = 0x08 carrying VoiceBurstPacket (burstID + seq + START/data/END/CANCELED, length-prefixed AAC frames) - 210-byte burst-content budget keeps each Noise packet inside the 256-byte padding bucket: one BLE frame, never the fragment scheduler - fire-and-forget: frames are dropped (never queued) without an established session; live is only offered when the peer is mesh-reachable Receive: - ChatLiveVoiceCoordinator assembles bursts (jitter-ordered, 0.5s gap skip, 3s idle end, flood/size caps), persists progressively as ADTS .aac so even a partial burst is a replayable bubble - live autoplay only when the conversation is on screen, app active, and the new app-info "live voice messages" toggle is on (also gates live sending) - one-playback-at-a-time via a shared ExclusivePlayback slot Capture: - PTTCaptureEngine taps AVAudioEngine, dual-encodes: live AAC frames + the finalized .m4a (same 16kHz/mono/16kbps settings as VoiceRecorder) - VoiceRecordingViewModel now drives a pluggable VoiceCaptureSession; the composer HUD shows a pulsing LIVE treatment when streaming Includes the push-to-talk design doc, 6 new localization keys across all 29 locales, and unit tests for framing, packetizer budget, ADTS output, codec round-trip, and the assembly/absorb lifecycle. Public-mesh PTT (MessageType 0x29) lands separately on top of this. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * PTT follow-ups from review + field test: peer-ID normalization, toggle gates inbound, drop-path diagnostics Codex review fixes (#1403): - makeVoiceCaptureSession normalizes the selected peer with toShort() before the reachability/session checks and binds the send target to that same routing ID — a conversation selected under the stable 64-hex Noise key no longer silently falls back to a classic note while the short-ID session is established - the live-voice toggle now gates inbound bursts too: off means classic-notes-only in both directions (no live bubble, partial file, or early notification; the finalized note still arrives), with a test Field-test diagnostics (first device run: DM frames decrypted but no bubble appeared, with no log evidence of which guard dropped them): - coordinator logs undecodable frames (size + hex prefix) and blocked drops - makeAssembly logs directory/file-handle failures instead of returning nil silently - PTTLiveVoiceSession logs capture start and finish (packet/frame/duration counts); PTTCaptureEngine logs engine start success/failure with the input format; BLEService.sendVoiceFrame logs no-session drops Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Fix iPhone live-capture failure: dead input unit (AURemoteIO -10851, 0 Hz) Field testing showed the phone's live capture failing at mic enable with AURemoteIO -10851 and an input format of 0 Hz / 2 ch — an input unit bound to an earlier (playback-only or settling) audio session. The Mac, which has no session lifecycle, captured fine, which is why public bursts from the Mac worked while phone-side sends degraded from working (first hold) to sporadic to dead across holds. Three layers of defense: - PTTCaptureEngine recreates its AVAudioEngine on every start(), after the session is configured, so the input unit binds to the session that is active now; a dead input (0 Hz or 0 channels) is now a distinct, logged error instead of a silent setup failure - PTTLiveVoiceSession retries the capture start once after a 150 ms route-settle pause - VoiceRecordingViewModel falls back to the classic VoiceRecorder within the same hold if the live engine still cannot start — a route glitch now costs the live stream, never the voice note Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: jack <jackjackbits@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
jack
co-authored by
jack
Claude Fable 5
jack
Claude Fable 5
parent
6886035632
commit
eacd8f0750
@@ -150,6 +150,17 @@ final class ConversationUIModel: ObservableObject {
|
||||
chatViewModel.sendVoiceNote(at: url)
|
||||
}
|
||||
|
||||
/// Capture backend for the mic gesture: live PTT when the current DM
|
||||
/// peer can hear it now, classic voice note otherwise.
|
||||
func makeVoiceCaptureSession() -> VoiceCaptureSession {
|
||||
chatViewModel.makeVoiceCaptureSession()
|
||||
}
|
||||
|
||||
/// Whether this message is a live voice burst still streaming in.
|
||||
func isLiveVoiceMessage(_ message: BitchatMessage) -> Bool {
|
||||
chatViewModel.liveVoiceCoordinator.isLiveVoiceMessage(message)
|
||||
}
|
||||
|
||||
func cancelMediaSend(messageID: String) {
|
||||
chatViewModel.cancelMediaSend(messageID: messageID)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,175 @@
|
||||
//
|
||||
// PTTAudioCodec.swift
|
||||
// bitchat
|
||||
//
|
||||
// This is free and unencumbered software released into the public domain.
|
||||
// For more information, see <https://unlicense.org>
|
||||
//
|
||||
|
||||
import AVFoundation
|
||||
import BitLogger
|
||||
import Foundation
|
||||
|
||||
/// Streaming PCM -> AAC-LC encoder for live voice. Stateful (the AAC encoder
|
||||
/// carries a bit reservoir across frames); one instance per burst.
|
||||
/// Not thread-safe — confine to one queue.
|
||||
final class PTTFrameEncoder {
|
||||
private let converter: AVAudioConverter
|
||||
private var pendingInput: [AVAudioPCMBuffer] = []
|
||||
|
||||
init?() {
|
||||
guard let pcm = PTTAudioFormat.pcmFormat,
|
||||
let aac = PTTAudioFormat.aacFormat,
|
||||
let converter = AVAudioConverter(from: pcm, to: aac)
|
||||
else { return nil }
|
||||
converter.bitRate = PTTAudioFormat.bitRate
|
||||
self.converter = converter
|
||||
}
|
||||
|
||||
/// Feeds PCM (16 kHz mono float) and returns every complete AAC frame the
|
||||
/// encoder produced. Frames come out ~130 bytes each at 16 kbps.
|
||||
func encode(_ buffer: AVAudioPCMBuffer) -> [Data] {
|
||||
pendingInput.append(buffer)
|
||||
return drainConverter()
|
||||
}
|
||||
|
||||
private func drainConverter() -> [Data] {
|
||||
var frames: [Data] = []
|
||||
while true {
|
||||
let output = AVAudioCompressedBuffer(
|
||||
format: converter.outputFormat,
|
||||
packetCapacity: 8,
|
||||
maximumPacketSize: max(converter.maximumOutputPacketSize, 1)
|
||||
)
|
||||
var error: NSError?
|
||||
let status = converter.convert(to: output, error: &error) { [weak self] _, outStatus in
|
||||
guard let self, let next = self.pendingInput.first else {
|
||||
outStatus.pointee = .noDataNow
|
||||
return nil
|
||||
}
|
||||
self.pendingInput.removeFirst()
|
||||
outStatus.pointee = .haveData
|
||||
return next
|
||||
}
|
||||
if status == .error {
|
||||
SecureLogger.error("PTT encode failed: \(error?.localizedDescription ?? "unknown")", category: .session)
|
||||
return frames
|
||||
}
|
||||
frames.append(contentsOf: Self.extractPackets(from: output))
|
||||
// .haveData means the output buffer filled and more may be ready;
|
||||
// anything else means the converter wants more input.
|
||||
if status != .haveData { return frames }
|
||||
}
|
||||
}
|
||||
|
||||
private static func extractPackets(from buffer: AVAudioCompressedBuffer) -> [Data] {
|
||||
guard buffer.packetCount > 0, let descriptions = buffer.packetDescriptions else { return [] }
|
||||
var frames: [Data] = []
|
||||
frames.reserveCapacity(Int(buffer.packetCount))
|
||||
for index in 0..<Int(buffer.packetCount) {
|
||||
let description = descriptions[index]
|
||||
guard description.mDataByteSize > 0 else { continue }
|
||||
let start = buffer.data.advanced(by: Int(description.mStartOffset))
|
||||
frames.append(Data(bytes: start, count: Int(description.mDataByteSize)))
|
||||
}
|
||||
return frames
|
||||
}
|
||||
}
|
||||
|
||||
/// Streaming AAC-LC -> PCM decoder for live voice. Stateful; one instance per
|
||||
/// inbound burst. Not thread-safe — confine to one queue/actor.
|
||||
final class PTTFrameDecoder {
|
||||
private let converter: AVAudioConverter
|
||||
private let pcmFormat: AVAudioFormat
|
||||
private let aacFormat: AVAudioFormat
|
||||
|
||||
init?() {
|
||||
guard let pcm = PTTAudioFormat.pcmFormat,
|
||||
let aac = PTTAudioFormat.aacFormat,
|
||||
let converter = AVAudioConverter(from: aac, to: pcm)
|
||||
else { return nil }
|
||||
self.converter = converter
|
||||
self.pcmFormat = pcm
|
||||
self.aacFormat = aac
|
||||
}
|
||||
|
||||
/// Decodes one raw AAC frame to PCM. Returns nil for malformed input or
|
||||
/// while the decoder is still priming (the first frame of a stream).
|
||||
func decode(_ frame: Data) -> AVAudioPCMBuffer? {
|
||||
guard !frame.isEmpty, frame.count <= 8 * 1024 else { return nil }
|
||||
|
||||
let input = AVAudioCompressedBuffer(format: aacFormat, packetCapacity: 1, maximumPacketSize: frame.count)
|
||||
frame.withUnsafeBytes { raw in
|
||||
guard let base = raw.baseAddress else { return }
|
||||
input.data.copyMemory(from: base, byteCount: frame.count)
|
||||
}
|
||||
input.byteLength = UInt32(frame.count)
|
||||
input.packetCount = 1
|
||||
input.packetDescriptions?.pointee = AudioStreamPacketDescription(
|
||||
mStartOffset: 0,
|
||||
mVariableFramesInPacket: 0,
|
||||
mDataByteSize: UInt32(frame.count)
|
||||
)
|
||||
|
||||
guard let output = AVAudioPCMBuffer(
|
||||
pcmFormat: pcmFormat,
|
||||
frameCapacity: PTTAudioFormat.samplesPerFrame * 2
|
||||
) else { return nil }
|
||||
|
||||
var consumed = false
|
||||
var error: NSError?
|
||||
let status = converter.convert(to: output, error: &error) { _, outStatus in
|
||||
if consumed {
|
||||
outStatus.pointee = .noDataNow
|
||||
return nil
|
||||
}
|
||||
consumed = true
|
||||
outStatus.pointee = .haveData
|
||||
return input
|
||||
}
|
||||
guard status != .error else {
|
||||
SecureLogger.debug("PTT decode failed: \(error?.localizedDescription ?? "unknown")", category: .session)
|
||||
return nil
|
||||
}
|
||||
return output.frameLength > 0 ? output : nil
|
||||
}
|
||||
}
|
||||
|
||||
/// Sample-rate/channel converter from the microphone's native format to the
|
||||
/// 16 kHz mono processing format. Stateful; not thread-safe.
|
||||
final class PTTInputResampler {
|
||||
private let converter: AVAudioConverter
|
||||
private let outputFormat: AVAudioFormat
|
||||
private let ratio: Double
|
||||
|
||||
init?(inputFormat: AVAudioFormat) {
|
||||
guard let pcm = PTTAudioFormat.pcmFormat,
|
||||
let converter = AVAudioConverter(from: inputFormat, to: pcm)
|
||||
else { return nil }
|
||||
self.converter = converter
|
||||
self.outputFormat = pcm
|
||||
self.ratio = PTTAudioFormat.sampleRate / inputFormat.sampleRate
|
||||
}
|
||||
|
||||
func resample(_ buffer: AVAudioPCMBuffer) -> AVAudioPCMBuffer? {
|
||||
let capacity = AVAudioFrameCount(Double(buffer.frameLength) * ratio) + 64
|
||||
guard let output = AVAudioPCMBuffer(pcmFormat: outputFormat, frameCapacity: capacity) else { return nil }
|
||||
|
||||
var consumed = false
|
||||
var error: NSError?
|
||||
let status = converter.convert(to: output, error: &error) { _, outStatus in
|
||||
if consumed {
|
||||
outStatus.pointee = .noDataNow
|
||||
return nil
|
||||
}
|
||||
consumed = true
|
||||
outStatus.pointee = .haveData
|
||||
return buffer
|
||||
}
|
||||
guard status != .error else {
|
||||
SecureLogger.debug("PTT resample failed: \(error?.localizedDescription ?? "unknown")", category: .session)
|
||||
return nil
|
||||
}
|
||||
return output.frameLength > 0 ? output : nil
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
//
|
||||
// PTTAudioFormat.swift
|
||||
// bitchat
|
||||
//
|
||||
// This is free and unencumbered software released into the public domain.
|
||||
// For more information, see <https://unlicense.org>
|
||||
//
|
||||
|
||||
import AVFoundation
|
||||
import Foundation
|
||||
|
||||
/// Shared audio parameters for live push-to-talk: AAC-LC, 16 kHz, mono,
|
||||
/// ~16 kbps — deliberately identical to `VoiceRecorder`'s voice-note settings
|
||||
/// so a burst's finalized `.m4a` and its live frames sound the same.
|
||||
enum PTTAudioFormat {
|
||||
static let sampleRate: Double = 16_000
|
||||
static let channelCount: AVAudioChannelCount = 1
|
||||
static let bitRate = 16_000
|
||||
/// AAC-LC frame size is fixed by the codec: 1024 samples = 64 ms at 16 kHz.
|
||||
static let samplesPerFrame: AVAudioFrameCount = 1024
|
||||
static var frameDuration: TimeInterval { Double(samplesPerFrame) / sampleRate }
|
||||
|
||||
/// Uncompressed processing format (deinterleaved float PCM).
|
||||
static var pcmFormat: AVAudioFormat? {
|
||||
AVAudioFormat(standardFormatWithSampleRate: sampleRate, channels: channelCount)
|
||||
}
|
||||
|
||||
/// Compressed wire format.
|
||||
static var aacFormat: AVAudioFormat? {
|
||||
var description = AudioStreamBasicDescription(
|
||||
mSampleRate: sampleRate,
|
||||
mFormatID: kAudioFormatMPEG4AAC,
|
||||
mFormatFlags: 0,
|
||||
mBytesPerPacket: 0,
|
||||
mFramesPerPacket: samplesPerFrame,
|
||||
mBytesPerFrame: 0,
|
||||
mChannelsPerFrame: channelCount,
|
||||
mBitsPerChannel: 0,
|
||||
mReserved: 0
|
||||
)
|
||||
return AVAudioFormat(streamDescription: &description)
|
||||
}
|
||||
|
||||
/// Voice-note container settings for the finalized `.m4a`, mirroring
|
||||
/// `VoiceRecorder.startRecording()`.
|
||||
static var voiceNoteFileSettings: [String: Any] {
|
||||
[
|
||||
AVFormatIDKey: kAudioFormatMPEG4AAC,
|
||||
AVSampleRateKey: sampleRate,
|
||||
AVNumberOfChannelsKey: Int(channelCount),
|
||||
AVEncoderBitRateKey: bitRate
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
/// Builds ADTS-framed AAC so a receiver can persist a burst progressively:
|
||||
/// unlike `.m4a` (whose moov atom only exists after close), an ADTS `.aac`
|
||||
/// stream is playable at any prefix — a partially received burst is still a
|
||||
/// replayable voice note.
|
||||
enum ADTSFramer {
|
||||
private static let headerSize = 7
|
||||
/// MPEG-4 sampling frequency index for 16 kHz.
|
||||
private static let samplingFrequencyIndex: UInt8 = 8
|
||||
private static let channelConfiguration: UInt8 = 1
|
||||
|
||||
/// Wraps one raw AAC-LC frame in an ADTS header.
|
||||
static func frame(_ aacFrame: Data) -> Data {
|
||||
let frameLength = aacFrame.count + headerSize
|
||||
var data = Data(capacity: frameLength)
|
||||
// Syncword 0xFFF, MPEG-4, layer 00, no CRC.
|
||||
data.append(0xFF)
|
||||
data.append(0xF1)
|
||||
// Profile AAC-LC (audio object type 2 -> bits 01), frequency index,
|
||||
// private bit 0, channel config high bit.
|
||||
data.append((0b01 << 6) | (samplingFrequencyIndex << 2) | ((channelConfiguration >> 2) & 0x1))
|
||||
data.append(((channelConfiguration & 0x3) << 6) | UInt8((frameLength >> 11) & 0x3))
|
||||
data.append(UInt8((frameLength >> 3) & 0xFF))
|
||||
data.append(UInt8((frameLength & 0x7) << 5) | 0x1F)
|
||||
// Buffer fullness 0x7FF (VBR), one AAC frame per ADTS frame.
|
||||
data.append(0xFC)
|
||||
data.append(aacFrame)
|
||||
return data
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,144 @@
|
||||
//
|
||||
// PTTBurstPlayer.swift
|
||||
// bitchat
|
||||
//
|
||||
// This is free and unencumbered software released into the public domain.
|
||||
// For more information, see <https://unlicense.org>
|
||||
//
|
||||
|
||||
import AVFoundation
|
||||
import BitLogger
|
||||
import Foundation
|
||||
|
||||
/// Plays one inbound live voice burst with a small jitter buffer.
|
||||
///
|
||||
/// Frames are decoded and scheduled back-to-back on an `AVAudioPlayerNode`;
|
||||
/// an underrun (missing/late packets) simply pauses output until the next
|
||||
/// buffer arrives, which self-heals timing without explicit silence
|
||||
/// insertion. Playback starts once `TransportConfig.pttJitterBufferSeconds`
|
||||
/// of audio is queued or `pttJitterDeadlineSeconds` has elapsed.
|
||||
@MainActor
|
||||
final class PTTBurstPlayer {
|
||||
private let engine = AVAudioEngine()
|
||||
private let node = AVAudioPlayerNode()
|
||||
private let decoder: PTTFrameDecoder
|
||||
|
||||
private var queuedBuffers: [AVAudioPCMBuffer] = []
|
||||
private var queuedDuration: TimeInterval = 0
|
||||
private var scheduledCount = 0
|
||||
private var engineStarted = false
|
||||
private var finished = false
|
||||
private var stopped = false
|
||||
private var deadlineTask: Task<Void, Never>?
|
||||
|
||||
private(set) var isPlaying = false
|
||||
|
||||
init?() {
|
||||
guard let format = PTTAudioFormat.pcmFormat, let decoder = PTTFrameDecoder() else { return nil }
|
||||
self.decoder = decoder
|
||||
engine.attach(node)
|
||||
engine.connect(node, to: engine.mainMixerNode, format: format)
|
||||
|
||||
deadlineTask = Task { [weak self] in
|
||||
try? await Task.sleep(nanoseconds: UInt64(TransportConfig.pttJitterDeadlineSeconds * 1_000_000_000))
|
||||
self?.startIfReady(force: true)
|
||||
}
|
||||
}
|
||||
|
||||
/// Decodes and queues frames (in burst order). Starts playback when the
|
||||
/// jitter buffer fills.
|
||||
func enqueue(_ frames: [Data]) {
|
||||
guard !stopped else { return }
|
||||
for frame in frames {
|
||||
guard let pcm = decoder.decode(frame) else { continue }
|
||||
if engineStarted {
|
||||
schedule(pcm)
|
||||
} else {
|
||||
queuedBuffers.append(pcm)
|
||||
queuedDuration += Double(pcm.frameLength) / PTTAudioFormat.sampleRate
|
||||
}
|
||||
}
|
||||
startIfReady(force: false)
|
||||
}
|
||||
|
||||
/// The burst ended: stop once everything scheduled has played out.
|
||||
func finishAfterDrain() {
|
||||
finished = true
|
||||
stopIfDrained()
|
||||
}
|
||||
|
||||
/// Immediate stop (cancel, another playback taking over, teardown).
|
||||
func stop() {
|
||||
guard !stopped else { return }
|
||||
stopped = true
|
||||
deadlineTask?.cancel()
|
||||
queuedBuffers = []
|
||||
if engineStarted {
|
||||
node.stop()
|
||||
engine.stop()
|
||||
}
|
||||
isPlaying = false
|
||||
VoiceNotePlaybackCoordinator.shared.deactivate(self)
|
||||
}
|
||||
|
||||
private func startIfReady(force: Bool) {
|
||||
guard !engineStarted, !stopped, !queuedBuffers.isEmpty else { return }
|
||||
guard force || queuedDuration >= TransportConfig.pttJitterBufferSeconds else { return }
|
||||
|
||||
#if os(iOS)
|
||||
do {
|
||||
let session = AVAudioSession.sharedInstance()
|
||||
try session.setCategory(.playback, mode: .spokenAudio, options: [.mixWithOthers])
|
||||
try session.setActive(true, options: [])
|
||||
} catch {
|
||||
SecureLogger.error("PTT playback session activation failed: \(error)", category: .session)
|
||||
}
|
||||
#endif
|
||||
|
||||
engine.prepare()
|
||||
do {
|
||||
try engine.start()
|
||||
} catch {
|
||||
SecureLogger.error("PTT playback engine failed to start: \(error)", category: .session)
|
||||
stopped = true
|
||||
return
|
||||
}
|
||||
engineStarted = true
|
||||
isPlaying = true
|
||||
VoiceNotePlaybackCoordinator.shared.activate(self)
|
||||
node.play()
|
||||
|
||||
let buffered = queuedBuffers
|
||||
queuedBuffers = []
|
||||
queuedDuration = 0
|
||||
for buffer in buffered {
|
||||
schedule(buffer)
|
||||
}
|
||||
}
|
||||
|
||||
private func schedule(_ buffer: AVAudioPCMBuffer) {
|
||||
scheduledCount += 1
|
||||
node.scheduleBuffer(buffer) { [weak self] in
|
||||
Task { @MainActor [weak self] in
|
||||
guard let self else { return }
|
||||
self.scheduledCount -= 1
|
||||
self.stopIfDrained()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func stopIfDrained() {
|
||||
guard finished, scheduledCount <= 0 else { return }
|
||||
stop()
|
||||
}
|
||||
}
|
||||
|
||||
extension PTTBurstPlayer: ExclusivePlayback {
|
||||
/// A live stream can't meaningfully pause; yielding the floor stops it.
|
||||
/// The burst keeps assembling to file, so nothing is lost.
|
||||
nonisolated func pauseForExclusivity() {
|
||||
Task { @MainActor [weak self] in
|
||||
self?.stop()
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,171 @@
|
||||
//
|
||||
// PTTCaptureEngine.swift
|
||||
// bitchat
|
||||
//
|
||||
// This is free and unencumbered software released into the public domain.
|
||||
// For more information, see <https://unlicense.org>
|
||||
//
|
||||
|
||||
import AVFoundation
|
||||
import BitLogger
|
||||
import Foundation
|
||||
|
||||
/// Captures microphone audio for a live push-to-talk burst, producing both:
|
||||
/// - live AAC frames via `onFrames` (called on the capture queue), and
|
||||
/// - a finalized `.m4a` voice note on `stop()` — the same artifact
|
||||
/// `VoiceRecorder` produces, so the existing voice-note send pipeline
|
||||
/// handles delivery to receivers that missed the live stream.
|
||||
final class PTTCaptureEngine {
|
||||
/// Hard cap matching `VoiceRecorder.maxRecordingDuration`: past it the
|
||||
/// engine keeps running (the UI owns the gesture) but stops encoding.
|
||||
private static let maxCaptureDuration: TimeInterval = 120
|
||||
|
||||
/// Recreated on every `start()`: an engine whose input unit was
|
||||
/// instantiated against an earlier (playback-only or inactive) audio
|
||||
/// session keeps reporting a dead 0 Hz / 2 ch input format and fails to
|
||||
/// enable the mic (AURemoteIO -10851, observed on iPhone field tests).
|
||||
private var engine = AVAudioEngine()
|
||||
private let queue = DispatchQueue(label: "chat.bitchat.ptt.capture", qos: .userInitiated)
|
||||
|
||||
// Capture-queue-confined state.
|
||||
private var resampler: PTTInputResampler?
|
||||
private var encoder: PTTFrameEncoder?
|
||||
private var file: AVAudioFile?
|
||||
private var fileURL: URL?
|
||||
private var encodedFrameCount = 0
|
||||
private var running = false
|
||||
private var captureStart = Date()
|
||||
|
||||
/// Called on the capture queue with each batch of encoded AAC frames.
|
||||
var onFrames: (([Data]) -> Void)?
|
||||
|
||||
enum CaptureError: Error {
|
||||
case inputUnavailable
|
||||
case audioSetupFailed
|
||||
}
|
||||
|
||||
func start(outputURL: URL) throws {
|
||||
#if os(iOS)
|
||||
try Self.configureAudioSession()
|
||||
#endif
|
||||
|
||||
// Fresh engine per capture so its input unit binds to the session
|
||||
// that is active *now* (see `engine` doc comment).
|
||||
engine = AVAudioEngine()
|
||||
let inputFormat = engine.inputNode.outputFormat(forBus: 0)
|
||||
guard inputFormat.sampleRate > 0, inputFormat.channelCount > 0 else {
|
||||
SecureLogger.error("PTT: capture input unavailable (input reports \(Int(inputFormat.sampleRate)) Hz, \(inputFormat.channelCount) ch)", category: .session)
|
||||
throw CaptureError.inputUnavailable
|
||||
}
|
||||
guard let resampler = PTTInputResampler(inputFormat: inputFormat),
|
||||
let encoder = PTTFrameEncoder(),
|
||||
let pcmFormat = PTTAudioFormat.pcmFormat
|
||||
else { throw CaptureError.audioSetupFailed }
|
||||
|
||||
let file = try AVAudioFile(
|
||||
forWriting: outputURL,
|
||||
settings: PTTAudioFormat.voiceNoteFileSettings,
|
||||
commonFormat: pcmFormat.commonFormat,
|
||||
interleaved: pcmFormat.isInterleaved
|
||||
)
|
||||
|
||||
queue.sync {
|
||||
self.resampler = resampler
|
||||
self.encoder = encoder
|
||||
self.file = file
|
||||
self.fileURL = outputURL
|
||||
self.encodedFrameCount = 0
|
||||
self.captureStart = Date()
|
||||
self.running = true
|
||||
}
|
||||
|
||||
engine.inputNode.installTap(onBus: 0, bufferSize: 4096, format: inputFormat) { [weak self] buffer, _ in
|
||||
self?.queue.async { self?.process(buffer) }
|
||||
}
|
||||
engine.prepare()
|
||||
do {
|
||||
try engine.start()
|
||||
} catch {
|
||||
SecureLogger.error("PTT: capture engine failed to start (input: \(Int(inputFormat.sampleRate)) Hz, \(inputFormat.channelCount) ch): \(error)", category: .session)
|
||||
engine.inputNode.removeTap(onBus: 0)
|
||||
queue.sync { self.teardown(deleteFile: true) }
|
||||
throw error
|
||||
}
|
||||
SecureLogger.info("PTT: capture engine running (input: \(Int(inputFormat.sampleRate)) Hz, \(inputFormat.channelCount) ch)", category: .session)
|
||||
}
|
||||
|
||||
/// Stops capture and finalizes the `.m4a`. Returns the file URL and the
|
||||
/// number of encoded AAC frames (each `PTTAudioFormat.frameDuration` long).
|
||||
func stop() -> (url: URL?, encodedFrames: Int) {
|
||||
engine.inputNode.removeTap(onBus: 0)
|
||||
engine.stop()
|
||||
let result: (URL?, Int) = queue.sync {
|
||||
let url = fileURL
|
||||
let frames = encodedFrameCount
|
||||
teardown(deleteFile: false)
|
||||
return (url, frames)
|
||||
}
|
||||
#if os(iOS)
|
||||
Self.deactivateAudioSession()
|
||||
#endif
|
||||
return result
|
||||
}
|
||||
|
||||
func cancel() {
|
||||
engine.inputNode.removeTap(onBus: 0)
|
||||
engine.stop()
|
||||
queue.sync { teardown(deleteFile: true) }
|
||||
#if os(iOS)
|
||||
Self.deactivateAudioSession()
|
||||
#endif
|
||||
}
|
||||
|
||||
// MARK: - Capture queue
|
||||
|
||||
private func process(_ buffer: AVAudioPCMBuffer) {
|
||||
guard running,
|
||||
Date().timeIntervalSince(captureStart) < Self.maxCaptureDuration,
|
||||
let resampled = resampler?.resample(buffer)
|
||||
else { return }
|
||||
|
||||
do {
|
||||
try file?.write(from: resampled)
|
||||
} catch {
|
||||
SecureLogger.error("PTT capture file write failed: \(error)", category: .session)
|
||||
}
|
||||
|
||||
guard let frames = encoder?.encode(resampled), !frames.isEmpty else { return }
|
||||
encodedFrameCount += frames.count
|
||||
onFrames?(frames)
|
||||
}
|
||||
|
||||
private func teardown(deleteFile: Bool) {
|
||||
running = false
|
||||
// Releasing the AVAudioFile finalizes the .m4a container.
|
||||
file = nil
|
||||
encoder = nil
|
||||
resampler = nil
|
||||
if deleteFile, let url = fileURL {
|
||||
try? FileManager.default.removeItem(at: url)
|
||||
}
|
||||
fileURL = nil
|
||||
}
|
||||
|
||||
// MARK: - Audio session (iOS)
|
||||
|
||||
#if os(iOS)
|
||||
private static func configureAudioSession() throws {
|
||||
let session = AVAudioSession.sharedInstance()
|
||||
#if targetEnvironment(simulator)
|
||||
try session.setCategory(.playAndRecord, mode: .default, options: [.defaultToSpeaker, .allowBluetoothA2DP])
|
||||
#else
|
||||
try session.setCategory(.playAndRecord, mode: .default, options: [.defaultToSpeaker, .allowBluetoothA2DP, .allowBluetoothHFP])
|
||||
#endif
|
||||
try session.setActive(true, options: .notifyOthersOnDeactivation)
|
||||
}
|
||||
|
||||
private static func deactivateAudioSession() {
|
||||
try? AVAudioSession.sharedInstance().setActive(false, options: .notifyOthersOnDeactivation)
|
||||
}
|
||||
#endif
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
//
|
||||
// PTTSettings.swift
|
||||
// bitchat
|
||||
//
|
||||
// This is free and unencumbered software released into the public domain.
|
||||
// For more information, see <https://unlicense.org>
|
||||
//
|
||||
|
||||
import Foundation
|
||||
#if os(iOS)
|
||||
import UIKit
|
||||
#elseif os(macOS)
|
||||
import AppKit
|
||||
#endif
|
||||
|
||||
/// User preference for live push-to-talk voice. One switch controls both
|
||||
/// directions: streaming your holds live, and auto-playing inbound bursts.
|
||||
/// Off means voice messages behave exactly like classic voice notes.
|
||||
enum PTTSettings {
|
||||
private static let liveVoiceEnabledKey = "ptt.liveVoiceEnabled"
|
||||
|
||||
static var liveVoiceEnabled: Bool {
|
||||
get { UserDefaults.standard.object(forKey: liveVoiceEnabledKey) as? Bool ?? true }
|
||||
set { UserDefaults.standard.set(newValue, forKey: liveVoiceEnabledKey) }
|
||||
}
|
||||
|
||||
/// Autoplay is foreground-only: audio must never start from the
|
||||
/// background.
|
||||
@MainActor
|
||||
static var isAppActive: Bool {
|
||||
#if os(iOS)
|
||||
return UIApplication.shared.applicationState == .active
|
||||
#elseif os(macOS)
|
||||
return NSApplication.shared.isActive
|
||||
#else
|
||||
return true
|
||||
#endif
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,183 @@
|
||||
//
|
||||
// VoiceCaptureSession.swift
|
||||
// bitchat
|
||||
//
|
||||
// This is free and unencumbered software released into the public domain.
|
||||
// For more information, see <https://unlicense.org>
|
||||
//
|
||||
|
||||
import BitFoundation
|
||||
import BitLogger
|
||||
import Foundation
|
||||
|
||||
/// Capture backend behind the composer's hold-to-record gesture.
|
||||
/// `VoiceRecordingViewModel` drives one session per press; the concrete type
|
||||
/// decides *how* audio leaves the device: `VoiceNoteCaptureSession` records a
|
||||
/// note delivered on release (today's behavior), `PTTLiveVoiceSession`
|
||||
/// additionally streams frames live while the button is held.
|
||||
@MainActor
|
||||
protocol VoiceCaptureSession: AnyObject {
|
||||
/// Whether audio is leaving the device in real time while recording —
|
||||
/// drives the composer's LIVE treatment.
|
||||
var isLive: Bool { get }
|
||||
func requestPermission() async -> Bool
|
||||
func start() async throws
|
||||
/// Stops capture and returns the finalized voice-note file, or nil when
|
||||
/// nothing valid was captured.
|
||||
func finish() async -> URL?
|
||||
func cancel() async
|
||||
}
|
||||
|
||||
/// The classic record-then-send backend, wrapping the shared `VoiceRecorder`.
|
||||
@MainActor
|
||||
final class VoiceNoteCaptureSession: VoiceCaptureSession {
|
||||
var isLive: Bool { false }
|
||||
|
||||
func requestPermission() async -> Bool {
|
||||
await VoiceRecorder.shared.requestPermission()
|
||||
}
|
||||
|
||||
func start() async throws {
|
||||
try await VoiceRecorder.shared.startRecording()
|
||||
}
|
||||
|
||||
func finish() async -> URL? {
|
||||
await VoiceRecorder.shared.stopRecording()
|
||||
}
|
||||
|
||||
func cancel() async {
|
||||
await VoiceRecorder.shared.cancelRecording()
|
||||
}
|
||||
}
|
||||
|
||||
/// Live push-to-talk backend: streams `VoiceBurstPacket`s to one peer while
|
||||
/// recording, then finalizes the same audio as a standard voice note whose
|
||||
/// file name carries the burst ID (`voice_<burstID>.m4a`) so receivers that
|
||||
/// heard the live stream absorb the note silently instead of seeing a
|
||||
/// duplicate.
|
||||
@MainActor
|
||||
final class PTTLiveVoiceSession: VoiceCaptureSession {
|
||||
let burstID: Data
|
||||
|
||||
private let sendPacket: (Data) -> Void
|
||||
private let capture = PTTCaptureEngine()
|
||||
/// Capture-queue-confined stream state: packetizes frames and lazily
|
||||
/// emits START so packet order is guaranteed by queue serialization.
|
||||
private final class StreamState {
|
||||
var packetizer: VoiceBurstPacketizer
|
||||
var sentStart = false
|
||||
init(burstID: Data) {
|
||||
packetizer = VoiceBurstPacketizer(burstID: burstID)
|
||||
}
|
||||
}
|
||||
private let stream: StreamState
|
||||
private var startDate: Date?
|
||||
private var completed = false
|
||||
|
||||
var isLive: Bool { true }
|
||||
|
||||
/// - Parameter sendPacket: delivers one encoded `VoiceBurstPacket` to the
|
||||
/// target peer; must be safe to call from any queue (BLEService hops to
|
||||
/// its own message queue internally).
|
||||
init(sendPacket: @escaping (Data) -> Void) {
|
||||
self.burstID = VoiceBurstPacket.makeBurstID()
|
||||
self.sendPacket = sendPacket
|
||||
self.stream = StreamState(burstID: burstID)
|
||||
}
|
||||
|
||||
func requestPermission() async -> Bool {
|
||||
await VoiceRecorder.shared.requestPermission()
|
||||
}
|
||||
|
||||
func start() async throws {
|
||||
let outputURL = try Self.makeOutputURL(burstID: burstID)
|
||||
let sendPacket = sendPacket
|
||||
let stream = stream
|
||||
capture.onFrames = { frames in
|
||||
if !stream.sentStart {
|
||||
stream.sentStart = true
|
||||
if let start = VoiceBurstPacket(
|
||||
burstID: stream.packetizer.burstID,
|
||||
seq: 0,
|
||||
kind: .start(codec: .aacLC16kMono)
|
||||
) {
|
||||
sendPacket(start.encode())
|
||||
}
|
||||
}
|
||||
for frame in frames {
|
||||
for packet in stream.packetizer.add(frame) {
|
||||
sendPacket(packet)
|
||||
}
|
||||
}
|
||||
// Flush per callback batch: at ~130-byte frames the budget fits
|
||||
// one frame per packet anyway, and holding residue would add
|
||||
// ~100 ms of avoidable latency.
|
||||
for packet in stream.packetizer.flush() {
|
||||
sendPacket(packet)
|
||||
}
|
||||
}
|
||||
do {
|
||||
try capture.start(outputURL: outputURL)
|
||||
} catch {
|
||||
// The HAL can briefly report a dead input right after the audio
|
||||
// session (re)activates while the route settles; one retry after
|
||||
// a short pause covers it (observed on iPhone field tests).
|
||||
SecureLogger.warning("PTT: capture start failed (\(error)) — retrying once after route settle", category: .session)
|
||||
try? await Task.sleep(nanoseconds: 150_000_000)
|
||||
try capture.start(outputURL: outputURL)
|
||||
}
|
||||
startDate = Date()
|
||||
SecureLogger.info("PTT: live burst \(burstID.hexEncodedString()) capture started", category: .session)
|
||||
}
|
||||
|
||||
func finish() async -> URL? {
|
||||
guard !completed else { return nil }
|
||||
completed = true
|
||||
|
||||
let elapsed = startDate.map { Date().timeIntervalSince($0) } ?? 0
|
||||
let (url, encodedFrames) = capture.stop()
|
||||
// stop() drained the capture queue, so touching `stream` is safe now.
|
||||
|
||||
guard elapsed >= VoiceRecorder.minRecordingDuration, let url else {
|
||||
sendControlPacket(.canceled)
|
||||
if let url {
|
||||
try? FileManager.default.removeItem(at: url)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
for packet in stream.packetizer.flush() {
|
||||
sendPacket(packet)
|
||||
}
|
||||
let durationMs = UInt32((Double(encodedFrames) * PTTAudioFormat.frameDuration * 1000).rounded())
|
||||
sendControlPacket(.end(totalDataPackets: stream.packetizer.dataPacketCount, durationMs: durationMs))
|
||||
SecureLogger.info("PTT: live burst \(burstID.hexEncodedString()) finished — \(stream.packetizer.dataPacketCount) data packets, \(encodedFrames) frames, \(durationMs) ms", category: .session)
|
||||
return url
|
||||
}
|
||||
|
||||
func cancel() async {
|
||||
guard !completed else { return }
|
||||
completed = true
|
||||
capture.cancel()
|
||||
sendControlPacket(.canceled)
|
||||
}
|
||||
|
||||
private func sendControlPacket(_ kind: VoiceBurstPacket.Kind) {
|
||||
guard let packet = VoiceBurstPacket(burstID: burstID, seq: stream.packetizer.nextSeq, kind: kind) else { return }
|
||||
sendPacket(packet.encode())
|
||||
}
|
||||
|
||||
private static func makeOutputURL(burstID: Data) throws -> URL {
|
||||
let base = try FileManager.default.url(
|
||||
for: .applicationSupportDirectory,
|
||||
in: .userDomainMask,
|
||||
appropriateFor: nil,
|
||||
create: true
|
||||
)
|
||||
let directory = base
|
||||
.appendingPathComponent("files", isDirectory: true)
|
||||
.appendingPathComponent("voicenotes/outgoing", isDirectory: true)
|
||||
try FileManager.default.createDirectory(at: directory, withIntermediateDirectories: true, attributes: nil)
|
||||
return directory.appendingPathComponent("voice_\(burstID.hexEncodedString()).m4a")
|
||||
}
|
||||
}
|
||||
@@ -181,23 +181,35 @@ final class VoiceNotePlaybackController: NSObject, ObservableObject, AVAudioPlay
|
||||
}
|
||||
}
|
||||
|
||||
/// Ensures only one voice note plays at a time.
|
||||
/// Something that can hold the app's single audio-playback slot and yield it
|
||||
/// when another playback starts (voice notes pause; live bursts stop).
|
||||
protocol ExclusivePlayback: AnyObject {
|
||||
func pauseForExclusivity()
|
||||
}
|
||||
|
||||
extension VoiceNotePlaybackController: ExclusivePlayback {
|
||||
func pauseForExclusivity() {
|
||||
pause()
|
||||
}
|
||||
}
|
||||
|
||||
/// Ensures only one voice playback (note or live burst) runs at a time.
|
||||
final class VoiceNotePlaybackCoordinator {
|
||||
static let shared = VoiceNotePlaybackCoordinator()
|
||||
|
||||
private weak var activeController: VoiceNotePlaybackController?
|
||||
private weak var activeController: (any ExclusivePlayback)?
|
||||
|
||||
private init() {}
|
||||
|
||||
func activate(_ controller: VoiceNotePlaybackController) {
|
||||
func activate(_ controller: any ExclusivePlayback) {
|
||||
if activeController === controller {
|
||||
return
|
||||
}
|
||||
activeController?.pause()
|
||||
activeController?.pauseForExclusivity()
|
||||
activeController = controller
|
||||
}
|
||||
|
||||
func deactivate(_ controller: VoiceNotePlaybackController) {
|
||||
func deactivate(_ controller: any ExclusivePlayback) {
|
||||
if activeController === controller {
|
||||
activeController = nil
|
||||
}
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -77,6 +77,8 @@ enum NoisePayloadType: UInt8 {
|
||||
// Private groups (0x04/0x05 reserved by other features)
|
||||
case groupInvite = 0x06 // Creator-signed group state (invite)
|
||||
case groupKeyUpdate = 0x07 // Creator-signed group state (key rotation / roster update)
|
||||
// Live voice (push-to-talk)
|
||||
case voiceFrame = 0x08 // One live voice-burst packet (see VoiceBurstPacket)
|
||||
// Verification (QR-based OOB binding)
|
||||
case verifyChallenge = 0x10 // Verification challenge
|
||||
case verifyResponse = 0x11 // Verification response
|
||||
@@ -90,6 +92,7 @@ enum NoisePayloadType: UInt8 {
|
||||
case .delivered: return "delivered"
|
||||
case .groupInvite: return "groupInvite"
|
||||
case .groupKeyUpdate: return "groupKeyUpdate"
|
||||
case .voiceFrame: return "voiceFrame"
|
||||
case .verifyChallenge: return "verifyChallenge"
|
||||
case .verifyResponse: return "verifyResponse"
|
||||
case .vouch: return "vouch"
|
||||
|
||||
@@ -0,0 +1,220 @@
|
||||
//
|
||||
// VoiceBurstPacket.swift
|
||||
// bitchat
|
||||
//
|
||||
// This is free and unencumbered software released into the public domain.
|
||||
// For more information, see <https://unlicense.org>
|
||||
//
|
||||
|
||||
import Foundation
|
||||
import Security
|
||||
|
||||
/// Audio codec of a live voice burst. START packets carry it so receivers can
|
||||
/// reject bursts they can't decode instead of feeding garbage to the decoder.
|
||||
enum VoiceBurstCodec: UInt8 {
|
||||
/// AAC-LC, 16 kHz, mono, ~16 kbps — matches the voice-note recorder, so
|
||||
/// the finalized `.m4a` and the live frames come from the same encoder
|
||||
/// settings.
|
||||
case aacLC16kMono = 0x01
|
||||
}
|
||||
|
||||
/// One packet of a live push-to-talk voice burst (the inner payload of
|
||||
/// `NoisePayloadType.voiceFrame`, and — for public mesh bursts — the payload
|
||||
/// of `MessageType.voiceFrame`).
|
||||
///
|
||||
/// Wire format:
|
||||
/// ```
|
||||
/// [burstID: 8][seq: UInt16 BE][flags: UInt8][payload…]
|
||||
/// ```
|
||||
/// - flags 0x01 (START): payload = [codec: UInt8]
|
||||
/// - flags 0x02 (END): payload = [totalDataPackets: UInt16 BE][durationMs: UInt32 BE]
|
||||
/// - flags 0x04 (CANCELED): empty payload; receivers discard the burst
|
||||
/// - flags 0x00 (data): payload = repeated [length: UInt16 BE][AAC frame]
|
||||
struct VoiceBurstPacket: Equatable {
|
||||
enum Kind: Equatable {
|
||||
case start(codec: VoiceBurstCodec)
|
||||
case frames([Data])
|
||||
case end(totalDataPackets: UInt16, durationMs: UInt32)
|
||||
case canceled
|
||||
}
|
||||
|
||||
static let burstIDSize = 8
|
||||
private static let headerSize = burstIDSize + 2 + 1
|
||||
/// Sanity cap on frames per packet; real packets carry 1-2 frames.
|
||||
static let maxFramesPerPacket = 8
|
||||
|
||||
private enum Flags {
|
||||
static let start: UInt8 = 0x01
|
||||
static let end: UInt8 = 0x02
|
||||
static let canceled: UInt8 = 0x04
|
||||
}
|
||||
|
||||
let burstID: Data
|
||||
let seq: UInt16
|
||||
let kind: Kind
|
||||
|
||||
init?(burstID: Data, seq: UInt16, kind: Kind) {
|
||||
guard burstID.count == Self.burstIDSize else { return nil }
|
||||
if case .frames(let frames) = kind {
|
||||
guard !frames.isEmpty,
|
||||
frames.count <= Self.maxFramesPerPacket,
|
||||
frames.allSatisfy({ !$0.isEmpty && $0.count <= Int(UInt16.max) })
|
||||
else { return nil }
|
||||
}
|
||||
self.burstID = burstID
|
||||
self.seq = seq
|
||||
self.kind = kind
|
||||
}
|
||||
|
||||
func encode() -> Data {
|
||||
var data = Data(capacity: Self.headerSize + payloadSize)
|
||||
data.append(burstID)
|
||||
data.append(UInt8((seq >> 8) & 0xFF))
|
||||
data.append(UInt8(seq & 0xFF))
|
||||
switch kind {
|
||||
case .start(let codec):
|
||||
data.append(Flags.start)
|
||||
data.append(codec.rawValue)
|
||||
case .frames(let frames):
|
||||
data.append(0)
|
||||
for frame in frames {
|
||||
let length = UInt16(frame.count)
|
||||
data.append(UInt8((length >> 8) & 0xFF))
|
||||
data.append(UInt8(length & 0xFF))
|
||||
data.append(frame)
|
||||
}
|
||||
case .end(let totalDataPackets, let durationMs):
|
||||
data.append(Flags.end)
|
||||
data.append(UInt8((totalDataPackets >> 8) & 0xFF))
|
||||
data.append(UInt8(totalDataPackets & 0xFF))
|
||||
for shift in stride(from: 24, through: 0, by: -8) {
|
||||
data.append(UInt8((durationMs >> UInt32(shift)) & 0xFF))
|
||||
}
|
||||
case .canceled:
|
||||
data.append(Flags.canceled)
|
||||
}
|
||||
return data
|
||||
}
|
||||
|
||||
static func decode(_ data: Data) -> VoiceBurstPacket? {
|
||||
// Work on a re-based copy so subscripting is offset-safe.
|
||||
let data = Data(data)
|
||||
guard data.count >= headerSize else { return nil }
|
||||
|
||||
let burstID = data.prefix(burstIDSize)
|
||||
let seq = (UInt16(data[burstIDSize]) << 8) | UInt16(data[burstIDSize + 1])
|
||||
let flags = data[burstIDSize + 2]
|
||||
let payload = data.dropFirst(headerSize)
|
||||
|
||||
let kind: Kind
|
||||
switch flags {
|
||||
case Flags.start:
|
||||
guard let codecByte = payload.first,
|
||||
let codec = VoiceBurstCodec(rawValue: codecByte)
|
||||
else { return nil }
|
||||
kind = .start(codec: codec)
|
||||
case Flags.end:
|
||||
guard payload.count >= 6 else { return nil }
|
||||
let bytes = Array(payload)
|
||||
let total = (UInt16(bytes[0]) << 8) | UInt16(bytes[1])
|
||||
let duration = bytes[2...5].reduce(UInt32(0)) { ($0 << 8) | UInt32($1) }
|
||||
kind = .end(totalDataPackets: total, durationMs: duration)
|
||||
case Flags.canceled:
|
||||
kind = .canceled
|
||||
case 0:
|
||||
var frames: [Data] = []
|
||||
var offset = payload.startIndex
|
||||
while offset < payload.endIndex {
|
||||
guard payload.distance(from: offset, to: payload.endIndex) >= 2 else { return nil }
|
||||
let length = (Int(payload[offset]) << 8) | Int(payload[payload.index(after: offset)])
|
||||
offset = payload.index(offset, offsetBy: 2)
|
||||
guard length > 0,
|
||||
payload.distance(from: offset, to: payload.endIndex) >= length,
|
||||
frames.count < maxFramesPerPacket
|
||||
else { return nil }
|
||||
let end = payload.index(offset, offsetBy: length)
|
||||
frames.append(Data(payload[offset..<end]))
|
||||
offset = end
|
||||
}
|
||||
guard !frames.isEmpty else { return nil }
|
||||
kind = .frames(frames)
|
||||
default:
|
||||
return nil
|
||||
}
|
||||
|
||||
return VoiceBurstPacket(burstID: Data(burstID), seq: seq, kind: kind)
|
||||
}
|
||||
|
||||
static func makeBurstID() -> Data {
|
||||
var bytes = Data(count: burstIDSize)
|
||||
let result = bytes.withUnsafeMutableBytes {
|
||||
SecRandomCopyBytes(kSecRandomDefault, burstIDSize, $0.baseAddress!)
|
||||
}
|
||||
guard result == errSecSuccess else {
|
||||
return Data((0..<burstIDSize).map { _ in UInt8.random(in: .min ... .max) })
|
||||
}
|
||||
return bytes
|
||||
}
|
||||
|
||||
private var payloadSize: Int {
|
||||
switch kind {
|
||||
case .start: return 1
|
||||
case .frames(let frames): return frames.reduce(0) { $0 + 2 + $1.count }
|
||||
case .end: return 6
|
||||
case .canceled: return 0
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Greedy packetizer for outgoing bursts: batches encoded frames into
|
||||
/// `VoiceBurstPacket`s without exceeding the byte budget that keeps each
|
||||
/// packet in a single BLE frame after Noise encryption and padding.
|
||||
/// Not thread-safe — confine to one queue.
|
||||
struct VoiceBurstPacketizer {
|
||||
let burstID: Data
|
||||
private let budget: Int
|
||||
private var pendingFrames: [Data] = []
|
||||
private var pendingSize = 0
|
||||
/// seq 0 is reserved for START; data packets start at 1.
|
||||
private(set) var nextSeq: UInt16 = 1
|
||||
private(set) var dataPacketCount: UInt16 = 0
|
||||
|
||||
init(burstID: Data, budget: Int = TransportConfig.pttMaxBurstContentBytes) {
|
||||
self.burstID = burstID
|
||||
self.budget = budget
|
||||
}
|
||||
|
||||
/// Adds one encoded frame, returning any packets that became full.
|
||||
/// Frames larger than the budget are dropped (the encoder's ~130-byte
|
||||
/// frames never hit this; it guards against misconfiguration looping).
|
||||
mutating func add(_ frame: Data) -> [Data] {
|
||||
let frameCost = 2 + frame.count
|
||||
guard VoiceBurstPacket.burstIDSize + 3 + frameCost <= budget else { return [] }
|
||||
|
||||
var packets: [Data] = []
|
||||
if !pendingFrames.isEmpty,
|
||||
VoiceBurstPacket.burstIDSize + 3 + pendingSize + frameCost > budget
|
||||
|| pendingFrames.count >= VoiceBurstPacket.maxFramesPerPacket {
|
||||
packets.append(contentsOf: flush())
|
||||
}
|
||||
pendingFrames.append(frame)
|
||||
pendingSize += frameCost
|
||||
return packets
|
||||
}
|
||||
|
||||
/// Emits any buffered frames as a final data packet.
|
||||
mutating func flush() -> [Data] {
|
||||
guard !pendingFrames.isEmpty,
|
||||
let packet = VoiceBurstPacket(burstID: burstID, seq: nextSeq, kind: .frames(pendingFrames))
|
||||
else {
|
||||
pendingFrames = []
|
||||
pendingSize = 0
|
||||
return []
|
||||
}
|
||||
pendingFrames = []
|
||||
pendingSize = 0
|
||||
nextSeq &+= 1
|
||||
dataPacketCount &+= 1
|
||||
return [packet.encode()]
|
||||
}
|
||||
}
|
||||
@@ -1456,6 +1456,27 @@ final class BLEService: NSObject {
|
||||
sendNoisePayload(NoisePayload(type: .vouch, data: payload).encode(), to: peerID)
|
||||
}
|
||||
|
||||
// MARK: Live Voice (PTT)
|
||||
|
||||
/// Sends one live voice-burst packet inside the Noise session. Unlike
|
||||
/// `sendNoisePayload` this never queues behind a handshake: live audio is
|
||||
/// only useful now, so without an established session frames are dropped.
|
||||
func sendVoiceFrame(_ burstContent: Data, to peerID: PeerID) {
|
||||
messageQueue.async { [weak self] in
|
||||
guard let self else { return }
|
||||
guard self.noiseService.hasEstablishedSession(with: peerID) else {
|
||||
SecureLogger.debug("PTT: dropping voice frame — no established session with \(peerID.id.prefix(8))…", category: .session)
|
||||
return
|
||||
}
|
||||
do {
|
||||
let typedPayload = NoisePayload(type: .voiceFrame, data: burstContent).encode()
|
||||
self.broadcastPacket(try self.makeEncryptedNoisePacket(typedPayload, to: peerID))
|
||||
} catch {
|
||||
SecureLogger.error("Failed to send voice frame: \(error)", category: .session)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func addPeerAuthenticatedObserver(_ handler: @escaping (PeerID, String) -> Void) {
|
||||
// Appends to the encryption service's handler array, so this never
|
||||
// displaces the callbacks installed by installNoiseSessionCallbacks.
|
||||
|
||||
@@ -155,6 +155,12 @@ protocol Transport: AnyObject {
|
||||
func sendFilePrivate(_ packet: BitchatFilePacket, to peerID: PeerID, transferId: String)
|
||||
func cancelTransfer(_ transferId: String)
|
||||
|
||||
// Live voice / push-to-talk (mesh transports only): one encoded
|
||||
// `VoiceBurstPacket`, fire-and-forget inside the Noise session. Frames are
|
||||
// only useful now — transports drop them (never queue) when no
|
||||
// established session exists.
|
||||
func sendVoiceFrame(_ burstContent: Data, to peerID: PeerID)
|
||||
|
||||
// Courier store-and-forward (mesh transports only): seal a message to the
|
||||
// recipient's static key and hand it to connected couriers for physical
|
||||
// delivery while the recipient is offline. Returns false when the
|
||||
@@ -231,6 +237,7 @@ extension Transport {
|
||||
func addPeerAuthenticatedObserver(_ handler: @escaping (PeerID, String) -> Void) {}
|
||||
func sendCourierMessage(_ content: String, messageID: String, recipientNoiseKey: Data, via couriers: [PeerID]) -> Bool { false }
|
||||
func sendBoardPayload(_ payload: Data) {}
|
||||
func sendVoiceFrame(_ burstContent: Data, to peerID: PeerID) {}
|
||||
|
||||
// Mesh diagnostics are mesh-transport-only; other transports report
|
||||
// "no reply"/"no path" rather than pretending to measure anything.
|
||||
|
||||
@@ -16,6 +16,25 @@ enum TransportConfig {
|
||||
static let bleFragmentRelayTtlCap: UInt8 = 7
|
||||
static let bleFragmentRelayTtlCapDense: UInt8 = 5 // Contain fragment floods in dense graphs
|
||||
|
||||
// Live voice (push-to-talk)
|
||||
// Burst-content budget per voice packet. Sized so the Noise ciphertext
|
||||
// (content + 1 type byte + 16 tag bytes) stays within MessagePadding's
|
||||
// 256-byte bucket and the whole directed packet (16 header + 8 sender +
|
||||
// 8 recipient + 256 payload = 288 bytes) rides one BLE frame — live audio
|
||||
// must never enter the fragment scheduler, which caps concurrent
|
||||
// transfers at 2 and would let voice starve file sends.
|
||||
static let pttMaxBurstContentBytes: Int = 210
|
||||
static let pttJitterBufferSeconds: TimeInterval = 0.35 // buffered audio before live playback starts
|
||||
static let pttJitterDeadlineSeconds: TimeInterval = 0.5 // start anyway after this wall-clock wait
|
||||
static let pttBurstEndTimeoutSeconds: TimeInterval = 3.0 // no frames -> burst considered ended
|
||||
static let pttAssemblyStaleSeconds: TimeInterval = 30.0 // drop half-open assemblies after this
|
||||
static let pttMaxConcurrentAssemblies: Int = 8 // concurrent inbound bursts cap
|
||||
static let pttMaxBurstBytes: Int = 384 * 1024 // 120s at ~2KB/s + generous slack
|
||||
static let pttFinishedBurstRegistrySeconds: TimeInterval = 600 // window to absorb the finalized note
|
||||
// Inbound flood guard: a real burst arrives at ~2KB/s; allow 3x plus a
|
||||
// small settling allowance before dropping a sender's frames.
|
||||
static let pttInboundMaxBytesPerSecond: Int = 6_000
|
||||
|
||||
// Mesh diagnostics (/ping)
|
||||
static let meshPingTimeoutSeconds: TimeInterval = 10 // Give up on a probe after this window
|
||||
static let meshPingInboundMaxPerLink: Int = 5 // Inbound ping budget per ingress link (claimed sender is spoofable)...
|
||||
|
||||
@@ -0,0 +1,434 @@
|
||||
import BitFoundation
|
||||
import BitLogger
|
||||
import Foundation
|
||||
|
||||
/// The narrow surface `ChatLiveVoiceCoordinator` needs from its owner.
|
||||
///
|
||||
/// Follows the `ChatDeliveryContext` exemplar: the coordinator depends on the
|
||||
/// minimal context it actually uses instead of holding an `unowned` back-ref
|
||||
/// to the whole `ChatViewModel`, keeping it independently testable.
|
||||
@MainActor
|
||||
protocol ChatLiveVoiceContext: AnyObject {
|
||||
var nickname: String { get }
|
||||
var selectedPrivateChatPeer: PeerID? { get }
|
||||
func isPeerBlocked(_ peerID: PeerID) -> Bool
|
||||
func resolveNickname(for peerID: PeerID) -> String
|
||||
/// Routes an inbound private message through the full pipeline
|
||||
/// (store append, unread state, notification, read receipt).
|
||||
func handlePrivateMessage(_ message: BitchatMessage)
|
||||
/// Replace-or-append by message ID via the single-writer store intent.
|
||||
func upsertPrivateMessage(_ message: BitchatMessage, in peerID: PeerID)
|
||||
@discardableResult
|
||||
func removePrivateMessage(withID messageID: String) -> BitchatMessage?
|
||||
func notifyUIChanged()
|
||||
}
|
||||
|
||||
extension ChatViewModel: ChatLiveVoiceContext {}
|
||||
|
||||
/// Assembles inbound live push-to-talk bursts (`NoisePayloadType.voiceFrame`):
|
||||
/// orders packets behind a jitter window, persists frames progressively as an
|
||||
/// ADTS `.aac` so even a partial burst is a replayable voice-note bubble,
|
||||
/// optionally plays the stream live, and absorbs the sender's finalized
|
||||
/// `.m4a` voice note (matched by the burst ID in its file name) into the same
|
||||
/// bubble so nobody sees a duplicate.
|
||||
@MainActor
|
||||
final class ChatLiveVoiceCoordinator {
|
||||
private final class Assembly {
|
||||
let burstID: Data
|
||||
let peerID: PeerID
|
||||
let message: BitchatMessage
|
||||
var messageID: String { message.id }
|
||||
var messageTimestamp: Date { message.timestamp }
|
||||
let fileURL: URL
|
||||
var fileHandle: FileHandle?
|
||||
/// Data packets buffered ahead of `nextSeq` (seq -> frames).
|
||||
var buffered: [UInt16: [Data]] = [:]
|
||||
/// Next data-packet seq to deliver (seq 0 is START).
|
||||
var nextSeq: UInt16 = 1
|
||||
var deliveredFrames = 0
|
||||
var receivedBytes = 0
|
||||
let firstPacketAt: Date
|
||||
var endInfo: (totalDataPackets: UInt16, durationMs: UInt32)?
|
||||
/// When a seq gap was first observed; after
|
||||
/// `ChatLiveVoiceCoordinator.gapSkipSeconds` the gap is skipped.
|
||||
var gapSince: Date?
|
||||
var player: PTTBurstPlayer?
|
||||
var idleTimeout: Task<Void, Never>?
|
||||
var gapRedrain: Task<Void, Never>?
|
||||
|
||||
init(burstID: Data, peerID: PeerID, message: BitchatMessage, fileURL: URL, fileHandle: FileHandle) {
|
||||
self.burstID = burstID
|
||||
self.peerID = peerID
|
||||
self.message = message
|
||||
self.fileURL = fileURL
|
||||
self.fileHandle = fileHandle
|
||||
self.firstPacketAt = Date()
|
||||
}
|
||||
}
|
||||
|
||||
private struct FinishedBurst {
|
||||
let messageID: String
|
||||
let peerID: PeerID
|
||||
let fileURL: URL
|
||||
let messageTimestamp: Date
|
||||
let expiresAt: Date
|
||||
}
|
||||
|
||||
/// How long a missing packet stalls delivery before being skipped.
|
||||
private static let gapSkipSeconds: TimeInterval = 0.5
|
||||
private static let finishedBurstsCap = 32
|
||||
|
||||
private unowned let context: any ChatLiveVoiceContext
|
||||
private var assemblies: [Data: Assembly] = [:]
|
||||
private var finishedBursts: [Data: FinishedBurst] = [:]
|
||||
|
||||
init(context: any ChatLiveVoiceContext) {
|
||||
self.context = context
|
||||
}
|
||||
|
||||
// MARK: - Inbound frames
|
||||
|
||||
func handleVoiceFramePayload(from peerID: PeerID, payload: Data, timestamp: Date) {
|
||||
// Live voice off means classic-notes-only in both directions: no live
|
||||
// bubble, no partial file, no early notification — the finalized
|
||||
// voice note still arrives through the normal pipeline.
|
||||
guard PTTSettings.liveVoiceEnabled else { return }
|
||||
guard let packet = VoiceBurstPacket.decode(payload) else {
|
||||
SecureLogger.warning("PTT: undecodable voice frame from \(peerID.id.prefix(8))… (\(payload.count) bytes: \(payload.prefix(16).hexEncodedString())…)", category: .session)
|
||||
return
|
||||
}
|
||||
guard !context.isPeerBlocked(peerID) else {
|
||||
SecureLogger.debug("PTT: dropping voice frame from blocked peer \(peerID.id.prefix(8))…", category: .session)
|
||||
return
|
||||
}
|
||||
|
||||
if let assembly = assemblies[packet.burstID] {
|
||||
// The sender is Noise-authenticated; a different peer reusing the
|
||||
// same burst ID is a collision or a replay — drop it.
|
||||
guard assembly.peerID == peerID else { return }
|
||||
apply(packet, to: assembly)
|
||||
return
|
||||
}
|
||||
|
||||
switch packet.kind {
|
||||
case .start, .frames:
|
||||
// A data packet with no prior START (lost or mid-burst state)
|
||||
// still opens the assembly with the default codec.
|
||||
guard assemblies.count < TransportConfig.pttMaxConcurrentAssemblies else {
|
||||
SecureLogger.debug("PTT: dropping burst from \(peerID.id.prefix(8))… — assembly cap reached", category: .session)
|
||||
return
|
||||
}
|
||||
guard let assembly = makeAssembly(burstID: packet.burstID, peerID: peerID, timestamp: timestamp) else { return }
|
||||
assemblies[packet.burstID] = assembly
|
||||
apply(packet, to: assembly)
|
||||
case .end, .canceled:
|
||||
// Control packet for a burst we never saw — nothing to do.
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether this message is the bubble of a burst still streaming in.
|
||||
func isLiveVoiceMessage(_ message: BitchatMessage) -> Bool {
|
||||
assemblies.values.contains { $0.messageID == message.id }
|
||||
}
|
||||
|
||||
/// Called for every inbound private message: when it is the finalized
|
||||
/// voice note of a burst we assembled (matched by burst ID in the file
|
||||
/// name), swap it into the existing live bubble and report `true` so the
|
||||
/// caller skips normal handling — no duplicate row, no second
|
||||
/// notification.
|
||||
func absorbFinalizedVoiceNote(_ message: BitchatMessage) -> Bool {
|
||||
let prefix = MimeType.Category.audio.messagePrefix
|
||||
guard message.content.hasPrefix(prefix),
|
||||
let burstID = Self.burstID(fromVoiceFileName: String(message.content.dropFirst(prefix.count)))
|
||||
else { return false }
|
||||
|
||||
// The note usually lands after END, but a lost END or a fast transfer
|
||||
// can beat it — close out the live assembly first.
|
||||
if let assembly = assemblies[burstID] {
|
||||
finalize(assembly)
|
||||
}
|
||||
|
||||
pruneFinishedBursts()
|
||||
guard let finished = finishedBursts[burstID] else { return false }
|
||||
// Bind the note to the burst's authenticated sender.
|
||||
guard message.senderPeerID == nil || message.senderPeerID == finished.peerID else { return false }
|
||||
|
||||
let replacement = BitchatMessage(
|
||||
id: finished.messageID,
|
||||
sender: message.sender,
|
||||
content: message.content,
|
||||
timestamp: finished.messageTimestamp,
|
||||
isRelay: false,
|
||||
originalSender: nil,
|
||||
isPrivate: true,
|
||||
recipientNickname: context.nickname,
|
||||
senderPeerID: finished.peerID,
|
||||
mentions: nil,
|
||||
deliveryStatus: message.deliveryStatus
|
||||
)
|
||||
context.upsertPrivateMessage(replacement, in: finished.peerID)
|
||||
|
||||
// The complete .m4a replaces the partial live capture.
|
||||
WaveformCache.shared.purge(url: finished.fileURL)
|
||||
try? FileManager.default.removeItem(at: finished.fileURL)
|
||||
finishedBursts.removeValue(forKey: burstID)
|
||||
|
||||
context.notifyUIChanged()
|
||||
SecureLogger.debug("PTT: absorbed finalized note for burst \(burstID.hexEncodedString())", category: .session)
|
||||
return true
|
||||
}
|
||||
|
||||
// MARK: - Assembly lifecycle
|
||||
|
||||
private func makeAssembly(burstID: Data, peerID: PeerID, timestamp: Date) -> Assembly? {
|
||||
guard let fileURL = Self.makeIncomingURL(burstID: burstID) else {
|
||||
SecureLogger.error("PTT: cannot resolve incoming media directory for burst \(burstID.hexEncodedString())", category: .session)
|
||||
return nil
|
||||
}
|
||||
FileManager.default.createFile(atPath: fileURL.path, contents: nil)
|
||||
guard let handle = try? FileHandle(forWritingTo: fileURL) else {
|
||||
SecureLogger.error("PTT: cannot open capture file for burst \(burstID.hexEncodedString())", category: .session)
|
||||
try? FileManager.default.removeItem(at: fileURL)
|
||||
return nil
|
||||
}
|
||||
|
||||
let message = BitchatMessage(
|
||||
sender: context.resolveNickname(for: peerID),
|
||||
content: "\(MimeType.Category.audio.messagePrefix)\(fileURL.lastPathComponent)",
|
||||
timestamp: timestamp,
|
||||
isRelay: false,
|
||||
originalSender: nil,
|
||||
isPrivate: true,
|
||||
recipientNickname: context.nickname,
|
||||
senderPeerID: peerID
|
||||
)
|
||||
|
||||
let assembly = Assembly(
|
||||
burstID: burstID,
|
||||
peerID: peerID,
|
||||
message: message,
|
||||
fileURL: fileURL,
|
||||
fileHandle: handle
|
||||
)
|
||||
|
||||
// Full inbound pipeline: store append, unread, notification.
|
||||
context.handlePrivateMessage(message)
|
||||
|
||||
// Live playback only when the user is looking at this conversation
|
||||
// with the app frontmost and live voice enabled.
|
||||
if PTTSettings.liveVoiceEnabled,
|
||||
PTTSettings.isAppActive,
|
||||
context.selectedPrivateChatPeer == peerID {
|
||||
assembly.player = PTTBurstPlayer()
|
||||
}
|
||||
|
||||
SecureLogger.debug("PTT: burst \(burstID.hexEncodedString()) started from \(peerID.id.prefix(8))…", category: .session)
|
||||
return assembly
|
||||
}
|
||||
|
||||
private func apply(_ packet: VoiceBurstPacket, to assembly: Assembly) {
|
||||
assembly.receivedBytes += packet.encode().count
|
||||
let elapsed = Date().timeIntervalSince(assembly.firstPacketAt)
|
||||
// Flood guards: a real burst arrives at ~2 KB/s.
|
||||
guard assembly.receivedBytes <= TransportConfig.pttInboundMaxBytesPerSecond * Int(elapsed + 2),
|
||||
assembly.receivedBytes <= TransportConfig.pttMaxBurstBytes
|
||||
else {
|
||||
SecureLogger.warning("PTT: burst from \(assembly.peerID.id.prefix(8))… exceeded rate/size caps — finalizing", category: .security)
|
||||
finalize(assembly)
|
||||
return
|
||||
}
|
||||
|
||||
rescheduleIdleTimeout(for: assembly)
|
||||
|
||||
switch packet.kind {
|
||||
case .start(let codec):
|
||||
guard codec == .aacLC16kMono else {
|
||||
// Codec we can't decode: drop the burst; the finalized note
|
||||
// (whose MIME/magic the file handler validates) still arrives.
|
||||
cancelAssembly(assembly)
|
||||
return
|
||||
}
|
||||
case .frames(let frames):
|
||||
guard packet.seq >= assembly.nextSeq, assembly.buffered[packet.seq] == nil else { return }
|
||||
assembly.buffered[packet.seq] = frames
|
||||
drainInOrder(assembly)
|
||||
case .end(let totalDataPackets, let durationMs):
|
||||
assembly.endInfo = (totalDataPackets, durationMs)
|
||||
drainInOrder(assembly)
|
||||
finalizeIfComplete(assembly)
|
||||
case .canceled:
|
||||
cancelAssembly(assembly)
|
||||
}
|
||||
}
|
||||
|
||||
private func drainInOrder(_ assembly: Assembly) {
|
||||
while true {
|
||||
if let frames = assembly.buffered.removeValue(forKey: assembly.nextSeq) {
|
||||
deliver(frames, to: assembly)
|
||||
assembly.nextSeq &+= 1
|
||||
assembly.gapSince = nil
|
||||
continue
|
||||
}
|
||||
guard !assembly.buffered.isEmpty else {
|
||||
assembly.gapSince = nil
|
||||
return
|
||||
}
|
||||
// Packets buffered ahead of a hole.
|
||||
if let since = assembly.gapSince {
|
||||
guard Date().timeIntervalSince(since) >= Self.gapSkipSeconds,
|
||||
let smallest = assembly.buffered.keys.min()
|
||||
else { return }
|
||||
// Give up on the missing packet(s); playback underrun already
|
||||
// covered the audible gap.
|
||||
assembly.nextSeq = smallest
|
||||
assembly.gapSince = nil
|
||||
} else {
|
||||
assembly.gapSince = Date()
|
||||
scheduleGapRedrain(for: assembly)
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func deliver(_ frames: [Data], to assembly: Assembly) {
|
||||
for frame in frames {
|
||||
do {
|
||||
try assembly.fileHandle?.write(contentsOf: ADTSFramer.frame(frame))
|
||||
} catch {
|
||||
SecureLogger.error("PTT: incoming burst write failed: \(error)", category: .session)
|
||||
assembly.fileHandle = nil
|
||||
}
|
||||
}
|
||||
assembly.deliveredFrames += frames.count
|
||||
assembly.player?.enqueue(frames)
|
||||
}
|
||||
|
||||
private func finalizeIfComplete(_ assembly: Assembly) {
|
||||
guard let end = assembly.endInfo else { return }
|
||||
// All data packets delivered when nextSeq passed the last one
|
||||
// (data seqs are 1...totalDataPackets). Otherwise stragglers may
|
||||
// still arrive; the gap-redrain or idle timeout closes the burst.
|
||||
if assembly.nextSeq > end.totalDataPackets {
|
||||
finalize(assembly)
|
||||
}
|
||||
}
|
||||
|
||||
private func finalize(_ assembly: Assembly) {
|
||||
assembly.idleTimeout?.cancel()
|
||||
assembly.gapRedrain?.cancel()
|
||||
// Deliver whatever is decodable past any remaining holes.
|
||||
while !assembly.buffered.isEmpty, let smallest = assembly.buffered.keys.min() {
|
||||
assembly.nextSeq = smallest
|
||||
if let frames = assembly.buffered.removeValue(forKey: smallest) {
|
||||
deliver(frames, to: assembly)
|
||||
assembly.nextSeq &+= 1
|
||||
}
|
||||
}
|
||||
try? assembly.fileHandle?.close()
|
||||
assembly.fileHandle = nil
|
||||
assemblies.removeValue(forKey: assembly.burstID)
|
||||
|
||||
guard assembly.deliveredFrames > 0 else {
|
||||
// Nothing audible ever arrived — drop the empty bubble.
|
||||
context.removePrivateMessage(withID: assembly.messageID)
|
||||
try? FileManager.default.removeItem(at: assembly.fileURL)
|
||||
context.notifyUIChanged()
|
||||
return
|
||||
}
|
||||
|
||||
assembly.player?.finishAfterDrain()
|
||||
// The bubble's waveform may have been computed from a partial file.
|
||||
WaveformCache.shared.purge(url: assembly.fileURL)
|
||||
// Republish so the row re-renders without its LIVE treatment even if
|
||||
// no finalized note ever arrives to swap in.
|
||||
context.upsertPrivateMessage(assembly.message, in: assembly.peerID)
|
||||
|
||||
pruneFinishedBursts()
|
||||
finishedBursts[assembly.burstID] = FinishedBurst(
|
||||
messageID: assembly.messageID,
|
||||
peerID: assembly.peerID,
|
||||
fileURL: assembly.fileURL,
|
||||
messageTimestamp: assembly.messageTimestamp,
|
||||
expiresAt: Date().addingTimeInterval(TransportConfig.pttFinishedBurstRegistrySeconds)
|
||||
)
|
||||
context.notifyUIChanged()
|
||||
SecureLogger.debug("PTT: burst \(assembly.burstID.hexEncodedString()) finalized (\(assembly.deliveredFrames) frames)", category: .session)
|
||||
}
|
||||
|
||||
private func cancelAssembly(_ assembly: Assembly) {
|
||||
assembly.idleTimeout?.cancel()
|
||||
assembly.gapRedrain?.cancel()
|
||||
assembly.player?.stop()
|
||||
try? assembly.fileHandle?.close()
|
||||
assembly.fileHandle = nil
|
||||
assemblies.removeValue(forKey: assembly.burstID)
|
||||
context.removePrivateMessage(withID: assembly.messageID)
|
||||
WaveformCache.shared.purge(url: assembly.fileURL)
|
||||
try? FileManager.default.removeItem(at: assembly.fileURL)
|
||||
context.notifyUIChanged()
|
||||
}
|
||||
|
||||
// MARK: - Timers
|
||||
|
||||
private func rescheduleIdleTimeout(for assembly: Assembly) {
|
||||
assembly.idleTimeout?.cancel()
|
||||
let burstID = assembly.burstID
|
||||
assembly.idleTimeout = Task { @MainActor [weak self] in
|
||||
try? await Task.sleep(nanoseconds: UInt64(TransportConfig.pttBurstEndTimeoutSeconds * 1_000_000_000))
|
||||
guard !Task.isCancelled, let self, let assembly = self.assemblies[burstID] else { return }
|
||||
// Talker went silent/out of range without an END.
|
||||
self.finalize(assembly)
|
||||
}
|
||||
}
|
||||
|
||||
private func scheduleGapRedrain(for assembly: Assembly) {
|
||||
assembly.gapRedrain?.cancel()
|
||||
let burstID = assembly.burstID
|
||||
assembly.gapRedrain = Task { @MainActor [weak self] in
|
||||
try? await Task.sleep(nanoseconds: UInt64((Self.gapSkipSeconds + 0.05) * 1_000_000_000))
|
||||
guard !Task.isCancelled, let self, let assembly = self.assemblies[burstID] else { return }
|
||||
self.drainInOrder(assembly)
|
||||
self.finalizeIfComplete(assembly)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Helpers
|
||||
|
||||
private func pruneFinishedBursts() {
|
||||
let now = Date()
|
||||
finishedBursts = finishedBursts.filter { $0.value.expiresAt > now }
|
||||
while finishedBursts.count >= Self.finishedBurstsCap {
|
||||
guard let oldest = finishedBursts.min(by: { $0.value.expiresAt < $1.value.expiresAt }) else { break }
|
||||
finishedBursts.removeValue(forKey: oldest.key)
|
||||
}
|
||||
}
|
||||
|
||||
/// Extracts the 8-byte burst ID from a finalized note's file name
|
||||
/// (`voice_<16 hex>.m4a`, possibly uniquified by the incoming file store).
|
||||
static func burstID(fromVoiceFileName fileName: String) -> Data? {
|
||||
guard fileName.hasPrefix("voice_") else { return nil }
|
||||
let afterPrefix = fileName.dropFirst("voice_".count)
|
||||
let hex = String(afterPrefix.prefix(16))
|
||||
guard hex.count == 16, hex.allSatisfy(\.isHexDigit) else { return nil }
|
||||
return Data(hexString: hex)
|
||||
}
|
||||
|
||||
private static func makeIncomingURL(burstID: Data) -> URL? {
|
||||
guard let base = try? FileManager.default.url(
|
||||
for: .applicationSupportDirectory,
|
||||
in: .userDomainMask,
|
||||
appropriateFor: nil,
|
||||
create: true
|
||||
) else { return nil }
|
||||
let directory = base
|
||||
.appendingPathComponent("files", isDirectory: true)
|
||||
.appendingPathComponent("\(MimeType.Category.audio.mediaDir)/incoming", isDirectory: true)
|
||||
do {
|
||||
try FileManager.default.createDirectory(at: directory, withIntermediateDirectories: true, attributes: nil)
|
||||
} catch {
|
||||
return nil
|
||||
}
|
||||
return directory.appendingPathComponent("voice_live_\(burstID.hexEncodedString()).aac")
|
||||
}
|
||||
}
|
||||
@@ -72,6 +72,9 @@ protocol ChatTransportEventContext: AnyObject {
|
||||
func handleVerifyChallengePayload(from peerID: PeerID, payload: Data)
|
||||
func handleVerifyResponsePayload(from peerID: PeerID, payload: Data)
|
||||
|
||||
// MARK: Live voice (push-to-talk)
|
||||
func handleVoiceFramePayload(from peerID: PeerID, payload: Data, timestamp: Date)
|
||||
|
||||
// MARK: Group payloads (creator-signed state over Noise)
|
||||
func handleGroupInvitePayload(from peerID: PeerID, payload: Data)
|
||||
func handleGroupKeyUpdatePayload(from peerID: PeerID, payload: Data)
|
||||
@@ -135,6 +138,10 @@ extension ChatViewModel: ChatTransportEventContext {
|
||||
verificationCoordinator.handleVerifyResponsePayload(from: peerID, payload: payload)
|
||||
}
|
||||
|
||||
// `handleVoiceFramePayload(from:payload:timestamp:)` lives in
|
||||
// ChatViewModel+PrivateChat.swift next to the rest of the live-voice
|
||||
// surface.
|
||||
|
||||
func handleGroupInvitePayload(from peerID: PeerID, payload: Data) {
|
||||
groupCoordinator.handleGroupInvitePayload(from: peerID, payload: payload)
|
||||
}
|
||||
@@ -397,6 +404,9 @@ private extension ChatTransportEventCoordinator {
|
||||
|
||||
case .vouch:
|
||||
context.handleVouchPayload(from: peerID, payload: payload)
|
||||
|
||||
case .voiceFrame:
|
||||
context.handleVoiceFramePayload(from: peerID, payload: payload, timestamp: timestamp)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -178,6 +178,7 @@ final class ChatViewModel: ObservableObject, BitchatDelegate, TransportEventDele
|
||||
lazy var privateConversationCoordinator = ChatPrivateConversationCoordinator(context: self)
|
||||
lazy var nostrCoordinator = ChatNostrCoordinator(context: self)
|
||||
lazy var mediaTransferCoordinator = ChatMediaTransferCoordinator(context: self)
|
||||
lazy var liveVoiceCoordinator = ChatLiveVoiceCoordinator(context: self)
|
||||
lazy var verificationCoordinator = ChatVerificationCoordinator(context: self)
|
||||
lazy var groupCoordinator = ChatGroupCoordinator(context: self)
|
||||
lazy var vouchCoordinator = ChatVouchCoordinator(context: self)
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
//
|
||||
|
||||
import BitFoundation
|
||||
import BitLogger
|
||||
import Foundation
|
||||
import SwiftUI
|
||||
|
||||
@@ -69,6 +70,41 @@ extension ChatViewModel {
|
||||
mediaTransferCoordinator.sendVoiceNote(at: url)
|
||||
}
|
||||
|
||||
/// Picks the capture backend for the composer's hold-to-record gesture:
|
||||
/// live push-to-talk when the selected DM peer can hear it now (mesh
|
||||
/// reachable + established Noise session), otherwise the classic
|
||||
/// record-then-send voice note. Either way the release delivers a normal
|
||||
/// voice note through `sendVoiceNote(at:)`.
|
||||
@MainActor
|
||||
func makeVoiceCaptureSession() -> VoiceCaptureSession {
|
||||
guard PTTSettings.liveVoiceEnabled,
|
||||
let selectedPeer = selectedPrivateChatPeer,
|
||||
!selectedPeer.isGeoDM, !selectedPeer.isGeoChat, !selectedPeer.isGroup
|
||||
else {
|
||||
return VoiceNoteCaptureSession()
|
||||
}
|
||||
// A conversation can be selected under the stable 64-hex Noise key
|
||||
// (e.g. after migration on disconnect), but Noise sessions are keyed
|
||||
// by the 16-hex routing ID — normalize once and send to that same
|
||||
// short ID, like the private-message/file paths do.
|
||||
let peerID = selectedPeer.toShort()
|
||||
guard meshService.isPeerReachable(peerID),
|
||||
case .established = meshService.getNoiseSessionState(for: peerID)
|
||||
else {
|
||||
SecureLogger.debug("PTT: live unavailable for \(peerID.id.prefix(8))… (reachable=\(meshService.isPeerReachable(peerID))) — using classic voice note", category: .session)
|
||||
return VoiceNoteCaptureSession()
|
||||
}
|
||||
return PTTLiveVoiceSession(sendPacket: { [meshService] packet in
|
||||
meshService.sendVoiceFrame(packet, to: peerID)
|
||||
})
|
||||
}
|
||||
|
||||
/// Inbound handler for `NoisePayloadType.voiceFrame`.
|
||||
@MainActor
|
||||
func handleVoiceFramePayload(from peerID: PeerID, payload: Data, timestamp: Date) {
|
||||
liveVoiceCoordinator.handleVoiceFramePayload(from: peerID, payload: payload, timestamp: timestamp)
|
||||
}
|
||||
|
||||
#if os(iOS)
|
||||
func processThenSendImage(_ image: UIImage?) {
|
||||
mediaTransferCoordinator.processThenSendImage(image)
|
||||
@@ -129,6 +165,9 @@ extension ChatViewModel {
|
||||
|
||||
@MainActor
|
||||
func handlePrivateMessage(_ message: BitchatMessage) {
|
||||
// A finalized voice note whose burst already streamed in live swaps
|
||||
// into the existing bubble instead of appearing (and notifying) twice.
|
||||
if liveVoiceCoordinator.absorbFinalizedVoiceNote(message) { return }
|
||||
privateConversationCoordinator.handlePrivateMessage(message)
|
||||
}
|
||||
|
||||
|
||||
@@ -305,7 +305,9 @@ final class NostrInboundPipeline {
|
||||
context.handleReadReceipt(noisePayload, senderPubkey: senderPubkey, convKey: convKey)
|
||||
// Group state travels only over mesh Noise sessions in v1; anything
|
||||
// claiming to be group traffic over Nostr is ignored.
|
||||
case .verifyChallenge, .verifyResponse, .groupInvite, .groupKeyUpdate, .vouch:
|
||||
// Live voice is mesh-only: latency and relay cost make it
|
||||
// meaningless over Nostr.
|
||||
case .verifyChallenge, .verifyResponse, .groupInvite, .groupKeyUpdate, .vouch, .voiceFrame:
|
||||
break
|
||||
}
|
||||
}
|
||||
@@ -359,7 +361,9 @@ final class NostrInboundPipeline {
|
||||
context.handleReadReceipt(payload, senderPubkey: senderPubkey, convKey: convKey)
|
||||
// Group state travels only over mesh Noise sessions in v1; anything
|
||||
// claiming to be group traffic over Nostr is ignored.
|
||||
case .verifyChallenge, .verifyResponse, .groupInvite, .groupKeyUpdate, .vouch:
|
||||
// Live voice is mesh-only: latency and relay cost make it
|
||||
// meaningless over Nostr.
|
||||
case .verifyChallenge, .verifyResponse, .groupInvite, .groupKeyUpdate, .vouch, .voiceFrame:
|
||||
break
|
||||
}
|
||||
}
|
||||
@@ -440,7 +444,9 @@ final class NostrInboundPipeline {
|
||||
context.handleReadReceipt(payload, senderPubkey: senderPubkey, convKey: targetPeerID)
|
||||
// Group state travels only over mesh Noise sessions
|
||||
// in v1; group traffic over Nostr is ignored.
|
||||
case .verifyChallenge, .verifyResponse, .groupInvite, .groupKeyUpdate, .vouch:
|
||||
// Live voice is mesh-only: latency and relay cost make it
|
||||
// meaningless over Nostr.
|
||||
case .verifyChallenge, .verifyResponse, .groupInvite, .groupKeyUpdate, .vouch, .voiceFrame:
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
@@ -55,6 +55,14 @@ final class VoiceRecordingViewModel: ObservableObject {
|
||||
}
|
||||
|
||||
@Published private(set) var state = State.idle
|
||||
/// True while the active session streams audio live (push-to-talk); the
|
||||
/// composer switches its recording HUD to the LIVE treatment.
|
||||
@Published private(set) var isLiveStreaming = false
|
||||
|
||||
/// Supplies the capture backend per press. `ChatViewModel` swaps in a
|
||||
/// live push-to-talk session when the current DM peer can hear it now.
|
||||
var sessionProvider: () -> VoiceCaptureSession = { VoiceNoteCaptureSession() }
|
||||
private var activeSession: VoiceCaptureSession?
|
||||
|
||||
func formattedDuration(for date: Date) -> String {
|
||||
let clamped = max(0, state.duration(for: date))
|
||||
@@ -67,26 +75,54 @@ final class VoiceRecordingViewModel: ObservableObject {
|
||||
|
||||
func start(shouldShow: Bool) {
|
||||
guard shouldShow, state == .idle else { return }
|
||||
let session = sessionProvider()
|
||||
activeSession = session
|
||||
state = .requestingPermission
|
||||
Task {
|
||||
let granted = await VoiceRecorder.shared.requestPermission()
|
||||
guard state == .requestingPermission else { return }
|
||||
let granted = await session.requestPermission()
|
||||
guard state == .requestingPermission, activeSession === session else { return }
|
||||
guard granted else {
|
||||
state = .permissionDenied
|
||||
activeSession = nil
|
||||
return
|
||||
}
|
||||
state = .preparing
|
||||
do {
|
||||
try await VoiceRecorder.shared.startRecording()
|
||||
guard state == .preparing else {
|
||||
cancel()
|
||||
try await session.start()
|
||||
guard state == .preparing, activeSession === session else {
|
||||
await session.cancel()
|
||||
return
|
||||
}
|
||||
state = .recording(startDate: Date())
|
||||
isLiveStreaming = session.isLive
|
||||
} catch {
|
||||
SecureLogger.error("Voice recording failed to start: \(error)", category: .session)
|
||||
await VoiceRecorder.shared.cancelRecording()
|
||||
guard state == .preparing else { return }
|
||||
await session.cancel()
|
||||
guard state == .preparing, activeSession === session else { return }
|
||||
// The live engine and the classic recorder are separate
|
||||
// capture stacks: when the live one hits an audio-route
|
||||
// glitch, fall back within the same hold so the user still
|
||||
// gets a voice note instead of an error.
|
||||
if session.isLive {
|
||||
let fallback = VoiceNoteCaptureSession()
|
||||
activeSession = fallback
|
||||
do {
|
||||
try await fallback.start()
|
||||
guard state == .preparing, activeSession === fallback else {
|
||||
await fallback.cancel()
|
||||
return
|
||||
}
|
||||
SecureLogger.warning("PTT: live capture failed — fell back to classic voice note", category: .session)
|
||||
state = .recording(startDate: Date())
|
||||
isLiveStreaming = false
|
||||
return
|
||||
} catch {
|
||||
SecureLogger.error("Voice recording fallback failed to start: \(error)", category: .session)
|
||||
await fallback.cancel()
|
||||
guard state == .preparing else { return }
|
||||
}
|
||||
}
|
||||
activeSession = nil
|
||||
state = .error(message: "Could not start recording.")
|
||||
}
|
||||
}
|
||||
@@ -103,15 +139,18 @@ final class VoiceRecordingViewModel: ObservableObject {
|
||||
}
|
||||
|
||||
state = .idle
|
||||
isLiveStreaming = false
|
||||
let session = activeSession
|
||||
activeSession = nil
|
||||
|
||||
guard case .recording(let startDate) = previousState, let completion else {
|
||||
Task { await VoiceRecorder.shared.cancelRecording() }
|
||||
guard case .recording(let startDate) = previousState, let completion, let session else {
|
||||
Task { await session?.cancel() }
|
||||
return
|
||||
}
|
||||
|
||||
Task {
|
||||
let finalDuration = Date().timeIntervalSince(startDate)
|
||||
if let url = await VoiceRecorder.shared.stopRecording(),
|
||||
if let url = await session.finish(),
|
||||
isValidRecording(at: url, duration: finalDuration) {
|
||||
completion(url)
|
||||
} else {
|
||||
|
||||
@@ -9,6 +9,7 @@ struct AppInfoView: View {
|
||||
/// hides the topology row entirely.
|
||||
var topologyProvider: (@MainActor () -> MeshTopologyDisplayModel)?
|
||||
@State private var showTopology = false
|
||||
@State private var liveVoiceEnabled = PTTSettings.liveVoiceEnabled
|
||||
|
||||
private var selectedTheme: AppTheme {
|
||||
AppTheme(rawValue: appThemeRawValue) ?? .matrix
|
||||
@@ -80,6 +81,15 @@ struct AppInfoView: View {
|
||||
]
|
||||
}
|
||||
|
||||
enum Voice {
|
||||
static let title: LocalizedStringKey = "app_info.voice.title"
|
||||
static let live = AppInfoFeatureInfo(
|
||||
icon: "dot.radiowaves.left.and.right",
|
||||
title: "app_info.voice.live.title",
|
||||
description: "app_info.voice.live.description"
|
||||
)
|
||||
}
|
||||
|
||||
enum Network {
|
||||
static let title: LocalizedStringKey = "app_info.network.title"
|
||||
static let topology = AppInfoFeatureInfo(
|
||||
@@ -223,6 +233,21 @@ struct AppInfoView: View {
|
||||
.foregroundColor(textColor)
|
||||
}
|
||||
|
||||
// Voice
|
||||
VStack(alignment: .leading, spacing: 16) {
|
||||
SectionHeader(Strings.Voice.title)
|
||||
|
||||
HStack(spacing: 0) {
|
||||
FeatureRow(info: Strings.Voice.live)
|
||||
Toggle(Strings.Voice.live.title, isOn: $liveVoiceEnabled)
|
||||
.labelsHidden()
|
||||
.tint(palette.accent)
|
||||
.onChange(of: liveVoiceEnabled) { newValue in
|
||||
PTTSettings.liveVoiceEnabled = newValue
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Network diagnostics
|
||||
if topologyProvider != nil {
|
||||
VStack(alignment: .leading, spacing: 16) {
|
||||
|
||||
@@ -137,16 +137,28 @@ private extension ContentComposerView {
|
||||
|
||||
var recordingIndicator: some View {
|
||||
HStack(spacing: 12) {
|
||||
Image(systemName: "waveform.circle.fill")
|
||||
Image(systemName: voiceRecordingVM.isLiveStreaming ? "dot.radiowaves.left.and.right" : "waveform.circle.fill")
|
||||
.foregroundColor(.red)
|
||||
.font(.bitchatSystem(size: 20))
|
||||
.modifier(PulsingOpacityModifier(active: voiceRecordingVM.isLiveStreaming))
|
||||
TimelineView(.periodic(from: .now, by: 0.05)) { context in
|
||||
Text(
|
||||
"recording \(voiceRecordingVM.formattedDuration(for: context.date))",
|
||||
comment: "Voice note recording duration indicator"
|
||||
)
|
||||
.bitchatFont(size: 13)
|
||||
.foregroundColor(.red)
|
||||
// Live streaming means audio is heard as you speak — the HUD
|
||||
// must make that unmistakable, not just show a timer.
|
||||
if voiceRecordingVM.isLiveStreaming {
|
||||
Text(
|
||||
"live \(voiceRecordingVM.formattedDuration(for: context.date))",
|
||||
comment: "Recording HUD label while a voice message streams live to the recipient"
|
||||
)
|
||||
.bitchatFont(size: 13, weight: .bold)
|
||||
.foregroundColor(.red)
|
||||
} else {
|
||||
Text(
|
||||
"recording \(voiceRecordingVM.formattedDuration(for: context.date))",
|
||||
comment: "Voice note recording duration indicator"
|
||||
)
|
||||
.bitchatFont(size: 13)
|
||||
.foregroundColor(.red)
|
||||
}
|
||||
}
|
||||
Spacer()
|
||||
Button(action: voiceRecordingVM.cancel) {
|
||||
|
||||
@@ -75,6 +75,9 @@ struct ContentView: View {
|
||||
.onAppear {
|
||||
conversationUIModel.setCurrentColorScheme(colorScheme)
|
||||
conversationUIModel.setCurrentTheme(appTheme)
|
||||
voiceRecordingVM.sessionProvider = { [weak conversationUIModel] in
|
||||
conversationUIModel?.makeVoiceCaptureSession() ?? VoiceNoteCaptureSession()
|
||||
}
|
||||
#if os(macOS)
|
||||
DispatchQueue.main.async {
|
||||
isNicknameFieldFocused = false
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
//
|
||||
// LiveVoiceBadge.swift
|
||||
// bitchat
|
||||
//
|
||||
// This is free and unencumbered software released into the public domain.
|
||||
// For more information, see <https://unlicense.org>
|
||||
//
|
||||
|
||||
import SwiftUI
|
||||
|
||||
/// Slow opacity pulse for live-voice indicators (composer HUD, bubble badge).
|
||||
struct PulsingOpacityModifier: ViewModifier {
|
||||
let active: Bool
|
||||
@State private var dimmed = false
|
||||
|
||||
func body(content: Content) -> some View {
|
||||
content
|
||||
.opacity(active && dimmed ? 0.35 : 1)
|
||||
.animation(active ? .easeInOut(duration: 0.7).repeatForever(autoreverses: true) : .default, value: dimmed)
|
||||
.onAppear {
|
||||
if active { dimmed = true }
|
||||
}
|
||||
.onChange(of: active) { nowActive in
|
||||
dimmed = nowActive
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The red pulsing "LIVE" chip shown on a voice bubble while its burst is
|
||||
/// still streaming in.
|
||||
struct LiveVoiceBadge: View {
|
||||
var body: some View {
|
||||
HStack(spacing: 4) {
|
||||
Circle()
|
||||
.fill(Color.red)
|
||||
.frame(width: 6, height: 6)
|
||||
Text("media.voice.live_badge", comment: "Badge on a voice message that is currently streaming in live")
|
||||
.bitchatFont(size: 10, weight: .bold)
|
||||
.foregroundColor(.red)
|
||||
}
|
||||
.padding(.horizontal, 6)
|
||||
.padding(.vertical, 3)
|
||||
.background(
|
||||
Capsule().fill(Color.red.opacity(0.15))
|
||||
)
|
||||
.modifier(PulsingOpacityModifier(active: true))
|
||||
.accessibilityLabel(
|
||||
String(localized: "media.voice.accessibility.live", comment: "Accessibility label announcing a live incoming voice message")
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -96,6 +96,7 @@ struct MediaMessageView: View {
|
||||
url: url,
|
||||
isSending: state.isSending,
|
||||
sendProgress: state.progress,
|
||||
isLive: conversationUIModel.isLiveVoiceMessage(message),
|
||||
onCancel: cancelAction
|
||||
)
|
||||
case .image(let url):
|
||||
|
||||
@@ -5,6 +5,7 @@ struct VoiceNoteView: View {
|
||||
private let url: URL
|
||||
private let isSending: Bool
|
||||
private let sendProgress: Double?
|
||||
private let isLive: Bool
|
||||
private let onCancel: (() -> Void)?
|
||||
|
||||
@Environment(\.colorScheme) private var colorScheme
|
||||
@@ -12,10 +13,11 @@ struct VoiceNoteView: View {
|
||||
@StateObject private var playback: VoiceNotePlaybackController
|
||||
@State private var waveform: [Float] = []
|
||||
|
||||
init(url: URL, isSending: Bool, sendProgress: Double?, onCancel: (() -> Void)?) {
|
||||
init(url: URL, isSending: Bool, sendProgress: Double?, isLive: Bool = false, onCancel: (() -> Void)?) {
|
||||
self.url = url
|
||||
self.isSending = isSending
|
||||
self.sendProgress = sendProgress
|
||||
self.isLive = isLive
|
||||
self.onCancel = onCancel
|
||||
_playback = StateObject(wrappedValue: VoiceNotePlaybackController(url: url))
|
||||
}
|
||||
@@ -69,9 +71,13 @@ struct VoiceNoteView: View {
|
||||
isInteractive: playback.isPlaying
|
||||
)
|
||||
|
||||
Text(playbackLabel)
|
||||
.bitchatFont(size: 13)
|
||||
.foregroundColor(palette.secondary)
|
||||
if isLive {
|
||||
LiveVoiceBadge()
|
||||
} else {
|
||||
Text(playbackLabel)
|
||||
.bitchatFont(size: 13)
|
||||
.foregroundColor(palette.secondary)
|
||||
}
|
||||
|
||||
if let onCancel = onCancel, isSending {
|
||||
Button(action: onCancel) {
|
||||
|
||||
@@ -0,0 +1,241 @@
|
||||
//
|
||||
// ChatLiveVoiceCoordinatorTests.swift
|
||||
// bitchatTests
|
||||
//
|
||||
// This is free and unencumbered software released into the public domain.
|
||||
// For more information, see <https://unlicense.org>
|
||||
//
|
||||
|
||||
import Testing
|
||||
import Foundation
|
||||
import BitFoundation
|
||||
@testable import bitchat
|
||||
|
||||
@MainActor
|
||||
private final class MockChatLiveVoiceContext: ChatLiveVoiceContext {
|
||||
var nickname = "me"
|
||||
var selectedPrivateChatPeer: PeerID?
|
||||
var blockedPeers: Set<PeerID> = []
|
||||
|
||||
private(set) var handledPrivateMessages: [BitchatMessage] = []
|
||||
private(set) var upsertedMessages: [(message: BitchatMessage, peerID: PeerID)] = []
|
||||
private(set) var removedMessageIDs: [String] = []
|
||||
|
||||
func isPeerBlocked(_ peerID: PeerID) -> Bool { blockedPeers.contains(peerID) }
|
||||
func resolveNickname(for peerID: PeerID) -> String { "alice" }
|
||||
func handlePrivateMessage(_ message: BitchatMessage) { handledPrivateMessages.append(message) }
|
||||
func upsertPrivateMessage(_ message: BitchatMessage, in peerID: PeerID) {
|
||||
upsertedMessages.append((message, peerID))
|
||||
}
|
||||
@discardableResult
|
||||
func removePrivateMessage(withID messageID: String) -> BitchatMessage? {
|
||||
removedMessageIDs.append(messageID)
|
||||
return nil
|
||||
}
|
||||
func notifyUIChanged() {}
|
||||
}
|
||||
|
||||
@MainActor
|
||||
struct ChatLiveVoiceCoordinatorTests {
|
||||
private let peer = PeerID(str: "aaaabbbbcccc0001")
|
||||
|
||||
private func makeBurstID(_ fill: UInt8) -> Data {
|
||||
Data(repeating: fill, count: VoiceBurstPacket.burstIDSize)
|
||||
}
|
||||
|
||||
private func send(_ packet: VoiceBurstPacket, to coordinator: ChatLiveVoiceCoordinator, from peerID: PeerID) {
|
||||
coordinator.handleVoiceFramePayload(from: peerID, payload: packet.encode(), timestamp: Date())
|
||||
}
|
||||
|
||||
private func incomingFileURL(burstID: Data) -> URL? {
|
||||
guard let base = try? FileManager.default.url(
|
||||
for: .applicationSupportDirectory, in: .userDomainMask, appropriateFor: nil, create: false
|
||||
) else { return nil }
|
||||
return base
|
||||
.appendingPathComponent("files/voicenotes/incoming", isDirectory: true)
|
||||
.appendingPathComponent("voice_live_\(burstID.hexEncodedString()).aac")
|
||||
}
|
||||
|
||||
@Test func burstCreatesBubbleAndPersistsFramesInOrder() throws {
|
||||
let context = MockChatLiveVoiceContext()
|
||||
let coordinator = ChatLiveVoiceCoordinator(context: context)
|
||||
let burstID = makeBurstID(0xA1)
|
||||
defer { incomingFileURL(burstID: burstID).map { try? FileManager.default.removeItem(at: $0) } }
|
||||
|
||||
let frame1 = Data(repeating: 0x01, count: 60)
|
||||
let frame2 = Data(repeating: 0x02, count: 60)
|
||||
|
||||
send(try #require(VoiceBurstPacket(burstID: burstID, seq: 0, kind: .start(codec: .aacLC16kMono))), to: coordinator, from: peer)
|
||||
#expect(context.handledPrivateMessages.count == 1)
|
||||
let bubble = try #require(context.handledPrivateMessages.first)
|
||||
#expect(bubble.isPrivate)
|
||||
#expect(bubble.senderPeerID == peer)
|
||||
#expect(bubble.content == "[voice] voice_live_\(burstID.hexEncodedString()).aac")
|
||||
#expect(coordinator.isLiveVoiceMessage(bubble))
|
||||
|
||||
// Deliver out of order: seq 2 buffers behind the seq-1 hole, then
|
||||
// seq 1 releases both in order.
|
||||
send(try #require(VoiceBurstPacket(burstID: burstID, seq: 2, kind: .frames([frame2]))), to: coordinator, from: peer)
|
||||
send(try #require(VoiceBurstPacket(burstID: burstID, seq: 1, kind: .frames([frame1]))), to: coordinator, from: peer)
|
||||
send(try #require(VoiceBurstPacket(burstID: burstID, seq: 3, kind: .end(totalDataPackets: 2, durationMs: 128))), to: coordinator, from: peer)
|
||||
|
||||
let url = try #require(incomingFileURL(burstID: burstID))
|
||||
let written = try Data(contentsOf: url)
|
||||
var expected = ADTSFramer.frame(frame1)
|
||||
expected.append(ADTSFramer.frame(frame2))
|
||||
#expect(written == expected)
|
||||
|
||||
// Burst ended: no longer live, bubble republished for re-render.
|
||||
#expect(!coordinator.isLiveVoiceMessage(bubble))
|
||||
#expect(context.upsertedMessages.contains { $0.message.id == bubble.id })
|
||||
#expect(context.removedMessageIDs.isEmpty)
|
||||
}
|
||||
|
||||
@Test func absorbsFinalizedNoteIntoLiveBubble() throws {
|
||||
let context = MockChatLiveVoiceContext()
|
||||
let coordinator = ChatLiveVoiceCoordinator(context: context)
|
||||
let burstID = makeBurstID(0xB2)
|
||||
let hex = burstID.hexEncodedString()
|
||||
|
||||
send(try #require(VoiceBurstPacket(burstID: burstID, seq: 1, kind: .frames([Data(repeating: 7, count: 50)]))), to: coordinator, from: peer)
|
||||
send(try #require(VoiceBurstPacket(burstID: burstID, seq: 2, kind: .end(totalDataPackets: 1, durationMs: 64))), to: coordinator, from: peer)
|
||||
let bubble = try #require(context.handledPrivateMessages.first)
|
||||
|
||||
let note = BitchatMessage(
|
||||
sender: "alice",
|
||||
content: "[voice] voice_\(hex).m4a",
|
||||
timestamp: Date(),
|
||||
isRelay: false,
|
||||
isPrivate: true,
|
||||
recipientNickname: "me",
|
||||
senderPeerID: peer
|
||||
)
|
||||
#expect(coordinator.absorbFinalizedVoiceNote(note))
|
||||
|
||||
// The note replaced the live bubble in place: same message ID, new
|
||||
// content, partial capture deleted.
|
||||
let replacement = try #require(context.upsertedMessages.last)
|
||||
#expect(replacement.message.id == bubble.id)
|
||||
#expect(replacement.message.content == note.content)
|
||||
#expect(replacement.peerID == peer)
|
||||
let url = try #require(incomingFileURL(burstID: burstID))
|
||||
#expect(!FileManager.default.fileExists(atPath: url.path))
|
||||
|
||||
// Absorption is one-shot.
|
||||
#expect(!coordinator.absorbFinalizedVoiceNote(note))
|
||||
}
|
||||
|
||||
@Test func absorbIgnoresUnrelatedVoiceNotes() throws {
|
||||
let context = MockChatLiveVoiceContext()
|
||||
let coordinator = ChatLiveVoiceCoordinator(context: context)
|
||||
|
||||
// A classic voice note (date-stamped name) and a live-capture name
|
||||
// must both pass through untouched.
|
||||
let classic = BitchatMessage(
|
||||
sender: "alice", content: "[voice] voice_20260708_1201.m4a", timestamp: Date(),
|
||||
isRelay: false, isPrivate: true, recipientNickname: "me", senderPeerID: peer
|
||||
)
|
||||
#expect(!coordinator.absorbFinalizedVoiceNote(classic))
|
||||
let liveCapture = BitchatMessage(
|
||||
sender: "alice", content: "[voice] voice_live_aabbccdd00112233.aac", timestamp: Date(),
|
||||
isRelay: false, isPrivate: true, recipientNickname: "me", senderPeerID: peer
|
||||
)
|
||||
#expect(!coordinator.absorbFinalizedVoiceNote(liveCapture))
|
||||
// Unknown burst ID.
|
||||
let unknown = BitchatMessage(
|
||||
sender: "alice", content: "[voice] voice_ffffffffffffffff.m4a", timestamp: Date(),
|
||||
isRelay: false, isPrivate: true, recipientNickname: "me", senderPeerID: peer
|
||||
)
|
||||
#expect(!coordinator.absorbFinalizedVoiceNote(unknown))
|
||||
}
|
||||
|
||||
@Test func canceledBurstRemovesBubbleAndFile() throws {
|
||||
let context = MockChatLiveVoiceContext()
|
||||
let coordinator = ChatLiveVoiceCoordinator(context: context)
|
||||
let burstID = makeBurstID(0xC3)
|
||||
|
||||
send(try #require(VoiceBurstPacket(burstID: burstID, seq: 1, kind: .frames([Data(repeating: 9, count: 40)]))), to: coordinator, from: peer)
|
||||
let bubble = try #require(context.handledPrivateMessages.first)
|
||||
send(try #require(VoiceBurstPacket(burstID: burstID, seq: 2, kind: .canceled)), to: coordinator, from: peer)
|
||||
|
||||
#expect(context.removedMessageIDs == [bubble.id])
|
||||
let url = try #require(incomingFileURL(burstID: burstID))
|
||||
#expect(!FileManager.default.fileExists(atPath: url.path))
|
||||
#expect(!coordinator.isLiveVoiceMessage(bubble))
|
||||
}
|
||||
|
||||
@Test func emptyBurstLeavesNoBubble() throws {
|
||||
let context = MockChatLiveVoiceContext()
|
||||
let coordinator = ChatLiveVoiceCoordinator(context: context)
|
||||
let burstID = makeBurstID(0xD4)
|
||||
|
||||
send(try #require(VoiceBurstPacket(burstID: burstID, seq: 0, kind: .start(codec: .aacLC16kMono))), to: coordinator, from: peer)
|
||||
let bubble = try #require(context.handledPrivateMessages.first)
|
||||
send(try #require(VoiceBurstPacket(burstID: burstID, seq: 1, kind: .end(totalDataPackets: 0, durationMs: 0))), to: coordinator, from: peer)
|
||||
|
||||
// Nothing audible arrived: the placeholder bubble is withdrawn.
|
||||
#expect(context.removedMessageIDs == [bubble.id])
|
||||
}
|
||||
|
||||
@Test func ignoresBlockedPeersAndUnknownControlPackets() throws {
|
||||
let context = MockChatLiveVoiceContext()
|
||||
let coordinator = ChatLiveVoiceCoordinator(context: context)
|
||||
|
||||
context.blockedPeers = [peer]
|
||||
send(try #require(VoiceBurstPacket(burstID: makeBurstID(0xE5), seq: 0, kind: .start(codec: .aacLC16kMono))), to: coordinator, from: peer)
|
||||
#expect(context.handledPrivateMessages.isEmpty)
|
||||
|
||||
context.blockedPeers = []
|
||||
// END/CANCELED for a burst that never started must not create state.
|
||||
send(try #require(VoiceBurstPacket(burstID: makeBurstID(0xE6), seq: 5, kind: .end(totalDataPackets: 4, durationMs: 256))), to: coordinator, from: peer)
|
||||
send(try #require(VoiceBurstPacket(burstID: makeBurstID(0xE7), seq: 5, kind: .canceled)), to: coordinator, from: peer)
|
||||
#expect(context.handledPrivateMessages.isEmpty)
|
||||
}
|
||||
|
||||
@Test func concurrentAssemblyCapDropsExtraBursts() throws {
|
||||
let context = MockChatLiveVoiceContext()
|
||||
let coordinator = ChatLiveVoiceCoordinator(context: context)
|
||||
|
||||
var cleanup: [Data] = []
|
||||
defer {
|
||||
for burstID in cleanup {
|
||||
incomingFileURL(burstID: burstID).map { try? FileManager.default.removeItem(at: $0) }
|
||||
}
|
||||
}
|
||||
for i in 0..<TransportConfig.pttMaxConcurrentAssemblies {
|
||||
let burstID = makeBurstID(UInt8(0x10 + i))
|
||||
cleanup.append(burstID)
|
||||
send(try #require(VoiceBurstPacket(burstID: burstID, seq: 0, kind: .start(codec: .aacLC16kMono))), to: coordinator, from: peer)
|
||||
}
|
||||
#expect(context.handledPrivateMessages.count == TransportConfig.pttMaxConcurrentAssemblies)
|
||||
|
||||
send(try #require(VoiceBurstPacket(burstID: makeBurstID(0xFF), seq: 0, kind: .start(codec: .aacLC16kMono))), to: coordinator, from: peer)
|
||||
#expect(context.handledPrivateMessages.count == TransportConfig.pttMaxConcurrentAssemblies)
|
||||
}
|
||||
|
||||
@Test func liveVoiceToggleOffDropsInboundFrames() throws {
|
||||
let previous = PTTSettings.liveVoiceEnabled
|
||||
PTTSettings.liveVoiceEnabled = false
|
||||
defer { PTTSettings.liveVoiceEnabled = previous }
|
||||
|
||||
let context = MockChatLiveVoiceContext()
|
||||
let coordinator = ChatLiveVoiceCoordinator(context: context)
|
||||
let burstID = makeBurstID(0xE8)
|
||||
|
||||
// Off means classic-notes-only: no live bubble, no partial file.
|
||||
send(try #require(VoiceBurstPacket(burstID: burstID, seq: 0, kind: .start(codec: .aacLC16kMono))), to: coordinator, from: peer)
|
||||
send(try #require(VoiceBurstPacket(burstID: burstID, seq: 1, kind: .frames([Data(repeating: 5, count: 40)]))), to: coordinator, from: peer)
|
||||
#expect(context.handledPrivateMessages.isEmpty)
|
||||
let url = try #require(incomingFileURL(burstID: burstID))
|
||||
#expect(!FileManager.default.fileExists(atPath: url.path))
|
||||
}
|
||||
|
||||
@Test func burstIDParsingFromFileNames() {
|
||||
#expect(ChatLiveVoiceCoordinator.burstID(fromVoiceFileName: "voice_00112233445566ff.m4a") == Data(hexString: "00112233445566ff"))
|
||||
// Uniquified copies keep the leading hex run.
|
||||
#expect(ChatLiveVoiceCoordinator.burstID(fromVoiceFileName: "voice_00112233445566ff (1).m4a") == Data(hexString: "00112233445566ff"))
|
||||
#expect(ChatLiveVoiceCoordinator.burstID(fromVoiceFileName: "voice_20260708_120000.m4a") == nil)
|
||||
#expect(ChatLiveVoiceCoordinator.burstID(fromVoiceFileName: "voice_live_00112233445566ff.aac") == nil)
|
||||
#expect(ChatLiveVoiceCoordinator.burstID(fromVoiceFileName: "other.m4a") == nil)
|
||||
}
|
||||
}
|
||||
@@ -174,6 +174,13 @@ private final class MockChatTransportEventContext: ChatTransportEventContext {
|
||||
func handleVouchPayload(from peerID: PeerID, payload: Data) {
|
||||
vouchPayloads.append((peerID, payload))
|
||||
}
|
||||
|
||||
// Live voice payloads
|
||||
private(set) var voiceFramePayloads: [(peerID: PeerID, payload: Data, timestamp: Date)] = []
|
||||
|
||||
func handleVoiceFramePayload(from peerID: PeerID, payload: Data, timestamp: Date) {
|
||||
voiceFramePayloads.append((peerID, payload, timestamp))
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Helpers
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
//
|
||||
// PTTAudioTests.swift
|
||||
// bitchatTests
|
||||
//
|
||||
// This is free and unencumbered software released into the public domain.
|
||||
// For more information, see <https://unlicense.org>
|
||||
//
|
||||
|
||||
import Testing
|
||||
import AVFoundation
|
||||
import Foundation
|
||||
@testable import bitchat
|
||||
|
||||
struct PTTAudioTests {
|
||||
// MARK: - ADTS framing
|
||||
|
||||
@Test func adtsHeaderEncodesFrameLengthAndFormat() {
|
||||
let payload = Data(repeating: 0xAB, count: 100)
|
||||
let framed = ADTSFramer.frame(payload)
|
||||
#expect(framed.count == 107)
|
||||
|
||||
// Syncword + MPEG-4 + layer 00 + no CRC.
|
||||
#expect(framed[0] == 0xFF)
|
||||
#expect(framed[1] == 0xF1)
|
||||
// AAC-LC (01), sampling index 8 (16 kHz), channel config 1.
|
||||
#expect(framed[2] == 0x60)
|
||||
#expect(framed[3] == 0x40 | UInt8((107 >> 11) & 0x3))
|
||||
#expect(framed[4] == UInt8((107 >> 3) & 0xFF))
|
||||
#expect(framed[5] == UInt8((107 & 0x7) << 5) | 0x1F)
|
||||
#expect(framed[6] == 0xFC)
|
||||
#expect(Data(framed.dropFirst(7)) == payload)
|
||||
}
|
||||
|
||||
@Test func adtsStreamIsReadableByCoreAudio() throws {
|
||||
// A receiver persists bursts as ADTS .aac; the file must be openable
|
||||
// by the same machinery the voice-note UI uses (AVAudioFile).
|
||||
let frames = try encodeSineFrames(seconds: 0.5)
|
||||
#expect(frames.count >= 4)
|
||||
|
||||
let url = FileManager.default.temporaryDirectory
|
||||
.appendingPathComponent("ptt-test-\(UUID().uuidString).aac")
|
||||
defer { try? FileManager.default.removeItem(at: url) }
|
||||
var stream = Data()
|
||||
for frame in frames {
|
||||
stream.append(ADTSFramer.frame(frame))
|
||||
}
|
||||
try stream.write(to: url)
|
||||
|
||||
let file = try AVAudioFile(forReading: url)
|
||||
#expect(file.length > 0)
|
||||
}
|
||||
|
||||
// MARK: - Codec round trip
|
||||
|
||||
@Test func encoderProducesRealtimeSizedFrames() throws {
|
||||
let frames = try encodeSineFrames(seconds: 1.0)
|
||||
// 1 s of 64 ms frames ≈ 15 (allow encoder priming slack).
|
||||
#expect(frames.count >= 10)
|
||||
// ~16 kbps -> ~130 bytes/frame; all frames must fit the wire budget.
|
||||
for frame in frames {
|
||||
#expect(frame.count > 0)
|
||||
#expect(frame.count < TransportConfig.pttMaxBurstContentBytes)
|
||||
}
|
||||
}
|
||||
|
||||
@Test func decoderRoundTripsEncodedAudio() throws {
|
||||
let frames = try encodeSineFrames(seconds: 0.5)
|
||||
let decoder = try #require(PTTFrameDecoder())
|
||||
|
||||
var decodedSamples = 0
|
||||
var energy: Float = 0
|
||||
for frame in frames {
|
||||
guard let pcm = decoder.decode(frame) else { continue } // priming
|
||||
decodedSamples += Int(pcm.frameLength)
|
||||
if let channel = pcm.floatChannelData?[0] {
|
||||
for i in 0..<Int(pcm.frameLength) {
|
||||
energy += abs(channel[i])
|
||||
}
|
||||
}
|
||||
}
|
||||
// Most of the audio must decode, and it must not be silence.
|
||||
#expect(decodedSamples >= 4 * Int(PTTAudioFormat.samplesPerFrame))
|
||||
#expect(energy > 1)
|
||||
}
|
||||
|
||||
// MARK: - Helpers
|
||||
|
||||
private func encodeSineFrames(seconds: Double) throws -> [Data] {
|
||||
let encoder = try #require(PTTFrameEncoder())
|
||||
let format = try #require(PTTAudioFormat.pcmFormat)
|
||||
let totalFrames = AVAudioFrameCount(seconds * PTTAudioFormat.sampleRate)
|
||||
let buffer = try #require(AVAudioPCMBuffer(pcmFormat: format, frameCapacity: totalFrames))
|
||||
buffer.frameLength = totalFrames
|
||||
let channel = try #require(buffer.floatChannelData?[0])
|
||||
for i in 0..<Int(totalFrames) {
|
||||
channel[i] = sinf(2 * .pi * 440 * Float(i) / Float(PTTAudioFormat.sampleRate)) * 0.5
|
||||
}
|
||||
return encoder.encode(buffer)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,161 @@
|
||||
//
|
||||
// VoiceBurstPacketTests.swift
|
||||
// bitchatTests
|
||||
//
|
||||
// This is free and unencumbered software released into the public domain.
|
||||
// For more information, see <https://unlicense.org>
|
||||
//
|
||||
|
||||
import Testing
|
||||
import Foundation
|
||||
@testable import bitchat
|
||||
|
||||
struct VoiceBurstPacketTests {
|
||||
private let burstID = Data((0..<8).map { UInt8($0 + 1) })
|
||||
|
||||
// MARK: - Round trips
|
||||
|
||||
@Test func startRoundTrip() throws {
|
||||
let packet = try #require(VoiceBurstPacket(burstID: burstID, seq: 0, kind: .start(codec: .aacLC16kMono)))
|
||||
let decoded = try #require(VoiceBurstPacket.decode(packet.encode()))
|
||||
#expect(decoded == packet)
|
||||
#expect(decoded.seq == 0)
|
||||
}
|
||||
|
||||
@Test func framesRoundTrip() throws {
|
||||
let frames = [Data([0xDE, 0xAD]), Data(repeating: 0x42, count: 130)]
|
||||
let packet = try #require(VoiceBurstPacket(burstID: burstID, seq: 7, kind: .frames(frames)))
|
||||
let decoded = try #require(VoiceBurstPacket.decode(packet.encode()))
|
||||
#expect(decoded == packet)
|
||||
guard case .frames(let decodedFrames) = decoded.kind else {
|
||||
Issue.record("expected frames")
|
||||
return
|
||||
}
|
||||
#expect(decodedFrames == frames)
|
||||
}
|
||||
|
||||
@Test func endRoundTrip() throws {
|
||||
let packet = try #require(VoiceBurstPacket(burstID: burstID, seq: 42, kind: .end(totalDataPackets: 41, durationMs: 2_688)))
|
||||
let decoded = try #require(VoiceBurstPacket.decode(packet.encode()))
|
||||
#expect(decoded == packet)
|
||||
}
|
||||
|
||||
@Test func canceledRoundTrip() throws {
|
||||
let packet = try #require(VoiceBurstPacket(burstID: burstID, seq: 3, kind: .canceled))
|
||||
let decoded = try #require(VoiceBurstPacket.decode(packet.encode()))
|
||||
#expect(decoded == packet)
|
||||
}
|
||||
|
||||
@Test func decodeSurvivesReslicedData() throws {
|
||||
// Simulates the payload arriving as a slice with a non-zero start index.
|
||||
let packet = try #require(VoiceBurstPacket(burstID: burstID, seq: 1, kind: .frames([Data([1, 2, 3])])))
|
||||
var padded = Data([0xAA, 0xBB])
|
||||
padded.append(packet.encode())
|
||||
let slice = padded.dropFirst(2)
|
||||
#expect(VoiceBurstPacket.decode(slice) == packet)
|
||||
}
|
||||
|
||||
// MARK: - Validation
|
||||
|
||||
@Test func rejectsMalformedInput() {
|
||||
#expect(VoiceBurstPacket.decode(Data()) == nil)
|
||||
#expect(VoiceBurstPacket.decode(Data(repeating: 0, count: 10)) == nil)
|
||||
// Unknown flags byte.
|
||||
var unknownFlags = burstID
|
||||
unknownFlags.append(contentsOf: [0, 1, 0xFF])
|
||||
#expect(VoiceBurstPacket.decode(unknownFlags) == nil)
|
||||
// Data packet with zero frames.
|
||||
var empty = burstID
|
||||
empty.append(contentsOf: [0, 1, 0])
|
||||
#expect(VoiceBurstPacket.decode(empty) == nil)
|
||||
// Truncated frame length.
|
||||
var truncated = burstID
|
||||
truncated.append(contentsOf: [0, 1, 0, 0x00, 0x10, 0xAB])
|
||||
#expect(VoiceBurstPacket.decode(truncated) == nil)
|
||||
// Unknown codec in START.
|
||||
var badCodec = burstID
|
||||
badCodec.append(contentsOf: [0, 0, 0x01, 0x7F])
|
||||
#expect(VoiceBurstPacket.decode(badCodec) == nil)
|
||||
}
|
||||
|
||||
@Test func rejectsInvalidConstruction() {
|
||||
#expect(VoiceBurstPacket(burstID: Data([1, 2]), seq: 0, kind: .canceled) == nil)
|
||||
#expect(VoiceBurstPacket(burstID: burstID, seq: 1, kind: .frames([])) == nil)
|
||||
#expect(VoiceBurstPacket(burstID: burstID, seq: 1, kind: .frames([Data()])) == nil)
|
||||
let tooMany = Array(repeating: Data([0x01]), count: VoiceBurstPacket.maxFramesPerPacket + 1)
|
||||
#expect(VoiceBurstPacket(burstID: burstID, seq: 1, kind: .frames(tooMany)) == nil)
|
||||
}
|
||||
|
||||
@Test func makeBurstIDProducesUniqueEightBytes() {
|
||||
let a = VoiceBurstPacket.makeBurstID()
|
||||
let b = VoiceBurstPacket.makeBurstID()
|
||||
#expect(a.count == VoiceBurstPacket.burstIDSize)
|
||||
#expect(a != b)
|
||||
}
|
||||
|
||||
// MARK: - Packetizer
|
||||
|
||||
@Test func packetizerRespectsBudgetAndCounts() throws {
|
||||
var packetizer = VoiceBurstPacketizer(burstID: burstID, budget: 210)
|
||||
let frame = Data(repeating: 0x55, count: 130) // realistic 16 kbps AAC frame
|
||||
|
||||
// First frame buffers, second forces a flush of the first.
|
||||
#expect(packetizer.add(frame).isEmpty)
|
||||
let flushed = packetizer.add(frame)
|
||||
#expect(flushed.count == 1)
|
||||
let first = try #require(VoiceBurstPacket.decode(flushed[0]))
|
||||
#expect(first.seq == 1)
|
||||
guard case .frames(let frames) = first.kind else {
|
||||
Issue.record("expected frames")
|
||||
return
|
||||
}
|
||||
#expect(frames == [frame])
|
||||
|
||||
// Remaining frame flushes on demand; counters advance.
|
||||
let final = packetizer.flush()
|
||||
#expect(final.count == 1)
|
||||
#expect(try #require(VoiceBurstPacket.decode(final[0])).seq == 2)
|
||||
#expect(packetizer.dataPacketCount == 2)
|
||||
#expect(packetizer.nextSeq == 3)
|
||||
#expect(packetizer.flush().isEmpty)
|
||||
}
|
||||
|
||||
@Test func packetizerBatchesSmallFrames() throws {
|
||||
var packetizer = VoiceBurstPacketizer(burstID: burstID, budget: 210)
|
||||
let small = Data(repeating: 0x11, count: 40)
|
||||
for _ in 0..<4 {
|
||||
#expect(packetizer.add(small).isEmpty) // 4 * 42 + 11 = 179 <= 210
|
||||
}
|
||||
let packets = packetizer.flush()
|
||||
#expect(packets.count == 1)
|
||||
guard case .frames(let frames) = try #require(VoiceBurstPacket.decode(packets[0])).kind else {
|
||||
Issue.record("expected frames")
|
||||
return
|
||||
}
|
||||
#expect(frames.count == 4)
|
||||
}
|
||||
|
||||
@Test func packetizerDropsOversizedFrame() {
|
||||
var packetizer = VoiceBurstPacketizer(burstID: burstID, budget: 210)
|
||||
#expect(packetizer.add(Data(repeating: 0, count: 500)).isEmpty)
|
||||
#expect(packetizer.flush().isEmpty)
|
||||
#expect(packetizer.dataPacketCount == 0)
|
||||
}
|
||||
|
||||
@Test func encodedPacketStaysWithinNoisePaddingBucket() throws {
|
||||
// The whole point of the budget: burst content + 1 type byte +
|
||||
// 16-byte Noise tag must stay within MessagePadding's 256 bucket.
|
||||
var packetizer = VoiceBurstPacketizer(burstID: burstID)
|
||||
var largest = 0
|
||||
for _ in 0..<3 {
|
||||
for packet in packetizer.add(Data(repeating: 0xAB, count: 160)) {
|
||||
largest = max(largest, packet.count)
|
||||
}
|
||||
}
|
||||
for packet in packetizer.flush() {
|
||||
largest = max(largest, packet.count)
|
||||
}
|
||||
#expect(largest > 0)
|
||||
#expect(largest + 1 + 16 <= 256)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,160 @@
|
||||
# Smart Push-to-Talk (PTT) — Design
|
||||
|
||||
**Status:** Draft for review — no code yet
|
||||
**Scope:** Live voice bursts over the BLE mesh, in public chat and Noise DMs, with graceful degradation to the existing voice-note pipeline.
|
||||
|
||||
## 1. Goal and core idea
|
||||
|
||||
Today, holding the mic button records an AAC `.m4a` and ships it as a `fileTransfer` (0x22) after you release — the receiver hears nothing until the whole file arrives. PTT makes the same gesture *live*: encoded audio frames stream over the mesh while you speak, and nearby receivers hear you with sub-second delay, walkie-talkie style.
|
||||
|
||||
The "smart" part is that PTT is not a separate mode the user must choose. It is a **delivery strategy**, picked automatically per conversation:
|
||||
|
||||
| Context | Delivery |
|
||||
|---|---|
|
||||
| DM, peer connected/reachable on mesh | **Live stream** (Noise-encrypted frames) + finalized voice note for reliability |
|
||||
| DM, peer only reachable via Nostr | Existing voice-note recording only (no live; media doesn't ride Nostr today) |
|
||||
| Public mesh chat | **Live broadcast stream**, best-effort, no file follow-up |
|
||||
| Geohash (Nostr) channels | PTT unavailable (matches existing `canSendMediaInCurrentContext` media policy) |
|
||||
|
||||
One gesture (hold mic), one mental model ("talk"), and the system degrades from live → reliable-note → unavailable based on what the transport can actually do.
|
||||
|
||||
**Bandwidth reality check:** the mesh moves ~15 KB/s per link (469 B fragments at 30 ms spacing); our voice codec needs ~2 KB/s. Live voice fits with a wide margin, even relayed.
|
||||
|
||||
## 2. What already exists (reused, not rebuilt)
|
||||
|
||||
- **Capture UX:** `ContentComposerView.micButtonView` already implements hold-to-record / release-to-send via `DragGesture`, backed by `VoiceRecordingViewModel`'s state machine and `VoiceRecorder` (AAC-LC, 16 kHz mono, 16 kbps). Mic permission string is already in Info.plist.
|
||||
- **Reliable delivery:** `BitchatFilePacket` TLV + `fileTransfer` (0x22) + fragmentation + transfer progress + `ChatMediaTransferCoordinator`.
|
||||
- **Playback:** `VoiceNotePlaybackController` + `VoiceNotePlaybackCoordinator` (single active playback), `WaveformView`, `VoiceNoteView`.
|
||||
- **Transport:** signed public packets, Noise sessions with typed inner payloads, `RelayController` flood control, `MessageDeduplicator`, `MessageRouter.canDeliverPromptly`.
|
||||
|
||||
New work is the **streaming path**: a frame encoder/packetizer, a wire format for bursts, a jitter-buffered receiver/player, relay policy, and the live-bubble UI.
|
||||
|
||||
## 3. Wire protocol
|
||||
|
||||
### 3.1 Burst framing (shared inner format)
|
||||
|
||||
A talk burst is a sequence of packets sharing an 8-byte random `burstID`:
|
||||
|
||||
```
|
||||
[burstID: 8][seq: UInt16 BE][flags: 1][payload…]
|
||||
```
|
||||
|
||||
`flags` bits:
|
||||
- `0x01 START` — payload is a header TLV: codec (1 B enum, 0x01 = AAC-LC/16kHz/mono), frame duration ms, batch size. Sent as seq 0 and re-attached (piggybacked TLV) every ~2 s so mid-burst joiners can sync.
|
||||
- `0x02 END` — payload: total data-packet count (UInt16), duration ms (UInt32). Lets receivers detect tail loss.
|
||||
- `0x04 CANCELED` — sender slid-to-cancel; receivers stop playback, discard buffered audio, and drop the bubble.
|
||||
- `0x00` (data) — payload is N length-prefixed AAC frames: `[len: UInt16 BE][ADTS-less raw AAC frame]…`
|
||||
|
||||
Audio math: AAC-LC at 16 kHz has 1024-sample frames = **64 ms/frame ≈ 130 B at 16 kbps**. A greedy packetizer batches frames up to a **210-byte burst-content budget** (`pttMaxBurstContentBytes`), chosen so the Noise ciphertext (content + 1 inner-type byte + 16-byte tag) stays inside `MessagePadding`'s 256-byte bucket — the whole directed packet is 288 bytes and voice **never enters the fragment scheduler** (which caps concurrent transfers at 2 and would starve file sends). At 16 kbps that works out to **1 frame/packet, ~15.6 pkt/s, ~5.3 KB/s wire** for DMs; public bursts (unpadded, planned for phase 2) can batch 3 frames.
|
||||
|
||||
### 3.2 Public mesh: new `MessageType.voiceFrame = 0x29`
|
||||
|
||||
- Broadcast (no recipient), **signed** like public messages; unsigned or signature-mismatched frames are dropped on receive.
|
||||
- **No padding** (add to `BLEOutboundPacketPolicy` alongside `fileTransfer`) — padding to the 512 block would push every packet over MTU into fragmentation.
|
||||
- TTL: `messageTTL` (7) at origin; `RelayController` gets a `voiceFrame` case mirroring the fragment policy — dense clamp to 5, jitter 8–25 ms — so live audio floods like fragments, not like announces. Per-hop cost is ~10–40 ms; 3 hops stays comfortably inside the jitter buffer.
|
||||
- Dedup: existing `MessageDeduplicator` (timestamp ms + seq make each packet unique).
|
||||
- 0x29 is the next free code after `nostrCarrier = 0x28`; unknown types are ignored by older clients (iOS and Android), so rollout is compatible — old clients simply don't hear live bursts.
|
||||
|
||||
### 3.3 DM: new `NoisePayloadType.voiceFrame = 0x08`
|
||||
|
||||
- Inner payload = the same burst framing, wrapped in `noiseEncrypted` (0x11) directed packets — wire-indistinguishable from other DM traffic by size/type (0x04/0x05 are reserved per the comment in `BitchatProtocol.swift`; 0x08 is the first free slot after `groupKeyUpdate = 0x07`).
|
||||
- Directed encrypted packets are already always-relayed by `RelayController`, so multi-hop DMs work unchanged.
|
||||
- Requires an established session; PTT-live is only offered when `noiseService.hasEstablishedSession` (otherwise first hold triggers the normal handshake + falls back to voice-note for that burst).
|
||||
- Fire-and-forget: **no delivery acks, no retransmit** for frames. Late audio is worthless; reliability comes from the finalized note (§5).
|
||||
|
||||
### 3.4 Known traffic-analysis tradeoff
|
||||
|
||||
A steady ~8 pkt/s cadence for the duration of a burst is a timing fingerprint even under Noise (observers can infer "someone is speaking to someone", not content). This is inherent to live voice; the doc records it as accepted for v1. Block padding is skipped for voiceFrame (size classes would leak little beyond what cadence already does).
|
||||
|
||||
## 4. Sender pipeline
|
||||
|
||||
`VoiceRecorder` (AVAudioRecorder → file) can't stream, so PTT capture uses a parallel path in a new `PTTStreamEncoder` actor:
|
||||
|
||||
```
|
||||
AVAudioEngine input tap (native format)
|
||||
→ PTTInputResampler → 16 kHz mono PCM
|
||||
→ PTTFrameEncoder (AVAudioConverter) → AAC-LC frames → packetizer → BLEService.sendVoiceFrame
|
||||
→ the same PCM is simultaneously written to an AVAudioFile .m4a (identical settings
|
||||
to VoiceRecorder), so finalization needs no remux step
|
||||
```
|
||||
|
||||
- On release: emit `END`, then hand the finalized `.m4a` to the **existing** `sendVoiceNote` flow. The note's `fileName` embeds the burst ID (`voice_<burstID hex>.m4a`) — that links note↔burst with **zero changes** to `BitchatFilePacket`/Android interop.
|
||||
- On slide-to-cancel: stop tap, emit `CANCELED`, delete local file, skip finalization. UX note: unlike voice notes, live audio already played on the far side cannot be unsent — the recording UI must make "you are live" unmistakable (see §7).
|
||||
- Caps: max burst 120 s (matching the voice-note recorder's cap), exactly one outbound burst at a time.
|
||||
- Audio session: reuse `VoiceRecorder`'s `.playAndRecord` config; add `.duckOthers`.
|
||||
|
||||
## 5. Receiver pipeline
|
||||
|
||||
New `VoiceBurstAssembler` (keyed by sender + burstID) feeding a `PTTBurstPlayer`:
|
||||
|
||||
- **Jitter buffer:** start playback after 350 ms of buffered audio or 500 ms wall-clock, whichever first. Frames decode via `AVAudioConverter` → `AVAudioPlayerNode`.
|
||||
- **Loss handling:** gap in seq → insert silence for the missing frames (64 ms each) and keep going. No PLC in v1; at these frame sizes brief dropouts are acceptable.
|
||||
- **Burst end:** on `END`, or 3 s with no frames (talker walked out of range).
|
||||
- **Persistence:** frames append to an incoming ADTS `.aac` file (already an allowed `MimeType`), so every burst becomes a replayable voice-note bubble containing whatever was captured — even a partial one.
|
||||
- **Dedup with the finalized note (DM):** when a `fileTransfer` arrives whose fileName carries a burstID we already assembled, it silently *replaces* the partial file behind the existing bubble (no new message row, no second notification). Receivers that heard everything live just get a lossless copy.
|
||||
- **Resource caps:** ≤ 8 concurrent assemblies, ≤ 256 KB per burst (60 s × 2 KB/s + slack), 30 s stale cleanup, and drop inbound frames beyond ~2× realtime per sender (spam/flood guard).
|
||||
|
||||
## 6. Playback policy — when does it actually make sound?
|
||||
|
||||
Auto-playing strangers' audio is the fastest way to make this feature hated. Rules:
|
||||
|
||||
1. **Live autoplay** only when *all* hold: app foregrounded, the burst's conversation is the one currently on screen, and the **"live voice messages"** toggle is on (app-level, in the app-info sheet, default on; per-conversation overrides are a v2 refinement). The same toggle gates live *sending* — off means voice behaves exactly like classic notes in both directions.
|
||||
2. Otherwise the burst appears as a **live bubble**: pulsing waveform + `LIVE` badge + sender name; tapping it joins playback at the live edge. When the burst ends it becomes a normal voice-note bubble.
|
||||
3. **One voice at a time:** route through `VoiceNotePlaybackCoordinator`. If two people talk simultaneously, the first burst holds the floor; the second shows as a tappable live bubble. No mixing in v1.
|
||||
4. Notifications: a live burst in a non-focused DM fires the normal message notification once (at START), not per-frame.
|
||||
|
||||
**Floor courtesy (public mesh):** while someone else's burst is live in the current channel, the mic button tints "busy" with the talker's name. Holding it still works — a decentralized mesh has no floor arbiter, and rejecting sends would desync under partitions — but the UI discourages talk-over. Hard floor control (token passing) is explicitly out of scope.
|
||||
|
||||
## 7. UX
|
||||
|
||||
- **Same gesture:** hold mic = talk. When the live path is active, the recording HUD shows a red pulsing **LIVE** treatment (vs. the current neutral recording UI) so the sender knows audio is leaving in real time, not on release. When live isn't available (Nostr-only peer, no session yet), the HUD looks like today's and behavior is unchanged.
|
||||
- Slide-to-cancel keeps working in both modes, with the §4 caveat surfaced simply: cancel stops and discards; it can't unplay what was heard.
|
||||
- VoiceOver: mirror the existing `accessibilityAction` toggle-record pattern on the mic button.
|
||||
- Foreground-only in v1: no `audio` UIBackgroundMode (App Store review + battery implications). If the app backgrounds mid-burst, capture stops cleanly with END. Background listen/talk is a v2 candidate.
|
||||
|
||||
## 8. Latency budget (1 hop, DM)
|
||||
|
||||
| Stage | Cost |
|
||||
|---|---|
|
||||
| Frame accumulation (1 × 64 ms) | 64 ms |
|
||||
| Encode + packetize | ~5 ms |
|
||||
| BLE write + delivery | 30–60 ms |
|
||||
| Jitter buffer | 350 ms |
|
||||
| **Mouth-to-ear** | **~470 ms** |
|
||||
|
||||
Each relay hop adds ~10–40 ms jitter + radio time. 2–3 hops stays under ~800 ms — solidly in walkie-talkie territory (commercial PoC apps run 300 ms–1 s+).
|
||||
|
||||
## 9. Security & privacy summary
|
||||
|
||||
- Public frames signed with the existing packet signature; verified against the sender's announce before decode. Unsigned → dropped.
|
||||
- DM frames ride inside Noise; content confidentiality/integrity as any DM.
|
||||
- Codec input is validated by frame-length prefixes and total-size caps before touching `AVAudioConverter` (malformed frames dropped, assembly aborted over cap).
|
||||
- Timing fingerprint accepted per §3.4. No PTT in geohash channels (would put voice on public Nostr relays).
|
||||
- Mic capture only while the button is held; recording state is always visible.
|
||||
|
||||
## 10. New components & touch points
|
||||
|
||||
| Piece | Where |
|
||||
|---|---|
|
||||
| `PTTStreamEncoder` (tap → AAC → packets) | `bitchat/Features/voice/` |
|
||||
| `PTTBurstPlayer` (jitter buffer → decode → engine) | `bitchat/Features/voice/` |
|
||||
| `VoiceBurstFramer` / `VoiceBurstAssembler` (wire encode/decode, caps) | `bitchat/Protocols/` + `bitchat/Services/PTT/` |
|
||||
| `MessageType.voiceFrame = 0x29` | `localPackages/BitFoundation/.../MessageType.swift` + `BLEReceivePipeline` / `BLEService.handleReceivedPacket` |
|
||||
| `NoisePayloadType.voiceFrame = 0x08` | `BitchatProtocol.swift` + `ChatTransportEventCoordinator` dispatch |
|
||||
| Relay policy case | `RelayController` (fragment-like clamp/jitter) |
|
||||
| No-padding rule | `BLEOutboundPacketPolicy` |
|
||||
| Pacing/cap constants | `TransportConfig` |
|
||||
| Live bubble + LIVE HUD + busy mic tint + settings toggle | `VoiceNoteView`/`MediaMessageView`, `ContentComposerView`, conversation settings |
|
||||
| Delivery-mode selection | `ChatViewModel` / `ChatMediaTransferCoordinator` (reuse `isPeerConnected` / `canDeliverPromptly` / `hasEstablishedSession`) |
|
||||
|
||||
## 11. Phasing
|
||||
|
||||
1. **Phase 1 — DM live (highest value, lowest blast radius):** encoder, framer, Noise inner type, assembler/player, live bubble, finalize-as-note dedup. Single-hop DMs are the dominant real-world case.
|
||||
2. **Phase 2 — public mesh:** `voiceFrame` 0x29, signing/verification, relay policy, floor-courtesy UI, autoplay defaults.
|
||||
3. **Phase 3 (v2 candidates):** background audio mode, mid-burst repair requests (piggyback on REQUEST_SYNC), talk-over mixing, AAC-ELD low-delay profile, dedicated walkie-mode screen, media-over-Nostr (unlocks live-ish PTT for internet peers), Android protocol spec sync.
|
||||
|
||||
## 12. Testing
|
||||
|
||||
- Unit: framer round-trip (START/data/END/CANCELED, seq gaps, oversize frames), assembler caps + stale cleanup + burstID note dedup, packetizer batch sizing vs. MTU (assert no fragmentation), relay-policy clamps.
|
||||
- Integration: two-simulator loopback via the existing mesh test harness; loss injection (drop every Nth frame) → verify silence-fill + partial-file persistence.
|
||||
- Device: two-phone live DM (latency measurement vs. §8 budget), three-phone relay chain, talk-over behavior, cancel semantics, mic-permission-denied path.
|
||||
Reference in New Issue
Block a user