Migrate private media without silent downgrades

docs/ANNOUNCEMENT_GOSSIP.md

@@ -8,7 +8,10 @@ Status: optional and backward-compatible.
 
 - Outer packet: BitChat binary packet with `type = 0x01` (ANNOUNCE). Header is unchanged.
 - Payload: A sequence of TLVs. Unknown TLVs MUST be ignored for forward compatibility.
-- Signature: The packet MAY be signed using the Ed25519 public key carried in TLV `0x03`. The gossip TLV (if present) is part of the payload and therefore covered by the signature.
+- Signature: The packet is signed using the Ed25519 public key carried in TLV
+  `0x03`. The gossip and capability TLVs are part of the payload and therefore
+  covered by the signature. Current clients require a valid signature before
+  applying identity or capability state.
 
 ## TLV Format
 
@@ -24,6 +27,12 @@ Existing TLVs (unchanged):
 - `0x02` NOISE_PUBLIC_KEY: Noise static public key bytes (typically 32 bytes for X25519)
 - `0x03` SIGNING_PUBLIC_KEY: Ed25519 public key bytes (typically 32 bytes)
 
+Other optional extension:
+
+- `0x05` CAPABILITIES: Minimal little-endian feature bitfield. Bit 8 advertises
+  authenticated Noise private media (`PRIVATE_MEDIA_V1`); its exact value is
+  `00 01`. An empty value is valid and means no advertised capabilities.
+
 New TLV (optional):
 
 - `0x04` DIRECT_NEIGHBORS: Concatenation of up to 10 peer IDs, each encoded as exactly 8 bytes. There is no inner count; the number of neighbors is `length / 8`. If `length` is not a multiple of 8, trailing partial bytes MUST be ignored.
@@ -44,7 +53,9 @@ This matches the on‑wire 8‑byte `senderID`/`recipientID` encoding used in th
 - Optionally append TLV `0x04` with up to 10 unique, directly connected peer IDs.
   - Remove duplicates before encoding.
   - Order is arbitrary and not semantically significant.
-- Sign the ANNOUNCE packet so the gossip TLV is covered (recommended):
+- Current capable senders also include TLV `0x05` with private-media bit 8 set
+  in every broadcast and peer-directed announcement.
+- Sign the ANNOUNCE packet so all extension TLVs are covered:
   - Signature algorithm: Ed25519 using the key in TLV `0x03`.
   - Signature input: the binary packet encoding with the signature field omitted and the TTL normalized to `0`. This allows TTL to change during relays without invalidating the signature.
 - The payload may be compressed per the base protocol; the gossip TLV is encoded prior to optional compression.
@@ -53,6 +64,11 @@ This matches the on‑wire 8‑byte `senderID`/`recipientID` encoding used in th
 
 - Decompress payload if the packet’s compression flag is set, then parse TLVs in order.
 - Parse TLVs `0x01`..`0x03` as usual; ignore any unknown TLVs.
+- Parse TLV `0x05`, when present, as a little-endian bitfield. Absence and an
+  explicitly empty value remain valid legacy capability states. A
+  security-sensitive bit is trusted only after the signed announcement key is
+  bound to the matching remote static key from a live Noise handshake; see
+  `PRIVATE_MEDIA_V1.md`.
 - If a `0x04` TLV is present:
   - Interpret the value as `N = length / 8` peer IDs (ignore trailing non‑aligned bytes).
   - Each 8‑byte chunk is decoded back to a 16‑hex‑char peer ID string (lowercase).
@@ -76,8 +92,8 @@ ANNOUNCE payload TLVs (concatenated):
 - `02 [len=32] [32 bytes X25519 pubkey]`
 - `03 [len=32] [32 bytes Ed25519 pubkey]`
 - `04 [len=8*M] [peerID1(8) || peerID2(8) || ... || peerIDM(8)]` (optional)
+- `05 02 00 01` (optional private-media-v1 capability)
 
 Where each `peerIDk(8)` is the 8‑byte binary form of the peer ID as specified above.
 
 That’s the entire change; the outer packet header, message type, and relay/TTL behavior are unchanged.
-

docs/PRIVATE_MEDIA_V1.md

@@ -0,0 +1,107 @@
+# Private media v1 interoperability and migration
+
+Private media reuses the canonical `BitchatFilePacket` TLV. A capable sender
+wraps the complete encoded file TLV as Noise payload type `0x20`, encrypts it
+for the recipient, and only then fragments the final outer packet.
+
+## Encrypted wire contract
+
+- Outer packet type: `MessageType.NOISE_ENCRYPTED` (`0x11`).
+- Decrypted Noise payload type: `NoisePayloadType.FILE_TRANSFER` (`0x20`).
+- Noise payload data: one complete encoded `BitchatFilePacket`.
+- Outer recipient: the target peer ID; never broadcast for private media.
+
+Do not allocate a second Noise payload type for this format.
+
+## Capability announcement
+
+Identity announcement TLV `0x05` is a minimal little-endian bitfield. Bit 8
+means the peer implements private-media v1, so its exact encoding is:
+
+```text
+05 02 00 01
+|  |  |----- capability bytes: 0x0100 little-endian
+|  |-------- value length
+|----------- capabilities TLV
+```
+
+Current Android builds include this TLV in broadcast and peer-directed
+announcements over both BLE and Wi-Fi Aware. Older clients safely skip the
+unknown TLV. Its absence, or a present TLV with bit 8 clear, describes a legacy
+client and does not invalidate the announcement.
+
+Decoders retain unknown low-64-bit capability bits and unknown announcement
+TLVs so a decode/re-encode cycle does not erase newer extensions.
+
+## Authenticated capability pinning
+
+The capability bit is security-sensitive and is not trusted just because an
+announcement is self-signed. A client may promote bit 8 only after both of
+these checks succeed, in either arrival order:
+
+1. The announcement has a valid Ed25519 signature using its advertised
+   signing key.
+2. Its advertised Noise static key exactly matches the remote static key
+   authenticated by the live Noise handshake.
+
+After promotion, store an HSTS-style pin keyed by the SHA-256 fingerprint of
+that authenticated Noise static key. The pin is persistent, encrypted at rest,
+and cleared by panic wipe. Never derive it from a peer-ID cache,
+`PeerFingerprintManager`, an unverified announcement, or a Noise key that does
+not match the signed announcement.
+
+A pin prevents a later missing or cleared capability bit from downgrading the
+same authenticated identity. It never substitutes for a live authenticated
+Noise session when sending.
+
+## Mixed-client send policy
+
+- No live authenticated Noise remote-static key: emit no media or local echo,
+  initiate one Noise handshake, and require the user to retry after it completes.
+- Authenticated and pinned, or authenticated with a matching verified bit-8
+  announcement: send encrypted `0x11` / `0x20`.
+- Authenticated with a matching verified legacy announcement (TLV absent or bit
+  clear): ask for explicit one-shot consent before sending this file.
+
+The legacy consent path sends a recipient-directed raw
+`MessageType.FILE_TRANSFER` (`0x22`) packet. Its contents are visible to relays,
+so the UI must say that it is not end-to-end encrypted. The final routed packet
+must carry a valid Ed25519 signature over the canonical packet bytes; signing
+failure aborts the send. Receivers reject unsigned or invalid signed directed
+raw files.
+
+Consent is consumed at most once. On approval the sender re-runs the policy and
+packet admission checks. If the capability became authenticated while the
+dialog was open, the send upgrades to encrypted mode. Cancellation, duplicate
+approval, panic wipe, and changed security state cannot cause a later send.
+
+## Final-packet admission
+
+Before creating a local echo or progress mapping, the sender builds the exact
+encrypted-or-legacy packet, attaches its final source route, signs it, and
+creates the exact transport fragment plan. Commit sends that prepared plan
+without rebuilding it.
+
+One packet may use at most 256 fragments. This limit is checked after route,
+signature, encryption, and envelope overhead are known; therefore there is no
+single safe file-byte estimate for every route. Fragment totals and indices
+must also fit their unsigned 16-bit wire fields without truncation. A rejected
+plan creates no local echo and sends no fragments.
+
+The 256-fragment limit is a transport/reassembly safety bound, not a capability
+negotiated through bit 8. A future larger transfer protocol needs a separate
+capability and bounded streaming design.
+
+## Compatibility summary
+
+- New Android to new iOS/Android: encrypted Noise `0x20` after authenticated
+  capability promotion.
+- New Android to a verified older client: blocked until the user explicitly
+  accepts one relay-visible signed raw `0x22` transfer.
+- Old clients receiving a new announcement: ignore TLV `0x05` and continue
+  operating normally.
+- A previously capable authenticated identity cannot force a silent downgrade
+  by later omitting the bit.
+
+Public media remains signed broadcast `MessageType.FILE_TRANSFER` (`0x22`) and
+is outside this private-media capability.

docs/file_transfer.md

@@ -3,13 +3,20 @@
 This document is the exhaustive implementation guide for Bitchat’s Bluetooth file transfer protocol for voice notes (audio) and images, including interactive features like waveform seeking. It describes the on‑wire packet format (both v1 and v2), fragmentation/progress/cancellation, sender/receiver behaviors, and the complete UX we implemented in the Android client so that other implementers can interoperate and match the user experience precisely.
 
 **Protocol Versions:**
-- **v1**: Original protocol with 2‑byte payload length (≤ 64 KiB files)
-- **v2**: Extended protocol with 4-byte payload length (≤ 4 GiB files) - use for all file transfers
-- File transfer packets use v2 format by default for optimal compatibility
+- **v1**: Original envelope with a 2-byte payload length.
+- **v2**: Extended envelope with a 4-byte payload length.
+- Public and legacy raw file-transfer envelopes use v2. A Noise-encrypted
+  private envelope may use v1 when its ciphertext fits, and source routing
+  upgrades the final envelope to v2.
+- The payload-length field is not the practical transfer limit. Private-media
+  admission is limited to 256 final fragments after route, signature,
+  encryption, and envelope overhead. Generic/public outbound fragmentation
+  retains the UInt16 wire range; receivers may impose a lower safety bound.
 
 **Interactive Features:**
 - **Waveform Seeking**: Tap anywhere on audio waveforms to jump to that playback position
-- **Large File Support**: v2 protocol enables multi-GiB file transfers through fragmentation
+- **Bounded Transfer Support**: v2 removes the 16-bit envelope-length limit,
+  while bounded fragmentation protects receivers from unbounded reassembly.
 - **Unified Experience**: Identical UX between platforms with enhanced user control
 
 The guide is organized into:
@@ -34,12 +41,15 @@ Bitchat BLE transport carries application messages inside the common `BitchatPac
 Fields (subset relevant to file transfer):
 
 - `version: UByte` — protocol version (`1` for v1, `2` for v2 with extended payload length).
-- `type: UByte` — message type. File transfer uses `MessageType.FILE_TRANSFER (0x22)`.
+- `type: UByte` — public file transfer uses `MessageType.FILE_TRANSFER (0x22)`;
+  private file transfer uses outer `MessageType.NOISE_ENCRYPTED (0x11)`.
 - `senderID: ByteArray (8)` — 8‑byte binary peer ID.
 - `recipientID: ByteArray (8)` — 8‑byte recipient. For public: `SpecialRecipients.BROADCAST (0xFF…FF)`; for private: the target peer’s 8‑byte ID.
 - `timestamp: ULong` — milliseconds since epoch.
-- `payload: ByteArray` — TLV file payload (see below).
-- `signature: ByteArray?` — optional signature (present for private sends in our implementation, to match iOS integrity path).
+- `payload: ByteArray` — the public form contains the file TLV directly. The
+  private form contains Noise ciphertext which authenticates payload type
+  `FILE_TRANSFER (0x20)` followed by the same file TLV.
+- `signature: ByteArray?` — packet signature added by the mesh send path.
 - `ttl: UByte` — hop TTL (we use `MAX_TTL` for broadcast, `7` for private).
 
 Envelope creation and broadcast paths are implemented in:
@@ -48,7 +58,9 @@ Envelope creation and broadcast paths are implemented in:
 - `app/src/main/java/com/bitchat/android/mesh/BluetoothConnectionManager.kt` (/Users/cc/git/bitchat-android/app/src/main/java/com/bitchat/android/mesh/BluetoothConnectionManager.kt)
 - `app/src/main/java/com/bitchat/android/mesh/PacketProcessor.kt` (/Users/cc/git/bitchat-android/app/src/main/java/com/bitchat/android/mesh/PacketProcessor.kt)
 
-Private sends are additionally encrypted at the higher layer (Noise) for text messages, but file transfers use the `FILE_TRANSFER` message type in the clear at the envelope level with content carried inside a TLV. See code for any deployment‑specific enforcement.
+Private sends encrypt the complete file TLV as Noise payload `0x20` before
+outer fragmentation. See `PRIVATE_MEDIA_V1.md` for the capability and mixed-
+client interoperability contract.
 
 ### 1.2 Binary Protocol Extensions (v2)
 
@@ -77,19 +89,23 @@ PayloadLength: 4 bytes (big-endian, max ~4 GiB)
 ```
 
 - **Header Size**: Increased from 13 to 15 bytes.
-- **Payload Length Field**: Extended from 16 bits (2 bytes) to 32 bits (4 bytes), allowing file transfers up to ~4 GiB.
-- **Backward Compatibility**: Clients must support both v1 and v2 decoding. File transfer packets always use v2.
+- **Payload Length Field**: Extended from 16 bits (2 bytes) to 32 bits (4
+  bytes). That is the field's theoretical range, not the permitted mesh
+  reassembly size.
+- **Backward Compatibility**: Clients must support both v1 and v2 decoding.
 - **Implementation**: See `BinaryProtocol.kt` with `getHeaderSize(version)` logic.
 
-#### Use Cases for v2
-- **Large Audio Files**: Professional recordings, podcasts, or music samples.
-- **High-Resolution Images**: Full-resolution photos from modern smartphones.
-- **Future File Types**: PDFs, documents, archives, or other large media.
+#### Use cases for v2
+
+v2 carries file payloads that exceed the v1 envelope-length field and carries
+source-route metadata. It does not imply multi-gigabyte mesh transfer support.
 
 #### Interoperability Requirements
 - Clients receiving v2 packets must decode 4-byte `PayloadLength` fields.
 - Clients sending file transfers should preferentially use v2 format.
-- Fragmentation still applies: large files are split into fragments that fit within BLE MTU constraints (~128 KiB per fragment).
+- Fragmentation still applies. Each serialized mesh fragment fits the 512-byte
+  transport threshold; the data portion is at most 469 bytes and becomes
+  smaller when recipient or source-route overhead is present.
 
 ### 1.3 File Transfer TLV payload (BitchatFilePacket)
 
@@ -114,7 +130,8 @@ Encoding rules:
 
 - Standard TLVs use `1 byte type + 2 bytes big‑endian length + value`.
 - CONTENT uses a 4‑byte big‑endian length to allow payloads well beyond 64 KiB.
-- With the v2 envelope (4‑byte payload length), CONTENT can be large; transport still fragments oversize packets to fit BLE MTU.
+- With the v2 envelope (4-byte payload length), CONTENT can exceed 64 KiB, but
+  sender and receiver fragmentation policies still bound practical transfers.
 - Implementations should validate TLV boundaries; decoding should fail fast on malformed structures.
 
 Decoding rules (v2):
@@ -140,8 +157,13 @@ Legacy Compatibility (optional, for mixed‑version meshes):
 File transfers reuse the mesh broadcaster’s fragmentation logic:
 
 - `BluetoothPacketBroadcaster` checks if the serialized envelope exceeds the configured MTU and splits it into fragments via `FragmentManager`.
-- Fragments are sent with a short inter‑fragment delay (currently ~200 ms; matches iOS/Rust behavior notes in code).
+- Fragments are sent with a short inter-fragment delay (currently 20 ms).
 - When only one fragment is needed, send as a single packet.
+- Android receivers reject fragment sets declaring more than 256 fragments.
+  Private senders use that same 256-fragment limit and calculate it from the
+  exact final routed, signed, and encrypted packet before creating a local
+  echo. Generic/public outbound planning retains its prior UInt16 count range,
+  so this private-media migration does not silently change public sending.
 
 ### 2.2 Transfer ID and progress events
 
@@ -216,24 +238,38 @@ Files:
    - Files saved under `files/voicenotes/outgoing/voice_YYYYMMDD_HHMMSS.m4a`.
 
 2) Local echo
-   - We create a `BitchatMessage` with content `"[voice] <path>"` and add to the appropriate timeline (public/channel/private).
-   - For private: `messageManager.addPrivateMessage(peerID, message)`. For public/channel: `messageManager.addMessage(message)` or add to channel.
+   - Public/channel sends create their timeline entry before dispatch.
+   - Private sends first finish capability policy and exact final-fragment
+     admission. They create the local echo and progress mapping only for an
+     admitted plan, then atomically commit that prepared plan.
 
 3) Packet creation
    - Build a `BitchatFilePacket`:
      - `fileName`: basename (e.g., `voice_… .m4a`)
      - `fileSize`: file length
      - `mimeType`: `audio/mp4`
-     - `content`: full bytes (ensure content ≤ 64 KiB; with chosen codec params typical short notes fit fragmentation constraints)
+     - `content`: full bytes; final-packet admission determines whether it fits
+       the 256-fragment limit.
    - Encode TLV; compute `transferId = sha256Hex(payload)`.
    - Map `transferId → messageId` for UI progress.
 
 4) Send
    - Public: `BluetoothMeshService.sendFileBroadcast(filePacket)`.
-   - Private: `BluetoothMeshService.sendFilePrivate(peerID, filePacket)`.
+   - Private UI: `prepareFilePrivate(...)`, followed by one-shot commit of a
+     ready plan. The non-interactive `sendFilePrivate(...)` entry point commits
+     only encrypted-ready plans and never silently chooses a legacy downgrade.
    - Broadcaster handles fragmentation and progress emission.
 
-5) Waveform
+5) Mixed-client private migration
+   - A verified legacy recipient with no private-media capability requires a
+     user warning and one-shot consent.
+   - Approval rechecks policy. The fallback is recipient-directed raw `0x22`,
+     visible to relays, and must have a valid Ed25519 packet signature.
+   - No authenticated Noise identity starts a handshake without a local echo
+     or media send; the user retries after it completes. Signing failure or a
+     private plan above 256 final fragments aborts without an echo or send.
+
+6) Waveform
    - We extract a 120‑bin waveform from the recorded file (the same extractor used for the receiver) and cache by file path, so sender and receiver waveforms are identical.
 
 Core files:
@@ -373,8 +409,10 @@ Files:
 - Path markers in messages
   - We use simple content markers: `"[voice] <abs path>", "[image] <abs path>", "[file] <abs path>"` for local rendering. These are not sent on the wire; the actual file bytes are inside the TLV payload.
 - Progress math for images relies on `(sent / total)` from `TransferProgressManager` (fragment‑level granularity). The block grid density can be tuned; currently 24×16.
-- Private vs public: both use the same file TLV; only the envelope `recipientID` differs. Private may have signatures; code shows a signing step consistent with iOS behavior prior to broadcast to ensure integrity.
-- BLE timing: there is a 200 ms inter‑fragment delay for stability. Adjust as needed for your radio stack while maintaining compatibility.
+- Private vs public: both use the same file TLV. Private media wraps it in a
+  Noise payload of type `0x20`; public media carries it directly in a `0x22`
+  packet.
+- BLE timing: there is a 20 ms inter-fragment delay. Adjust as needed for your radio stack while maintaining compatibility.
 
 
 ---
@@ -426,7 +464,9 @@ Fullscreen image:
    - FILE_NAME and MIME_TYPE: `type(1) + len(2) + value`
    - FILE_SIZE: `type(1) + len(2=4) + value(4, UInt32 BE)`
    - CONTENT: `type(1) + len(4) + value`
-3. Embed the TLV into a `BitchatPacket` envelope with `type = FILE_TRANSFER (0x22)` and the correct `recipientID` (broadcast vs private).
+3. For public media, embed the TLV in `FILE_TRANSFER (0x22)`. For private media,
+   prefix the TLV with Noise payload type `0x20`, encrypt it for the recipient,
+   and put the ciphertext in `NOISE_ENCRYPTED (0x11)`.
 4. Fragment, send, and report progress using a transfer ID derived from `sha256(payload)` so the UI can map progress to a message.
 5. Support cancellation at the fragment sender: stop sending remaining fragments and propagate a cancel to the UI (we remove the message).
 6. On receive, decode TLV, persist to an app directory (separate audio/images/other), and create a chat message with content marker `"[voice] path"`, `"[image] path"`, or `"[file] path"` for local rendering.