Commit Graph
16 Commits
Author SHA1 Message Date
jackandClaude Fable 5 18dcc747d7 Originate v2 source routes and wire fragmentIdFilter targeted resync
Part A — source-route origination policy:
- Gate route application (BLESourceRouteOriginationPolicy): only packets we
  author, directed at a single peer, with TTL headroom, whose recipient is
  not directly connected. Relays no longer attach routes to (and re-sign)
  packets they merely forward.
- Version-gate paths: MeshTopologyTracker records the highest protocol
  version observed per peer; BFS routes require every intermediate hop and
  the recipient to be v2-observed, capped at 4 intermediate hops.
- Degrade on failure: BLESourceRouteFailureCache marks a routed send that
  sees no inbound traffic from the recipient within 10s as failed and floods
  for 60s before retrying routes.

Part B — REQUEST_SYNC fragmentIdFilter (TLV 0x06):
- Requester: BLEFragmentAssemblyBuffer reports stalled broadcast
  reassemblies (no new fragment for 5s, retried at most every 10s); the
  maintenance pass sends a types=fragment REQUEST_SYNC naming the stalled
  8-byte fragment stream IDs to each connected peer.
- Responder: GossipSyncManager restricts the fragment diff to exactly the
  named streams, bypassing the since-cursor while the GCS filter still
  excludes pieces the requester holds; RSR/TTL-0/rate-limit semantics
  unchanged and REQUEST_SYNC stays link-local.
- Bounds: at most 60 IDs per request (60*17-1 = 1019 bytes <= the 1024-byte
  decoder cap); oversized 0x06 values are ignored, not fatal.

Docs: SOURCE_ROUTING.md gains the iOS origination policy (§8);
REQUEST_SYNC_MANAGER.md documents 0x05/0x06 as implemented.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 20:02:12 +02:00
jackandClaude Fable 5 38331e62f1 Cut views over to ConversationStore; delete legacy store, bridge, resolver
Feature models observe per-conversation objects directly: PublicChatModel
forwards the active Conversation's objectWillChange, PrivateInboxModel
republishes only for the selected peer's conversation - background
appends no longer invalidate foreground views. LegacyConversationStore,
the coalescing bridge, and IdentityResolver are deleted (resolver
canonicalization proved display-invisible: nothing enumerates direct
conversations, lookups are by exact peer ID, and raw keying is strictly
more robust - documented as a design deviation). Selection state moves
into the store. ChatViewModel.messages/privateChats survive as derived
read views for coordinators that genuinely need them; hot paths use a
new store-direct privateMessages(for:) witness.

Final numbers vs pre-migration baselines:
pipeline.privateIngest 9.7k -> 24.0k msg/s (2.5x)
pipeline.publicIngest  6.8k -> 13.7k msg/s (2.0x)
delivery updates       38k  -> 117-133k/s

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 13:57:12 +02:00
jackandClaude Fable 5 7fb1f4a219 Route delivery status through the store; delete the location index
ConversationStore maintains an exact messageID -> Set<ConversationID>
map at every mutation point (append/upsert/remove/migrate/trim/clear),
so delivery updates are ID-only lookups that fan out to mirrored
ephemeral/stable copies. ChatDeliveryCoordinator shrinks 327 -> 119
lines: the positional location index, its growth-detection/rebuild
machinery, and the duplicate no-downgrade check are deleted - the rule
now lives in exactly one place. The middle-insertion regression tests
are rewritten against the store since stale positional locations are
structurally impossible now.

delivery updates: 38k -> 262k/s (~6.9x); ingest pipelines unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 13:26:00 +02:00
jackandClaude Fable 5 99d1d1dccd Cut public message path over to ConversationStore; delete PublicTimelineStore
Mesh and geohash timelines are now store conversations. All public
mutation sites flow through store intents; PublicMessagePipeline keeps
its 80ms UI batching but commits batches via store appends with each
buffered entry carrying its destination conversation (a mid-batch
channel switch now flushes instead of dropping the buffer).
ChatViewModel.messages becomes a cached get-only view of the active
conversation, invalidated through the change subject. The mesh
late-insert threshold is consciously removed: it only ever ordered the
non-rendered messages copy, so strict timestamp insertion makes the
working set agree with rendered order. PublicTimelineStore and the
per-message full-array legacy sync are deleted; the coalescing bridge
mirrors public conversations for the remaining legacy readers.

pipeline.publicIngest: 6.6k -> 9.5k msg/s (+45%); private steady;
store.append 237k/s.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 13:03:07 +02:00
jackandClaude Fable 5 45650854e7 Add conversation-store design doc and end-to-end ingest baselines
docs/CONVERSATION-STORE-DESIGN.md records the approved design: a
single-writer ConversationStore of per-conversation ObservableObjects
(per-conversation publishing, incremental ID index, folded caps, typed
change subject) replacing today's four-store/three-bridge topology,
with a five-step migration plan and explicit deletions/non-goals.

New pipeline benchmarks measure the CURRENT architecture end-to-end so
every migration step is judged against real before-numbers:
pipeline.privateIngest ~9.7k msg/s, pipeline.publicIngest ~6.8k msg/s
(200-message passes, stable within 1.5%).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 10:51:24 +02:00
jackandClaude Fable 5 4093ee6733 Rebuild Arti from audited source with enforced provenance
Vendored arti.xcframework rebuilt from source (Rust 1.96.0, normalized
archive metadata for reproducible hashes). New ARTI-BINARY-PROVENANCE.md
records toolchain, rebuild steps, and a SHA256 manifest for every file
in the xcframework. A new CI workflow turns that policy into a gate:
PRs must keep the binary matching the manifest, and binary changes must
ship with source/lockfile/build-script evidence.

Also raises TorManager.awaitReady's default timeout from 25s to 75s to
match the bootstrap monitor deadline - a shorter wait reported "not
ready" while Arti was still legitimately bootstrapping, silently
stranding queued relay work.

Privacy policy, Tor integration doc, and privacy assessment updated to
match the current implementation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 16:22:15 +01:00
df36b19afe [codex] Refine BLE ingress fanout (#1280)
* Refactor BLE transport event handling

* Make image output paths unique

* Keep queued Nostr read receipts alive

* Refine BLE ingress fanout

* Rediscover BLE service after invalidation

* Extract BLE notification retry buffer

* Extract BLE inbound write buffer

* Extract BLE fragment assembly buffer

* Tidy secure log handling from device run

* Allow self-authored RSR ingress replies

* Harden read receipt queue test timing

---------

Co-authored-by: jack <jackjackbits@users.noreply.github.com>
2026-05-31 14:08:30 +02:00
ab0da61533 [codex] Refactor BLE transport event handling (#1266)
* Refactor BLE transport event handling

* Make image output paths unique

* Keep queued Nostr read receipts alive

* Allow self-authored RSR ingress replies

---------

Co-authored-by: jack <jackjackbits@users.noreply.github.com>
2026-05-31 13:58:27 +02:00
764f016d17 [codex] Refactor app runtime and ownership architecture (#1104)
* Refactor app runtime and view model architecture

* Move app ownership into stores and coordinators

* Fix smoke test environment injection

* Stabilize fragmentation package tests

* Fix coordinator build warnings

* Clean up chat view model warnings

* Fix Nostr relay startup coalescing

---------

Co-authored-by: jack <jackjackbits@users.noreply.github.com>
2026-05-30 14:13:26 +02:00
3fc64f6168 feat: Implement Request Sync Manager (V2 Sync) (#965)
* feat: Implement Request Sync Manager (V2 Sync)

- Add RequestSyncManager to track and attribute sync requests
- Update BitchatPacket and BinaryProtocol to support IS_RSR flag (0x10)
- Update RequestSyncPacket with new TLV fields (sinceTimestamp, fragmentIdFilter)
- Update GossipSyncManager to use unicast sync requests and mark responses as RSR
- Update BLEService to enforce timestamp validation for normal packets and exempt valid RSRs
- Add documentation for the new sync manager mechanism

* fix: Resolve compilation errors in V2 Sync implementation

- Remove duplicate restartGossipManager in BLEService
- Add missing TransportConfig constants for sync
- Add 'sync' log category to BitLogger
- Add missing BitLogger import in GossipSyncManager

* fix: Update tests for V2 Sync changes

- Add requestSyncManager parameter to GossipSyncManager init in tests
- Implement getConnectedPeers stub in RecordingDelegate
- Remove unused variable warning in SubscriptionRateLimitTests

---------

Co-authored-by: a1denvalu3 <>
Co-authored-by: jack <212554440+jackjackbits@users.noreply.github.com>
2026-01-17 07:43:02 -10:00
jackandGitHub 10b7c1fd80 Merge branch 'main' into source-routing-packet-format 2026-01-12 10:09:29 -10:00
callebtc 9404c03477 fix doc 2026-01-12 16:56:45 +07:00
callebtc 0f5299a0f5 announce and read presence 2026-01-12 14:12:49 +07:00
callebtc eb3bbfd861 source routing v2 2026-01-12 10:18:04 +07:00
5f44af19da tor by default, small (#564)
* feat(tor): Tor-by-default scaffold and integration

- Add TorManager with static/dlopen start, torrc generation, SOCKS probe
- Add TorURLSession; route Nostr/Web fetches via SOCKS proxy
- Add chat system messages for Tor status; show progress (macOS) and ready
- Disable ControlPort bootstrap monitor on iOS; keep it on macOS
- Make Tor waits non-blocking; avoid main-actor stalls on startup
- Queue & flush Nostr subscriptions on relay connect; skip duplicates
- Always rewrite torrc at launch to fix iOS container path mismatches
- Link libz; add project wiring for tor-nolzma.xcframework
- Minor fixes: SOCKS probe resumeOnce guard, entitlement for network.server (macOS)

* iOS: deterministic Tor recovery + 100% gating; BLE-first; session rebuild

- Restart/wake Tor on foreground via ControlPort (ACTIVE/SHUTDOWN),
  avoid restarts during bootstrap; add NWPathMonitor to trigger checks
- Use NWConnection control polling for GETINFO; remove blocking CFStream
  readers to avoid QoS inversions; compute readiness from SOCKS + 100%
- Rebuild TorURLSession on resume; reset Nostr connections to rebind
- Gate all internet after full bootstrap; keep BLE mesh startup fast
- Fix Swift 6 capture issues; hop UI updates to @MainActor
- Remove Tor progress spam; persist initial "starting tor..." system message

* UI: show Tor system messages only in geohash channels (not mesh)

- Gate "starting tor..." and readiness/timeout messages to geohash view
- Add helper addGeohashOnlySystemMessage() to avoid posting to mesh timeline
- Persist system messages in geohash backing store via addPublicSystemMessage()

* Relays: treat repeated -1011 handshake failures as permanent; skip reconnects

- Classify NSURLErrorBadServerResponse as permanent and stop retrying
- Filter permanently-failed relays from subscribe/connect attempts
- Avoid reconnect scheduling for permanently failed relays

* Embed Tor via tor_api; deterministic restart + Nostr gating; add Tor notifications

- Run Tor via tor_api in a dedicated thread with OwningControllerFD
- Cleanly stop Tor on background; restart on .active (single instance)
- Avoid fallback to tor_main/dlopen; add is-running check to prevent duplicates
- Fix argv lifetime in C glue to avoid strcmp crash on start
- Gate Nostr connect/subscribe/send until Tor is fully ready
- Rebuild URLSession + reset relays after Tor readiness (scene-based)
- Remove TorDidBecomeReady double-reset and appDidBecomeActive resubscribe
- Add TorWillRestart/TorDidBecomeReady notifications and chat system messages
- Debounce path-change restarts; ACTIVE poke first; coalesce subs; cancel stale reconnect timers
- Project: add CTorHost.c and TorNotifications.swift to targets; fix libz.tbd path

* Defer Nostr setup logs until Tor is ready; fix subscribe coalescing and reconnect generation

- Move "Connecting to Nostr relays" log after awaitReady()
- Log "Queuing subscription" when Tor not ready; only coalesce when handler exists
- Clear coalescer on unsubscribe
- Cancel stale reconnect timers using connectionGeneration
- Remove app-level TorDidBecomeReady reset to avoid duplicate reconnects
- Debounce path-change restarts

* Gate Nostr init/subscription logs until Tor is ready

- ChatViewModel: await Tor readiness before initializing Nostr and logging
- Only log GeoDM subscription when Tor is ready to avoid early noise

* Make Nostr connect single-sourced; defer DM subscription until connected

- Remove duplicate connect call from ChatViewModel; let scene-based flow connect
- Setup DM subscription once on first connection via  sink
- Reduce early subscription send/cancel noise after Tor restarts

* On launch, queue Nostr subscriptions without initiating connects; let centralized connect handle it

- In subscribe(), if no connections exist, just list relays and queue subs
- Avoids early send/cancel churn before connect() runs post-Tor-ready

* Always queue subscriptions and flush on connection; avoid immediate sends

- Prevents early send/cancel churn at launch and during reconnects
- If relays are already connected, flush immediately; otherwise pending until connected

* UI: scope Tor restart messages to geohash channels; skip initial foreground restart on cold launch to avoid confusing system message in #mesh

* geo: disable background sampling + notifications\n- Gate sampling to foreground only (beginGeohashSampling, watchers)\n- Suppress geohash activity notifications unless app is active\n- Stop sampling explicitly on background scene phase

* Update BitchatApp.swift

Co-authored-by: asmo <asmogo@protonmail.com>

* Update BitchatApp.swift

Co-authored-by: asmo <asmogo@protonmail.com>

* Update BitchatApp.swift

Co-authored-by: asmo <asmogo@protonmail.com>

* Update bitchat/BitchatApp.swift

Co-authored-by: asmo <asmogo@protonmail.com>

* Update bitchat/BitchatApp.swift

Co-authored-by: asmo <asmogo@protonmail.com>

* fix(iOS App): resolve merge artifacts in scenePhase handler\n- Remove duplicate didEnterBackground state\n- Fix switch/if braces and logic for foreground restart gating

---------

Co-authored-by: jack <jackjackbits@users.noreply.github.com>
Co-authored-by: asmo <asmogo@protonmail.com>
2025-09-11 19:08:43 +02:00
4f1ac30f12 Feat/mesh robustness efficiency (#451)
* chat: de-dup private chats across ephemeral/stable IDs; prefer most advanced delivery status\n\n- Fixes LazyVStack duplicate ID warnings and blank row in PM\n- Merges messages by id from ephemeral and Noise-key stores\n- Chooses read > delivered > partiallyDelivered > sent > sending > failed (newer wins on tie)\n- Ensures status icon updates immediately without waiting for another send\n- Adds exhaustive handling for DeliveryStatus in ranking

* logging: reduce noisy info logs to debug; keep errors/warnings\n\n- Downgrade routing/ACK/subscription/connect logs to debug\n- Retain security/fingerprint/keychain info logs\n- Keep errors and warnings intact\n\ndocs: add docs/privacy-assessment.md covering BLE privacy, routing TTL/jitter, Nostr E2E gift wraps, ACK throttling, and logging posture

---------

Co-authored-by: jack <jackjackbits@users.noreply.github.com>
2025-08-17 21:25:25 +02:00