permissionlesstech/bitchat
1
0
Fork 0
mirror of https://github.com/permissionlesstech/bitchat.git synced 2026-07-24 21:45:20 +00:00
Code Issues Packages Projects Releases Wiki Activity

Recalibrate perf floors to slowest-observed-CI basis

Browse Source 
The gate tripped on a uniformly slow runner: every benchmark ran at
~2/3 of the previous CI run and nostrInbound.duplicate fell to 87% of
its floor. Root cause: floors were derived from local numbers, but CI
slowdown is benchmark-dependent - sub-millisecond passes amplify runner
overhead (the duplicate path runs at ~20% of local speed on CI while
most benchmarks run at 40-60%). Floors are now ~50% of the slowest
observed CI run, recorded alongside the local references. Every floor
remains 10-200x above known regression values (the pre-optimization
duplicate path measured 2.2k/sec against the 250k floor), so order-of-
magnitude regressions still fail loudly. Verified against the slow
run's numbers: all 11 pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
jack
2026-06-11 21:45:59 +02:00
co-authored by Claude Fable 5
parent 1a6a08f92a
commit 4791114406
1 changed files with 29 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
bitchatTests/Performance/perf-floors.json
+29 -13
View File
@@ -2,11 +2,14 @@
"_philosophy": [
"Floor throughputs for the PERF[...] lines printed by PerformanceBaselineTests.",
"Floors catch algorithmic regressions (O(n) -> O(n^2), accidental sync I/O,",
"quadratic re-scans), NOT tuning noise: each floor is deliberately set at",
"~25% of the throughput measured on a local dev machine (2026-06, Apple",
"Silicon), leaving ~4x headroom for CI runner variance so the gate never",
"flakes on a slow runner while still failing loudly on order-of-magnitude",
"regressions.",
"quadratic re-scans), NOT tuning noise. Basis: ~50% of the SLOWEST observed",
"CI run (GitHub macos-latest), not local numbers - CI slowdown is",
"benchmark-dependent (sub-millisecond passes amplify runner overhead, e.g.",
"nostrInbound.duplicate runs at ~20% of local speed on CI while most",
"benchmarks run at ~40-60%). Every floor remains 10-200x above known",
"regression values (the pre-optimization duplicate path measured 2.2k/sec",
"against a 250k floor), so the gate still fails loudly on order-of-",
"magnitude regressions while never flaking on a slow runner.",
"Raise floors deliberately after intentional performance improvements;",
"lower them only with a written justification in the PR. If a benchmark is",
"renamed or removed, update this file in the same change - the gate fails",
@@ -28,16 +31,29 @@
"store.audit": 362
},
"floors": {
"nostrInbound.fresh": 530,
"nostrInbound.duplicate": 600000,
"nostrInbound.fresh": 450,
"nostrInbound.duplicate": 250000,
"bleInbound.roundTripAndDedup": 9500,
"gcs.buildAndDecode": 190,
"delivery.incrementalUpdate": 43000,
"delivery.storeUpdate": 39000,
"formatting.formatMessage": 3000,
"pipeline.privateIngest": 6000,
"pipeline.publicIngest": 3200,
"store.append": 53000,
"store.audit": 90
"formatting.formatMessage": 2200,
"pipeline.privateIngest": 3000,
"pipeline.publicIngest": 2400,
"store.append": 48000,
"store.audit": 70
},
"_slowest_observed_ci_numbers_2026_06": {
"nostrInbound.fresh": 918,
"nostrInbound.duplicate": 524924,
"bleInbound.roundTripAndDedup": 20489,
"gcs.buildAndDecode": 504,
"delivery.incrementalUpdate": 110892,
"delivery.storeUpdate": 96531,
"formatting.formatMessage": 4575,
"pipeline.privateIngest": 6388,
"pipeline.publicIngest": 5006,
"store.append": 97423,
"store.audit": 140
}
}
}