permissionlesstech/bitchat
1
0
Fork 0
mirror of https://github.com/permissionlesstech/bitchat.git synced 2026-07-25 15:05:20 +00:00
Code Issues Packages Projects Releases Wiki Activity

Run perf benchmarks serially in their own CI step

Browse Source 
The intermittent CI hang was caught by the new job timeout: the run
froze on the last remaining parallel test slot, an XCTest measure
benchmark (testNostrInboundEventHandling_freshEvents), after 4 hangs
in 5 runs - while the same suite completes in seconds locally and the
test itself is bounded. Independent of the micro-cause, benchmarks
do not belong inside the parallel suite: measuring while test processes
contend for cores is where our 2x CI variance came from. The parallel
run now skips benchmarks (BITCHAT_SKIP_PERF_BASELINES=1) and a
dedicated serial step runs them on an otherwise idle runner with a
6-minute step timeout, feeding the floor gate as before.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
jack
2026-06-11 22:18:55 +02:00
co-authored by Claude Fable 5
parent 7fbe6a4a7e
commit fa136d8973
1 changed files with 16 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/swift-tests.yml
+16 -8
View File
@@ -42,17 +42,25 @@ jobs:
${{ runner.os }}-${{ matrix.name }}- ${{ runner.os }}-${{ matrix.name }}-
- name: Run Tests - name: Run Tests
# BITCHAT_PERF_LOG captures the PERF[...] lines that # Perf benchmarks are excluded here and run in their own serial step
# PerformanceBaselineTests reports (swift test --parallel swallows # below: measuring while parallel test processes contend for cores
# stdout of passing tests, so the floor gate reads this file instead). # produces noisy numbers, and the XCTest measure machinery has hung
# intermittently under parallel workers on loaded runners.
env: env:
BITCHAT_PERF_LOG: ${{ github.workspace }}/perf-output.log BITCHAT_SKIP_PERF_BASELINES: "1"
run: swift test --parallel --quiet --enable-code-coverage --package-path ${{ matrix.path }} run: swift test --parallel --quiet --enable-code-coverage --package-path ${{ matrix.path }}
# Order-of-magnitude performance regression gate (app tests only — the # Benchmarks run serially on an otherwise idle runner for stable
# package matrix entries write no PERF lines and the gate would skip # numbers; BITCHAT_PERF_LOG captures the PERF[...] lines for the gate.
# anyway). Floors are deliberately generous (~25% of healthy local - name: Run performance benchmarks (serial)
# throughput, see bitchatTests/Performance/perf-floors.json) so this if: matrix.name == 'app'
timeout-minutes: 6
env:
BITCHAT_PERF_LOG: ${{ github.workspace }}/perf-output.log
run: swift test --quiet --filter PerformanceBaselineTests
# Order-of-magnitude performance regression gate. Floors are deliberately
# generous (see bitchatTests/Performance/perf-floors.json) so this
# catches algorithmic regressions, never runner variance. # catches algorithmic regressions, never runner variance.
- name: Performance floor gate - name: Performance floor gate
if: matrix.name == 'app' if: matrix.name == 'app'