mirror of
https://github.com/permissionlesstech/bitchat.git
synced 2026-07-24 22:45:19 +00:00
Run perf benchmarks serially in their own CI step
The intermittent CI hang was caught by the new job timeout: the run froze on the last remaining parallel test slot, an XCTest measure benchmark (testNostrInboundEventHandling_freshEvents), after 4 hangs in 5 runs - while the same suite completes in seconds locally and the test itself is bounded. Independent of the micro-cause, benchmarks do not belong inside the parallel suite: measuring while test processes contend for cores is where our 2x CI variance came from. The parallel run now skips benchmarks (BITCHAT_SKIP_PERF_BASELINES=1) and a dedicated serial step runs them on an otherwise idle runner with a 6-minute step timeout, feeding the floor gate as before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -42,17 +42,25 @@ jobs:
|
||||
${{ runner.os }}-${{ matrix.name }}-
|
||||
|
||||
- name: Run Tests
|
||||
# BITCHAT_PERF_LOG captures the PERF[...] lines that
|
||||
# PerformanceBaselineTests reports (swift test --parallel swallows
|
||||
# stdout of passing tests, so the floor gate reads this file instead).
|
||||
# Perf benchmarks are excluded here and run in their own serial step
|
||||
# below: measuring while parallel test processes contend for cores
|
||||
# produces noisy numbers, and the XCTest measure machinery has hung
|
||||
# intermittently under parallel workers on loaded runners.
|
||||
env:
|
||||
BITCHAT_PERF_LOG: ${{ github.workspace }}/perf-output.log
|
||||
BITCHAT_SKIP_PERF_BASELINES: "1"
|
||||
run: swift test --parallel --quiet --enable-code-coverage --package-path ${{ matrix.path }}
|
||||
|
||||
# Order-of-magnitude performance regression gate (app tests only — the
|
||||
# package matrix entries write no PERF lines and the gate would skip
|
||||
# anyway). Floors are deliberately generous (~25% of healthy local
|
||||
# throughput, see bitchatTests/Performance/perf-floors.json) so this
|
||||
# Benchmarks run serially on an otherwise idle runner for stable
|
||||
# numbers; BITCHAT_PERF_LOG captures the PERF[...] lines for the gate.
|
||||
- name: Run performance benchmarks (serial)
|
||||
if: matrix.name == 'app'
|
||||
timeout-minutes: 6
|
||||
env:
|
||||
BITCHAT_PERF_LOG: ${{ github.workspace }}/perf-output.log
|
||||
run: swift test --quiet --filter PerformanceBaselineTests
|
||||
|
||||
# Order-of-magnitude performance regression gate. Floors are deliberately
|
||||
# generous (see bitchatTests/Performance/perf-floors.json) so this
|
||||
# catches algorithmic regressions, never runner variance.
|
||||
- name: Performance floor gate
|
||||
if: matrix.name == 'app'
|
||||
|
||||
Reference in New Issue
Block a user