mirror of
https://github.com/permissionlesstech/bitchat.git
synced 2026-07-25 16:45:19 +00:00
* Exclude perf baselines from parallel CI via --skip; sample hung tests Every app-suite CI run since the perf baselines landed (#1335) has timed out at the 15-minute job limit — main has been red for five consecutive runs. The job logs show the PerformanceBaselineTests fixtures dispatched into the parallel phase despite the BITCHAT_SKIP_PERF_BASELINES env guard from #1336, followed by ~11 minutes of silence until the timeout kills swiftpm-testing. The suite passes locally in seconds with identical flags, so the hang is specific to the CI toolchain/runners — consistent with the known XCTest-measure-under-parallel-workers hang the serial step was created to avoid. Two changes: - Exclude the baselines from the parallel phase with --skip at the SPM level, which removes them from the worker processes entirely instead of relying on the env guard reaching setUpWithError. They still run (and gate) in the dedicated serial step. - Wrap the parallel run in a 10-minute watchdog that samples any still-running test processes before killing them, so if anything else ever hangs, the run fails fast with thread stacks in the log instead of a silent 15-minute timeout. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Arm the test-hang watchdog only for the test execution phase Codex review: swift test builds before running, so the watchdog timer included dependency resolution and compilation — a cold-cache coverage build on a slow runner could be killed before tests ever started. Build the tests in their own step (bounded by the 15-minute job timeout like any build) and run the watchdog around swift test --skip-build, tightened to 5 minutes now that it times only test execution. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Hammer transport thread-safety tests from the dispatch pool, not the cooperative pool The watchdog added in this PR captured the actual CI hang twice, with identical stacks both times: NostrTransportTests' 100-task groups park every Swift Concurrency cooperative thread in a blocking queue.sync (the pool has one thread per core — 3 on CI runners, 10+ on dev machines, which is why this never reproduced locally). Blocking the entire cooperative pool violates the forward-progress contract, and the runners' dispatch wedges under the resulting asyncAndWait flood — taking concurrently running tests down with it (the panic-reset test deadlocked in a serviceQueue barrier that never got scheduled). Run the same 100 concurrent hammer iterations via DispatchQueue.concurrentPerform from a single global-queue hop instead: identical thread-safety coverage, executed on dispatch worker threads where blocking is legal, zero cooperative threads parked. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: jack <jackjackbits@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
143 lines
5.7 KiB
YAML
143 lines
5.7 KiB
YAML
name: Build & Test
|
|
|
|
on:
|
|
push:
|
|
branches:
|
|
- main
|
|
pull_request:
|
|
|
|
jobs:
|
|
test:
|
|
name: Run Swift Tests (${{ matrix.name }})
|
|
runs-on: macos-latest
|
|
# A hung test must fail fast, not hold a runner for GitHub's 360-minute
|
|
# default (observed: intermittent app-suite hangs starving the queue).
|
|
timeout-minutes: 15
|
|
|
|
strategy:
|
|
fail-fast: false # Don't cancel other matrix jobs when one fails
|
|
matrix:
|
|
include:
|
|
- name: app
|
|
path: .
|
|
- name: BitLogger
|
|
path: localPackages/BitLogger
|
|
- name: BitFoundation
|
|
path: localPackages/BitFoundation
|
|
|
|
steps:
|
|
- name: Checkout code
|
|
uses: actions/checkout@v5
|
|
|
|
- name: Set up Swift
|
|
uses: swift-actions/setup-swift@v2
|
|
|
|
- name: Cache build artifacts
|
|
uses: actions/cache@v4
|
|
with:
|
|
path: ${{ matrix.path }}/.build
|
|
key: ${{ runner.os }}-${{ matrix.name }}-${{ hashFiles(format('{0}/**/*.swift', matrix.path), format('{0}/**/Package.resolved', matrix.path)) }}
|
|
restore-keys: |
|
|
${{ runner.os }}-${{ matrix.name }}-${{ hashFiles(format('{0}/**/Package.resolved', matrix.path)) }}
|
|
${{ runner.os }}-${{ matrix.name }}-
|
|
|
|
- name: Build tests
|
|
# Built separately so the hang watchdog below times only test
|
|
# execution: a cold-cache coverage build on a slow runner can
|
|
# legitimately take several minutes, and is already bounded by the
|
|
# 15-minute job timeout.
|
|
run: swift build --build-tests --enable-code-coverage --package-path ${{ matrix.path }}
|
|
|
|
- name: Run Tests
|
|
# Perf benchmarks are excluded here and run in their own serial step
|
|
# below: measuring while parallel test processes contend for cores
|
|
# produces noisy numbers, and the XCTest measure machinery has hung
|
|
# intermittently under parallel workers on loaded runners. Excluded
|
|
# via --skip (not just the env guard): every app run since the
|
|
# baselines landed timed out at the 15-minute job limit with the
|
|
# baseline tests dispatched into the parallel phase.
|
|
#
|
|
# The watchdog samples any still-running test processes after 5
|
|
# minutes (the suite passes in seconds when healthy; the build is
|
|
# done by this step) and kills the run, so a hang fails fast with
|
|
# stacks in the log instead of a silent timeout.
|
|
env:
|
|
BITCHAT_SKIP_PERF_BASELINES: "1"
|
|
run: |
|
|
swift test --skip-build --parallel --quiet --enable-code-coverage \
|
|
--skip PerformanceBaselineTests \
|
|
--package-path ${{ matrix.path }} &
|
|
test_pid=$!
|
|
(
|
|
sleep 300
|
|
if kill -0 "$test_pid" 2>/dev/null; then
|
|
echo "::group::Tests still running after 5 minutes — sampling before kill"
|
|
for pid in $(pgrep -if 'swiftpm-testing|xctest|PackageTests' || true); do
|
|
echo "--- sample of pid $pid ---"
|
|
sample "$pid" 5 2>/dev/null || true
|
|
done
|
|
echo "::endgroup::"
|
|
pkill -KILL -P "$test_pid" 2>/dev/null || true
|
|
kill -KILL "$test_pid" 2>/dev/null || true
|
|
fi
|
|
) &
|
|
watchdog_pid=$!
|
|
wait "$test_pid" && status=0 || status=$?
|
|
kill "$watchdog_pid" 2>/dev/null || true
|
|
exit "$status"
|
|
|
|
# Benchmarks run serially on an otherwise idle runner for stable
|
|
# numbers; BITCHAT_PERF_LOG captures the PERF[...] lines for the gate.
|
|
- name: Run performance benchmarks (serial)
|
|
if: matrix.name == 'app'
|
|
timeout-minutes: 6
|
|
env:
|
|
BITCHAT_PERF_LOG: ${{ github.workspace }}/perf-output.log
|
|
run: swift test --quiet --filter PerformanceBaselineTests
|
|
|
|
# Order-of-magnitude performance regression gate. Floors are deliberately
|
|
# generous (see bitchatTests/Performance/perf-floors.json) so this
|
|
# catches algorithmic regressions, never runner variance.
|
|
- name: Performance floor gate
|
|
if: matrix.name == 'app'
|
|
run: ./scripts/check-perf-floors.sh perf-output.log
|
|
|
|
# Informational only: surfaces per-file and total line coverage in the
|
|
# job log so coverage trends are visible on every PR. No thresholds —
|
|
# this must never be the reason a build goes red.
|
|
- name: Coverage summary
|
|
run: |
|
|
BIN_PATH=$(swift build --show-bin-path --package-path ${{ matrix.path }})
|
|
PROF="$BIN_PATH/codecov/default.profdata"
|
|
XCTEST=$(find "$BIN_PATH" -maxdepth 1 -name '*.xctest' | head -1)
|
|
BINARY="$XCTEST/Contents/MacOS/$(basename "$XCTEST" .xctest)"
|
|
if [ -f "$PROF" ] && [ -f "$BINARY" ]; then
|
|
xcrun llvm-cov report "$BINARY" -instr-profile "$PROF" \
|
|
-ignore-filename-regex='(Tests|\.build|checkouts|Mocks|_PreviewHelpers)' || true
|
|
else
|
|
echo "No coverage data found; skipping summary."
|
|
fi
|
|
|
|
# SPM tests above only compile the macOS slice; this job covers the
|
|
# iOS-conditional code paths (UIKit, CoreBluetooth restoration, etc.).
|
|
ios-build:
|
|
name: Build iOS app (simulator)
|
|
runs-on: macos-latest
|
|
timeout-minutes: 15
|
|
|
|
steps:
|
|
- name: Checkout code
|
|
uses: actions/checkout@v5
|
|
|
|
- name: Build iOS (simulator, no signing)
|
|
# arm64 only: the vendored arti.xcframework has no x86_64 simulator slice.
|
|
run: |
|
|
set -o pipefail
|
|
xcodebuild -project bitchat.xcodeproj \
|
|
-scheme "bitchat (iOS)" \
|
|
-sdk iphonesimulator \
|
|
-destination 'generic/platform=iOS Simulator' \
|
|
ARCHS=arm64 \
|
|
CODE_SIGNING_ALLOWED=NO \
|
|
build
|