* CI: bound each parallel-make-check config with its own timeout
A wedged config ran until the job's timeout-minutes killed the whole job.
That reports nothing: the other configs' results are lost, and the upload
step never runs because a job timeout cancels rather than fails.
Add --timeout to parallel-make-check.py to kill and fail just that config,
and set it to 30 min in trackmemory, where 168 of 3508 job runs since
2026-06-12 died this way - every single-config stall one of the two
wolfEntropy configs. Also upload logs on cancellation, and refresh the
stale "minutes" estimates from measured medians.
* CI: drop the runner's third-party apt repos before apt-get update
The hosted images carry Google Chrome and Microsoft apt repos. No wolfSSL
job installs from either, but apt-get update fails if either serves a bad
index, which took out a falcon-interop job on 2026-09-09 with a Hash Sum
mismatch on dl.google.com while the Ubuntu archive was fine.
Remove them by matching the repo host in the file, not by filename or by
wiping sources.list.d, so Ubuntu's own sources survive wherever they live
(24.04 keeps them in sources.list.d/ubuntu.sources). Container jobs are
left alone: those images carry neither repo, nor sudo.
* Address review: keep the trackmemory config list sorted by minutes
The refreshed estimates left all-smallstack-cache-trace (8.9) ahead of
all-wolfentropy (13.4), against the convention documented above the list.
* CI: resolve the ghcr .deb bundle against its own apt index
The bundle was resolved on master right after apt-get update, against the
live archive; consumers resolved against the apt lists frozen into the
runner image days earlier. Any version published in between made
--no-download ask for a .deb the bundle did not carry, and the
all-or-nothing install sent the whole set to the mirror. 25% of PR jobs
took that path, and in 16 of 20 sampled cases the mirror then supplied
zero bytes - the bundle was complete, only the index disagreed.
ci-deps-image now ships a dpkg-scanpackages index in each bundle and
install-apt-deps resolves against only that, as a local file:// repository
with its own lists dir. Producer and consumer agree by construction, so
only a package genuinely absent from the bundle falls back.
Also:
- rebuild the static bundles daily and on a merged package-list change,
instead of weekly
- pq-all gets its own -cross bundle; crossbuild-essential-* were in no
24.04 list, so it fetched 118 MB from the mirror on every run
- ccache-setup installs from the repository install-apt-deps exported,
rather than from debs staged in /var/cache/apt/archives
- check-ci-deps.py fails a PR whose install-apt-deps call names a package
its bundle does not carry, a tag that does not exist, or a tag for the
wrong Ubuntu release; wired into check-source-text
- ci-deps-canary asserts the same contract dynamically after every
rebuild, with require-bundle turning a fallback into an error
- whitebox-smoke used a bare apt-get; it now uses the bundle
* Address review: fail check-ci-deps.py in --matrix/--sets mode
Static contract violations printed ::error but exited 0, so ci-deps-canary
could not fail on them. Those modes also hand stdout to their caller as
data, so findings went to stderr and would have corrupted it.
Route findings to stderr in the two data modes, return non-zero when any
were found, and read --sets output from a file in the canary so the exit
status is not swallowed by process substitution.
* CI: make the .deb bundles reach container jobs
sssd.yml runs in a container, and install-apt-deps skipped the bundle
there for two reasons, both of which left it on the apt mirror - where a
slow archive killed apt-get update twice and failed the job.
1. No docker CLI inside a job container, so the bundle was never pulled.
ghcr-pull.sh pulls it from the registry with curl and tar instead.
2. A bundle is a closure relative to what was already installed where it
was resolved. -full is resolved on the runner, which already has bc and
libcap2, so neither .deb was in it and the all-or-nothing offline
install could never succeed inside the container. ci-deps-image now
takes an `image` per matrix entry and resolves that bundle inside it;
sssd.yml gets its own ubuntu-24.04-sssd bundle, a handful of .debs
rather than 480 MB.
check-ci-deps.py holds the two sides together - a container job must name
a bundle built in its own image, and no other job may name it - and
ci-deps-canary proves each image-tied bundle inside that image after every
rebuild. install-apt-deps also checks the bundle's release against
/etc/os-release, which the static check cannot see for a container job.
Also fix the apt fallback's budget split, which is what actually failed
the job: update was capped at a sixth of an attempt, 50s, well under the
~90s apt's own Acquire retries need to get past a stalled mirror. It is
now half an attempt capped at 90s, taken from the budget still unspent,
so the loop uses the 600s it was given instead of giving up after 105s.
* Address review: record the image the closure was resolved in
bundle-info reported ${ImageOS}/${ImageVersion}, which is the runner's
OS even for a matrix entry whose closure is resolved inside a container.
Report the container image for those entries and keep the runner on its
own line.
pq-all is the only caller whose packages are not a subset of its
bundle: crossbuild-essential-* appear only in the 22.04 lists, and the
offline install is all-or-nothing, so it always takes the apt path -
the one the 45-minute timeout was widened for. The default budget caps
each install attempt at 250s and hard-fails a merely slow download.
Also state the real budget-seconds arithmetic: the loop overshoots by
retry-delay plus the SIGKILL grace, the floors make small values inert,
and the defaults need ~16 minutes rather than the 15 claimed.
This loop has no budget input and chains onto install-apt-deps in the
same job, several of which are capped at 10 minutes. Drop the install
deadline to 90s (ccache is ~700 KB) and stop sleeping after the final
failed attempt: worst case 510s -> 250s.
The per-attempt deadlines did not fit callers with a four minute
timeout-minutes, so a wedged mirror cancelled the job before the loop
could report it. Spend a configurable budget across the attempts instead
and set one that fits on the short jobs.
sudo resets the environment, so an exported DEBIAN_FRONTEND never
reached apt-get and a debconf prompt could block until the timeout.
cross-library's containers are bare images with no bash, so Actions runs the
step under sh; the bash array APT_OPTS=(...) was a syntax error and took out
all six cross-library builds. Use a plain word-splitting variable.
The retry budget also has to fit the caller's timeout-minutes - install-apt-deps
is used by jobs with 4 minutes - or the last attempt is cut off before it can
report. Two attempts at 60s/300s instead of three at 120s/900s. The Acquire
timeouts are what actually detect a wedge, so the outer bound only backstops
apt wedging outside its own I/O loop, and apt resumes from archives/partial/.
Also run dpkg --configure -a before the offline ccache probe, not only in the
fallback loop: an interrupted dpkg would otherwise push it to the mirror.
ccache-setup ran a raw apt-get update, so jobs whose ghcr .deb bundle had
already installed cleanly still reached the mirror. On 2026-08-19 that step
stalled on archive.ubuntu.com for 9-43 min and took out 11 jobs. The .deb is
already in /var/cache/apt/archives - install-apt-deps stages the whole bundle
- so install it with --no-download first and only fall back to the mirror.
ccache was missing from the 22.04-minimal and linuxkm bundles; add it.
Neither the fallback in install-apt-deps nor any of the raw apt sites had a
timeout, so a wedged mirror hung instead of failing and the retry loops never
fired. ci-deps-image already solved this for itself (Acquire timeouts plus
`timeout`); apply the same to the consumers, and to the linuxkm producer job
that was still missing it. Also recover from a dpkg interrupted by a kill,
and bound the bundle pull.
arduino.yml's per-core actions/cache layer stored the installed cores and
toolchains (~/.arduino15) - several GB, dominated by the esp32 and mbed
cores - in the 10 GB Actions cache. For esp32 it was also ineffective: the
disk-cleanup step deletes the esp32 toolchain before actions/cache saves it,
so esp32 re-downloaded every run anyway.
- New arduino-cores-image workflow resolves each of the 9 distinct cores and
publishes a tar of ~/.arduino15 + ~/Arduino/libraries to
ghcr.io/<owner>/wolfssl-ci-arduino:<core>. It runs monthly: esp32, the
fastest-moving core, releases ~monthly and the rest far less often.
- New install-arduino-core composite action restores that bundle offline and
verifies the core is present, falling back to `arduino-cli core install`
when the bundle is unavailable - so nothing breaks until the image is first
published and made public.
- arduino.yml calls the action in place of the inline core install and the
actions/cache step.
This takes the flaky espressif / esp8266.com / pjrc.com downloads off the PR
critical path and frees the Actions cache of the largest binaries it held.
The ci-cache-offload work added a ghcr .deb bundle path to
install-apt-deps, making the actions/cache apt-archive layer redundant.
Remove it so no apt-deps-* cache entries are produced. Apt packages now
install either offline from the ghcr bundle (when ghcr-debs-tag is set)
or via plain apt-get with the existing retry/backoff.
- Strip the Compute/Restore/Pre-seed/Collect/Save cache steps and the
cache-hit fast path; drop the now-unused 'cache' input.
- Update callers that passed 'cache': membrowse-onboard, membrowse-report
(and the apt_cache matrix key in membrowse-targets.json), and sssd.
The ghcr offline path and the ccache actions/cache usage are untouched.
Rebasing onto master (which migrated JS actions to Node.js 24 runtimes)
left a few action refs that this branch added in new steps still on the
old major versions. Bring them in line with master:
- ccache-setup read-only restore: actions/cache/restore@v4 -> @v5
- smoke-test / os-check ccache save: actions/cache/save@v4 -> @v5
- ci-deps-image checkout: actions/checkout@v4 -> @v5
- os-check.yml linux shard: add a schedule-gated CCACHE_RECACHE=1 step so
the weekday seed reseeds from clean compiles rather than only accumulating
deltas. This shard manages ccache directly (its own restore/save) and so
was not covered by the ccache-setup composite's reseed.
- install-apt-deps: hardcode the ghcr bundle owner to wolfssl. The bundle is
only published under ghcr.io/wolfssl by ci-deps-image, so fork PRs now read
the public upstream image instead of a nonexistent ghcr.io/<fork>/wolfssl-ci-debs.
- ccache-setup: document that the read-only restore key reuses the save
key shape for symmetry and is never an exact hit by design.
Skoll F3 (a packages-subset-of-bundle CI guard) is deferred to a follow-up;
F4 (release-branch ccache saves) is left as the intended seed-on-schedule /
everything-else-reads model.
Tighten three pieces of documentation to match the implementation; no
behaviour change:
- install-apt-deps (ghcr-debs-tag description): the apt mirror is avoided
only on the successful offline path. The offline install is a single
--no-download install of the whole package set, so any miss (bundle
absent/private/incomplete) falls back to the apt path.
- ci-deps-image header: each bundle is every requested package plus the
dependencies not already present on the matching runner image - tied to
that runner, not a portable/self-contained .deb closure.
- ci-deps-image schedule note: a package missing from the bundle fails the
whole offline install (it is not per-package), falling back to the full
apt path.
- ccache-setup: gate the scheduled-reseed step with
`if: github.event_name == 'schedule'` again. The github context IS
available in a composite action's step-level if: (install-apt-deps
already relies on it), so the earlier $GITHUB_EVENT_NAME workaround and
its comment were based on a wrong premise. The real load failure was the
${{ }} expression in the read-only input description, fixed separately.
- smoke-test.yml: include github.event_name in the concurrency group. The
workflow pushes to master/main and now also runs on a weekday schedule;
both share github.ref on the default branch, so under
cancel-in-progress a seed run and a master push could cancel each other.
The read-only input description embedded `${{ github.event_name ==
'pull_request' }}` as example text. GitHub validates ${{ }} expressions in
an action's input definitions at manifest-load time, where the github
context is not available, so the action failed to load ("Unrecognized
named-value: 'github'", action.yml line 27) and every ccache-setup
consumer died at "Set up ccache" (build library, make check, Compiler
test, Multi-arch test, ...). Describe the expression in prose instead of
embedding it as a live ${{ }} template; the github.* references that
remain are in step with:/run: blocks, where the context is available.
The scheduled-reseed step gated CCACHE_RECACHE with
`if: github.event_name == 'schedule'`, but the github context is not
available in a composite action's step-level if:. The action manifest
therefore failed to load ("Unrecognized named-value: 'github'"), and
every workflow using ccache-setup broke at the "Set up ccache" step
(build library, make check, Compiler test, Multi-arch test, ...).
Gate on the built-in $GITHUB_EVENT_NAME env var in the shell instead,
which keeps the schedule-only reseed behaviour with no caller changes.
The scheduled (cron) refresh restored the prior ccache and recompiled
only the translation units that changed, so unchanged objects were never
rebuilt and the shared cache could drift indefinitely. Set
CCACHE_RECACHE=1 on schedule events - gated inside the ccache-setup
action, so none of the calling workflows change - to force fresh
compiles that re-store every result. PR and push runs are unaffected and
keep their warm hits; only the scheduled jobs pay the full recompile.
- install-apt-deps: the ghcr offline-install path now honors the
no-install-recommends input; it was always installing recommends,
diverging from the regular apt path.
- install-apt-deps: correct the ghcr-debs-tag example to a real tag
(ubuntu-24.04-minimal) - ci-deps-image publishes -minimal/-full
variants, not a bare <ver> tag.
- os-check: fix the schedule header comment - macOS runs --build-only on
the weekday cron to seed its ccache (like the linux shards); only
Windows is skipped on schedule.
The 10 GB, LRU-evicted, PR-scoped Actions cache was being thrashed - the
docker simulator buildx layers (~6 GiB), plus per-PR ccache and apt-archive
writes whose keys never hit - which kept evicting the shared ccache, while
the apt mirror timed out often enough to break PR CI. Move the heavy caches
to ghcr (free, separate pool) and make PR runs read-only against the Actions
cache.
apt dependencies from prebuilt ghcr .deb bundles
- ci-deps-image.yml resolves each package list under .github/ci-deps/ into
its .deb closure and publishes ghcr.io/<owner>/wolfssl-ci-debs:<tag> in
two tiers: <ver>-minimal (make-check family) and <ver>-full (interop
superset), for ubuntu-22.04 and 24.04.
- install-apt-deps gains a ghcr-debs-tag input: pull the bundle and install
offline (--no-download) so the apt mirror is never on the PR critical
path. Any failure (bundle missing/not public/incomplete) falls through to
the existing apt path, so it is always safe to set.
sim-test buildx layers to a shared ghcr registry cache
- the 7 docker simulator workflows switch from cache-to: type=gha to
ghcr.io/wolfssl/wolfssl-sim-cache:<scope>. cache-from reads on every run
(anonymous); cache-to writes only on the weekend cron and manual
workflow_dispatch. Per-distinct-image tags and de-duplicated writers keep
parallel matrix jobs from racing on one ref.
ccache: PRs read, the schedule writes
- ccache-setup gains read-only: PR runs restore the shared master-scoped
cache but never upload; schedule/push runs refresh it. Wired across
os-check (linux + macOS), pq-all, smoke-test and the 12 small make-check
workflows.
- parallel-make-check.py gains --build-only (compile every config, skip the
test phase) so weekday-morning seed crons warm the cache PR runs consume.
artifact retention capped at 7 days on the failure-log/result uploads that
previously defaulted to 90.
ONE-TIME SETUP: after their first publish, make the ghcr packages
wolfssl-ci-debs and wolfssl-sim-cache PUBLIC so anonymous pulls work from PR
(including fork) runs; until then everything falls back cleanly.
GitHub Actions now emits "Node.js 20 actions are deprecated" warnings:
actions are forced to Node.js 24 by default starting 2026-06-16, and
Node.js 20 is removed from the runners on 2026-09-16. Update every
JavaScript action referenced by the workflows and the local composite
actions to the lowest release that runs on Node.js 24:
actions/checkout v4 -> v5
actions/checkout (SHA pin) v4.1.7 -> v5
actions/upload-artifact v4 -> v6 (v5 still Node.js 20)
actions/download-artifact v4 -> v7 (v5/v6 still Node.js 20)
actions/cache[/restore|/save] v4 -> v5
actions/setup-python v5 -> v6
actions/github-script v7 -> v8
docker/setup-buildx-action v3 -> v4
docker/build-push-action v5 -> v7 (v6 still Node.js 20)
docker/login-action v3 -> v4
microsoft/setup-msbuild v2 -> v3
open-watcom/setup-watcom v0 -> v1
Actions already running on Node.js 24 (jwlawson/actions-setup-cmake,
shogo82148/actions-setup-perl, msys2/setup-msys2, dorny/paths-filter)
are left unchanged. These bumps are runtime-only; no workflow uses an
input or output removed by the new majors, and v4-format artifacts
remain compatible across the upload v6 / download v7 backends.
Replace the one-runner-per-configuration matrices across the
make-check workflow family with a generic pooled runner,
.github/scripts/parallel-make-check.py. Each workflow keeps its
configuration list as JSON next to the invocation; one runner (or a
small fixed set of shards, balanced by measured per-config minutes)
builds every config in its own out-of-tree (VPATH) build directory off
a single checkout/autogen, on a pool of one-per-CPU worker threads,
longest first. Concurrent checks are isolated with bubblewrap network
namespaces, compilations are cached with ccache, the first failure
aborts the rest (fail-fast, with --no-fail-fast to run everything),
and per-config timings plus pool efficiency land in the step summary.
Failure logs upload as artifacts. smoke-test.yml is likewise reworked
into a single pooled job that runs its nine configs on one runner.
Converted workflows (runner jobs per full pass):
os-check.yml 101 -> 8 (92 Ubuntu configs -> 4 shards;
the macOS matrix, the user-settings jobs and
the standalone
macos-apple-native-cert-validation.yml fold
into one macOS runner; Windows unchanged)
pq-all.yml 21 -> 2 shards
disable-pk-algs.yml 15 -> 1
wolfCrypt-Wconversion.yml 11 -> 1
trackmemory.yml 7 -> 1
cryptocb-only.yml 8 -> 1 (incl. the two new SHA512 entries)
multi-compiler.yml 6 -> 1
smallStackSize.yml 6 -> 1
multi-arch.yml 6 -> 1
async.yml 5 -> 1
psk.yml 5 -> 1
no-malloc.yml 3 -> 1
wolfsm.yml 3 -> 1
opensslcoexist.yml 2 -> 1
Measured against current upstream passing runs (job execution time,
queue excluded): ~200 runner jobs / ~374 runner-minutes per full pass
become 23 jobs / ~168 runner-minutes, with more coverage than before.
multi-arch's old matrix combined an "include" list of four
architectures with an "opts" axis; GitHub's include-merge rules made
each arch entry overwrite the previous one, so only the armel
combinations actually ran. The pooled list restores the intended
aarch64/armhf/riscv64 coverage (23 combinations; riscv64 x sp-math is
omitted as invalid - configure rejects sp-math without SP, and
--enable-riscv-asm, unlike --enable-sp-asm, does not bring SP in).
Out-of-tree build fixes this depends on:
- Makefile.am: symlink the read-only test data (certs/, tests/ config
files, sniffer captures and helpers, examples/crypto_policies,
input, quit) into the build tree via a BUILT_SOURCES stamp, removed
again in distclean-local. ChangeToWolfRoot() and the script tests
resolve everything relative to the working directory, so out-of-tree
make check and make distcheck now pass.
- scripts/multi-msg-record.py: locate the client binary from the build
tree working directory rather than the script's source directory.
- configure.ac + wolfssl/include.am: run
support/gen-debug-trace-error-codes.sh from $srcdir; it reads the
error-code headers from the source tree and generates into the build
tree.
- tests/swdev: a WOLFBUILD variable points the sub-make at the build
tree for the configure-generated headers (wolfssl/options.h,
wolfssl/version.h); the in-tree-only guards are dropped.
Portions of PR #10649 are incorporated: the cross-platform
ccache-setup composite action, repository_owner gates on check-headers
and check-source-text, the docs-only paths-ignore on os-check, and the
libspdm timeout bumps.
