Skip to content

Conversation

@spalladino
Copy link
Contributor

Summary

  • Adds an optional local filesystem cache layer for build artifacts that sits in front of the S3 remote cache, controlled via the CACHE_LOCAL_DIR env var
  • On download: checks local cache first (instant), on miss downloads from S3 and saves to local cache for next time
  • On upload: saves the artifact to the local cache alongside the S3 upload
  • When CACHE_LOCAL_DIR is unset, behavior is identical to before (zero impact on CI or other devs)
  • Gracefully falls through to S3 if the local cache directory cannot be created (e.g. permission issues)

This reduces full bootstrap time from 165s to 77s by avoiding redundant S3 downloads for artifacts that haven't changed.

Usage

export CACHE_LOCAL_DIR="$HOME/.aztec-cache"
./bootstrap.sh

Test plan

  • Added ci3/cache_local.test.sh with 8 tests (12 assertions) covering: local cache hit/miss, upload save, roundtrip, disabled-cache bypass, inaccessible directory fallthrough
  • Verified no rm -rf on any path derived from CACHE_LOCAL_DIR
  • Run bash ci3/cache_local.test.sh to execute tests

🤖 Generated with Claude Code

@AztecBot
Copy link
Collaborator

AztecBot commented Feb 9, 2026

Flakey Tests

🤖 says: This CI run detected 1 tests that failed, but were tolerated due to a .test_patterns.yml entry.

\033FLAKED\033 (8;;http://ci.aztec-labs.com/1843848965a8dceb�1843848965a8dceb8;;�):  yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_epochs/epochs_mbps.parallel.test.ts "builds multiple blocks per slot with transactions anchored to proposed blocks" (218s) (code: 1) group:e2e-p2p-epoch-flakes


echo_stderr "Local cache miss for $tar_file, downloading from remote."

if [[ -n "${S3_BUILD_CACHE_AWS_PARAMS:-}" ]]; then
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not a fan of this redundancy. extract a function?

@ludamad
Copy link
Collaborator

ludamad commented Feb 9, 2026

mostly lgtm, just the one comment. we should be able to extract a function that puts the tar into stdout that can be used in both cases

@spalladino
Copy link
Contributor Author

@ludamad done

@ludamad
Copy link
Collaborator

ludamad commented Feb 10, 2026

Note that we don't really cache upload locally so that change isn't super necessary but I'm ok with it

Should speed up bootstrap when changing back to a branch for which we
had already downloaded artifacts.
@spalladino spalladino force-pushed the palla/ci3-local-cache branch from a9a7713 to 0907b28 Compare February 10, 2026 12:51
@spalladino spalladino enabled auto-merge (squash) February 10, 2026 12:53
@spalladino spalladino merged commit dfdd2e3 into merge-train/spartan Feb 10, 2026
11 checks passed
@spalladino spalladino deleted the palla/ci3-local-cache branch February 10, 2026 13:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants