Skip to content

Commit

Permalink
fix(ci): use PR addrs to compare layouts (#139)
Browse files Browse the repository at this point in the history 
* fix(ci): use PR addrs to compare layouts

* fix(ci): use PR addrs to compare layouts

The `compare-layouts.yml` workflow runs via `workflow_run` and is thus
triggered in the context of the base branch. It is, therefore, reading
potentially outdated information from the `deployedContract.json` file.
To avoid that, we use an intermediate artifact generated from the PR,
which contains the updated addresses, during the step that fetches the
storage layouts from Etherscan.

Simpler validation, such as all required addresses are present and
correctly formatted, is retained in the parent CI to permit quicker
short circuit upon receiving invalid data.

* feat: combine extract layout CI into one

No need to have separate job for base and non-base branches

* fix(ci): update matrix

* fix(ci): use correct key for artifact

* fix(ci): remove not working restore

* fix(ci): checkout recursive module

* fix(ci): exclude unnecessary file from artifact

* fix(ci): rename layout to `.proposed.json`

* remove superfluous line

* fix(ci): drop `validatedContracts.json` correctly

* trigger a test CI run

* fix(ci): use correct keys for validated contracts

* fix(ci): use the original names for layouts

To avoid unnecessary failure and workarounds, use `.compiled.json` where
possible

* fix: add `set -e`
  • Loading branch information
@MaxMustermann2
MaxMustermann2 authored Jan 20, 2025
1 parent a887167 commit 02e51e3
Show file tree
Hide file tree
Showing 2 changed files with 102 additions and 69 deletions.
33 changes: 12 additions & 21 deletions .github/workflows/compare-layouts.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -105,32 +105,20 @@ jobs:
outputs:
matrix: ${{ steps.generate-matrix.outputs.matrix }}
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Download validated contracts json
uses: dawidd6/action-download-artifact@v6
with:
name: validated-contracts-${{ github.event.workflow_run.head_commit.id }}
run_id: ${{ github.event.workflow_run.id }}
- name: Generate matrix from deployedContracts.json
id: generate-matrix
run: |
set -e
data=$(cat script/deployedContracts.json)
bootstrap=$(echo "$data" | jq -r '.clientChain.bootstrapLogic // empty')
clientGateway=$(echo "$data" | jq -r '.clientChain.clientGatewayLogic // empty')
vault=$(echo "$data" | jq -r '.clientChain.vaultImplementation // empty')
rewardVault=$(echo "$data" | jq -r '.clientChain.rewardVaultImplementation // empty')
capsule=$(echo "$data" | jq -r '.clientChain.capsuleImplementation // empty')
# Read the JSON file
data=$(cat validatedContracts.json)
# Create the matrix as a JSON array
matrix=$(jq -n \
--arg bootstrap "$bootstrap" \
--arg clientGateway "$clientGateway" \
--arg vault "$vault" \
--arg rewardVault "$rewardVault" \
--arg capsule "$capsule" \
'[{name: "Bootstrap", address: $bootstrap},
{name: "ClientChainGateway", address: $clientGateway},
{name: "Vault", address: $vault},
{name: "RewardVault", address: $rewardVault},
{name: "ExoCapsule", address: $capsule}] | map(select(.address != ""))')
# Generate the matrix dynamically from the JSON content
matrix=$(echo "$data" | jq -c 'to_entries | map({name: .key, address: .value})')
echo "Matrix: $matrix"
echo "matrix=$(echo "$matrix" | jq -c .)" >> "${GITHUB_OUTPUT}"
Expand Down Expand Up @@ -217,12 +205,15 @@ jobs:
uses: actions/checkout@v4
- name: Restore the compiled layout files from the artifact
if: ${{ github.event.workflow_run.conclusion == 'success' }}
# Use this workflow if the artifact was generated by another workflow.
uses: dawidd6/action-download-artifact@v6
with:
name: compiled-layouts-${{ github.event.workflow_run.head_commit.id }}
run_id: ${{ github.event.workflow_run.id }}
- name: Restore the deployed layout files from the artifact
if: ${{ github.event.workflow_run.conclusion == 'success' }}
# Use this workflow if the artifact was generated by the same workflow.
# It is faster than the other one and conversely, a bit limited in skill.
uses: actions/download-artifact@v4
with:
name: deployed-layouts-${{ github.event.workflow_run.head_commit.id }}
Expand Down
138 changes: 90 additions & 48 deletions .github/workflows/forge-ci.yml
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
name: Forge CI


on:
merge_group:
pull_request:
Expand All @@ -10,6 +11,7 @@ on:
tags:
- "*"


jobs:
setup:
# A full job can be used as a reusable workflow but not a step.
Expand Down Expand Up @@ -46,7 +48,7 @@ jobs:
submodules: recursive
- name: Build
run: forge build
- name: Cache build artifacts
- name: Cache `forge build` results
uses: actions/cache/save@v3
with:
path: |
Expand All @@ -71,7 +73,7 @@ jobs:
run: echo "${{ needs.build.outputs.installation-dir }}" >> "$GITHUB_PATH"
- name: Checkout repository
uses: actions/checkout@v4
- name: Restore build artifacts
- name: Restore `forge build` results
uses: actions/cache/restore@v3
with:
path: |
Expand Down Expand Up @@ -107,7 +109,7 @@ jobs:
run: echo "${{ needs.build.outputs.installation-dir }}" >> "$GITHUB_PATH"
- name: Checkout repository
uses: actions/checkout@v4
- name: Restore build artifacts
- name: Restore `forge build` results
uses: actions/cache/restore@v3
with:
path: |
Expand All @@ -120,8 +122,8 @@ jobs:
run: forge fmt --check

check-contract-deployments:
# Takes less than 30s
timeout-minutes: 5
# Takes less than 60s
timeout-minutes: 10
runs-on: ubuntu-latest
needs: build
steps:
Expand All @@ -134,7 +136,7 @@ jobs:
run: echo "${{ needs.build.outputs.installation-dir }}" >> "$GITHUB_PATH"
- name: Checkout repository
uses: actions/checkout@v4
- name: Validate deployedContracts.json
- name: Validate deployedContracts.json and prepare artifact
run: |
data=$(cat script/deployedContracts.json)
Expand Down Expand Up @@ -172,49 +174,66 @@ jobs:
echo "Validating capsule address..."
validate_address "$capsule"
echo "Validation passed: All fields are non-empty and valid Ethereum checksum addresses"
# Prepare JSON for artifact. Instead of using the file within the repo,
# we create an artifact from the PR because the `compare-layouts` workflow
# runs on the base branch and doesn't have access to the PR's files.
# Given the presence of the artifact, technically, the above validation
# could be offloaded to the `compare-layouts` workflow, but this is a
# basic short circuit to avoid running the `compare-layouts` workflow
# if the PR has invalid addresses. Other validations to include, if
# possible, would be the verification of the contract on Etherscan;
# however, we cannot do that in the PR context without leaking the
# Etherscan API key.
extract-base-storage-layout-exocore-gateway:
# Takes less than 30 seconds, but add some margin for git clone
timeout-minutes: 10
runs-on: ubuntu-latest
needs: build
steps:
- name: Restore cached Foundry toolchain
uses: actions/cache/restore@v3
with:
path: ${{ needs.build.outputs.installation-dir }}
key: ${{ needs.build.outputs.cache-key }}
- name: Add Foundry to PATH
run: echo "${{ needs.build.outputs.installation-dir }}" >> "$GITHUB_PATH"
- name: Checkout base branch or previous commit
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.base.ref || github.event.before }}
# We don't have a `lib` folder to restore for this step, so we
# recursively checkout the submodules. In other steps, we use the
# `lib` folder from the `build` job.
submodules: recursive
- name: Generate base branch layout file
# Note that this `run` will do a `forge build` so we don't need to do it ourselves.
# The build artifacts of this step are not relevant to us either, so we don't need to
# cache them.
run: |
forge inspect --json src/core/ExocoreGateway.sol:ExocoreGateway storage-layout > ExocoreGateway.base.json
- name: Upload storage layout file as an artifact
# Note that the keys of the json dict below are case sensitive and
# must match the keys `x.deployed.json` defined in `compareLayouts.js`
# exactly.
jq -n \
--arg bootstrap "$bootstrap" \
--arg clientGateway "$clientGateway" \
--arg vault "$vault" \
--arg rewardVault "$rewardVault" \
--arg capsule "$capsule" \
'{
Bootstrap: $bootstrap,
ClientChainGateway: $clientGateway,
Vault: $vault,
RewardVault: $rewardVault,
ExoCapsule: $capsule
}' > validatedContracts.json
echo "Validation passed: All fields are non-empty and valid Ethereum checksum addresses"
- name: Upload validated contracts artifact
uses: actions/upload-artifact@v4
with:
path: ExocoreGateway.base.json
name: compiled-layout-ExocoreGateway-base-${{ github.event.pull_request.base.sha || github.event.after || github.sha }}
name: validated-contracts-${{ github.event.pull_request.head.sha || github.event.after || github.sha }}
path: validatedContracts.json

extract-storage-layout:
# Takes less than 30 seconds
# Takes less than 30 seconds per matrix member
timeout-minutes: 5
runs-on: ubuntu-latest
needs: build
outputs:
storage-layout-file: ${{ steps.generate-storage-layout.outputs.output-file }}
artifact-name: ${{ steps.generate-storage-layout.outputs.artifact-name }}
strategy:
matrix:
contract: [Bootstrap, ClientChainGateway, RewardVault, Vault, ExocoreGateway, ExoCapsule]
include:
- contract: ExocoreGateway
base: true
- contract: Bootstrap
base: false
- contract: ClientChainGateway
base: false
- contract: RewardVault
base: false
- contract: Vault
base: false
- contract: ExocoreGateway
base: false
- contract: ExoCapsule
base: false
steps:
- name: Restore cached Foundry toolchain
uses: actions/cache/restore@v3
Expand All @@ -225,40 +244,63 @@ jobs:
run: echo "${{ needs.build.outputs.installation-dir }}" >> "$GITHUB_PATH"
- name: Checkout repository
uses: actions/checkout@v4
- name: Restore build artifacts
if: ${{ !matrix.base }}
- name: Checkout base branch of repository
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.base.ref || github.event.before }}
submodules: recursive
if: ${{ matrix.base }}
- name: Restore `forge build` results
uses: actions/cache/restore@v3
# The restoration can only happen for the non-base case. For the base case,
# the context is different and anything created within the PR context will
# not be available to the cache restore action.
if: ${{ !matrix.base }}
with:
path: |
./lib
./out
./cache
./broadcast
key: build-${{ github.event.pull_request.head.sha || github.event.after || github.sha }}
- name: Generate storage layout file for ${{ matrix.contract }}
- name: Generate storage layout file
id: generate-storage-layout
run: |
forge inspect --json src/core/${{ matrix.contract }}.sol:${{ matrix.contract }} storage-layout > ${{ matrix.contract }}.compiled.json;
set -e
artifact_name="compiled-layout-${{ matrix.contract }}"
if [ "${{ matrix.base }}" = "true" ]; then
output_file="${{ matrix.contract }}.base.json"
artifact_name="${artifact_name}-base"
else
output_file="${{ matrix.contract }}.compiled.json"
artifact_name="${artifact_name}-proposed"
fi
artifact_name="${artifact_name}-${{ github.event.pull_request.head.sha || github.event.after || github.sha }}"
forge inspect --json src/core/${{ matrix.contract }}.sol:${{ matrix.contract }} storage-layout > $output_file
echo "output-file=$output_file" >> "$GITHUB_OUTPUT"
echo "artifact-name=$artifact_name" >> "$GITHUB_OUTPUT"
- name: Upload storage layout file as an artifact
uses: actions/upload-artifact@v4
with:
path: ${{ matrix.contract }}.compiled.json
name: compiled-layout-${{ matrix.contract}}-${{ github.event.pull_request.head.sha || github.event.after || github.sha }}
path: ${{ steps.generate-storage-layout.outputs.output-file }}
name: ${{ steps.generate-storage-layout.outputs.artifact-name }}

combine-storage-layouts:
# Takes less than 10 seconds
timeout-minutes: 5
runs-on: ubuntu-latest
needs:
- extract-base-storage-layout-exocore-gateway
- extract-storage-layout
steps:
- name: Download artifacts
uses: actions/download-artifact@v4
# No name means all artifacts are downloaded within their respective subfolders
# inside the provided path.
# No name means all artifacts created by this workflow are downloaded
# within their respective subfolders (paths) inside the provided path (`combined`).
with:
path: combined
- name: Zip up the compiled layouts
run: zip -j compiled-layouts.zip combined/*/*.json
run: find combined -type f -name "*.json" ! -name "validatedContracts.json" -exec zip -j compiled-layouts.zip {} +
- name: Upload the compiled layouts file as an artifact
uses: actions/upload-artifact@v4
with:
Expand Down

0 comments on commit 02e51e3

Please sign in to comment.