Skip to content

ci: build speed improvements #76

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 89 commits into from
Apr 22, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
48c23ed
ci: try to skip externalNativeBuildDebug
williscool Apr 15, 2025
3325cf0
ci: try to skip native build altogether
williscool Apr 15, 2025
bc66d88
ci: try to skip task altogther
williscool Apr 15, 2025
d9f8b1e
ci: another skip try also have gradle show task graph to debug
williscool Apr 15, 2025
ad1aadf
ci: task graph debug
williscool Apr 17, 2025
0501da0
fix: try harder to skip unneeded building
williscool Apr 17, 2025
57cf9bd
fix: maybe packageList
williscool Apr 18, 2025
4d85bbc
ci: maybe skip build stuff we want?
williscool Apr 18, 2025
50cabae
ci: build rearrage
williscool Apr 18, 2025
9e594cd
ci: stop build if no native artifacts
williscool Apr 18, 2025
fe53cd1
ci: centralize checking for artifacts
williscool Apr 18, 2025
41263ef
ci: dont need emulator for main build
williscool Apr 18, 2025
eefe111
ci: hopefully finally actually skip the shit
williscool Apr 18, 2025
ee3c43b
fix: maybe shell stuff ?
williscool Apr 18, 2025
3c28cc4
ci: time for tmate
williscool Apr 18, 2025
434a067
ci: im not givin up
williscool Apr 18, 2025
0b0bcf5
ci: remove old cache update step
williscool Apr 18, 2025
5b8715b
ci: givin it another shot
williscool Apr 18, 2025
14bb6e2
ci: throw away stuff I know doesn't work
williscool Apr 18, 2025
43317b6
ci: ccache and configuration cache
williscool Apr 18, 2025
ad9fda8
ci: use ccache
williscool Apr 18, 2025
13b376e
ci: fix
williscool Apr 18, 2025
7dbc684
fix: get rid of more stuff we know does nothing
williscool Apr 18, 2025
e13a04d
ci: might as well try to switch to instrument tests
williscool Apr 18, 2025
2abbe4a
fix: gotta install the apks first!
williscool Apr 18, 2025
f84e817
ci: maybe debug ccache
williscool Apr 18, 2025
05f98a6
ci: maybe only build abis we want
williscool Apr 18, 2025
c11bc23
ci: new cache we need
williscool Apr 18, 2025
3a5356c
ci: git rid of a bunch of shit we don't need anymore
williscool Apr 18, 2025
d7dce11
ci: more steps unneeded
williscool Apr 18, 2025
24c6947
fix: ccache
williscool Apr 18, 2025
9418125
fix: fix
williscool Apr 19, 2025
d24383a
feat: github codespace setup
williscool Apr 19, 2025
3001dd6
fix: maybe devcontainer
williscool Apr 19, 2025
b5fc02b
fix: last bit of codespace setup
williscool Apr 19, 2025
0d379d1
feat: android setup
williscool Apr 19, 2025
c91aacb
feat: mcfly_history
williscool Apr 20, 2025
08142c0
fix: ccachify_native_modules script
williscool Apr 20, 2025
30c520a
fix: ccachify all the things and remove other stuff
williscool Apr 20, 2025
d0c9b09
fix: scrap seperate native build task altogether
williscool Apr 20, 2025
6068de1
fix: remove native build seperate file too
williscool Apr 20, 2025
dd6d427
feat: bunch more goodies for codespace
williscool Apr 20, 2025
e281371
feat: version of test runner that works!
williscool Apr 20, 2025
75effe3
fix: actions
williscool Apr 20, 2025
05144a8
feat: tests and coverage report from script!
williscool Apr 20, 2025
2e2193e
feat: split test running and code coverage
williscool Apr 20, 2025
73760f5
fix: simplify coverage puller
williscool Apr 20, 2025
de1bedd
fix: cleanup
williscool Apr 20, 2025
264f5c9
ci: action for new run way
williscool Apr 20, 2025
de3c7d0
ci: latest mcfly_history
williscool Apr 20, 2025
081e482
fix: kvm setup
williscool Apr 20, 2025
0b5b606
fix: download apks for tests
williscool Apr 20, 2025
eb9def9
ci: bring back configuration cache
williscool Apr 20, 2025
70734b9
ci: make cache writeable for this pr
williscool Apr 20, 2025
f9a3947
ci: maybe put apks in right place for integration tests
williscool Apr 20, 2025
ef9e10a
ci: switch to java 21 in ci
williscool Apr 20, 2025
fa9b416
fix: drop bundle logic from build.gradle
williscool Apr 20, 2025
b8c4ae0
feat: let go of configuration cache again for ci
williscool Apr 20, 2025
feb028e
ci: coverage generator needs emulator connection too
williscool Apr 20, 2025
8928306
ci: another shot at coverage pulling
williscool Apr 20, 2025
d272257
ci: addressing comments
williscool Apr 20, 2025
706914a
ci: another try
williscool Apr 20, 2025
5e73c12
fix: fat finger
williscool Apr 20, 2025
d5b6c77
fix: test run action
williscool Apr 21, 2025
7072139
fix: maybe caches
williscool Apr 21, 2025
c708d36
fix: maybe cache more
williscool Apr 21, 2025
9c46630
fix: more cache cleanup
williscool Apr 21, 2025
4a7da99
fix: cache vs arch
williscool Apr 21, 2025
d046256
fix: maybe xml reporter
williscool Apr 21, 2025
29a9ca3
fix: mostly right coverage update
williscool Apr 21, 2025
3ab69fa
fix: coverage
williscool Apr 21, 2025
3754f59
fix: Improve caching seperate saving and restoring
williscool Apr 21, 2025
0d9e994
ci: log export
williscool Apr 21, 2025
d940a35
fix: report all formats!
williscool Apr 21, 2025
5131b22
fix: android cache dir
williscool Apr 21, 2025
440482e
ci: bring back old way of running tests for now
williscool Apr 21, 2025
af910c7
docs: todos
williscool Apr 21, 2025
ad80475
ci: maybe xml reporter
williscool Apr 21, 2025
619296c
fix: just include the XmlRunListener code in here
williscool Apr 21, 2025
5e281c1
fix: maybe xml reporter
williscool Apr 21, 2025
5870a14
fix: get rid of bullshit results
williscool Apr 21, 2025
16b4454
fix: get report from the right place
williscool Apr 21, 2025
98e0a25
test: seperate avd for old integration test
williscool Apr 21, 2025
2c06d76
fix: maybe dorny/test-reporte
williscool Apr 21, 2025
19afbfb
ci: just get all xml in outputs
williscool Apr 21, 2025
45a93fc
fix: latest mcfly_history and codespace setup
williscool Apr 22, 2025
e63573e
fix: dorny reporter!
williscool Apr 22, 2025
d168acd
fix: no need to copy report in 2 places
williscool Apr 22, 2025
278deb7
ci: skip build outputs caches for integration tests
williscool Apr 22, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
// For format details, see https://aka.ms/devcontainer.json. For config options, see the
// README at: https://github.com/devcontainers/templates/tree/main/src/universal
{
"name": "Default Linux Universal",
// Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
"image": "mcr.microsoft.com/devcontainers/universal:2-linux",
"features": {
"ghcr.io/meaningful-ooo/devcontainer-features/homebrew:2": {"shallowClone": true}
},

// Features to add to the dev container. More info: https://containers.dev/features.
// "features": {},

// Use 'forwardPorts' to make a list of ports inside the container available locally.
// "forwardPorts": [],

// Use 'postCreateCommand' to run commands after the container is created.
"postCreateCommand": "bash -i ./scripts/setup_github_codespace.sh",

// Configure tool-specific properties.
// "customizations": {},

// Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
// "remoteUser": "root"
}
Binary file added .devcontainer/mcfly_history.db
Binary file not shown.
121 changes: 55 additions & 66 deletions .github/actions/cache-update/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@ name: 'Cache Update'
description: 'Updates caches for Android builds'

inputs:
optional_cache_key:
description: 'Optional cache key'
required: false
default: ""
arch:
description: 'Architecture (arm64-v8a, x86_64)'
required: false
Expand All @@ -10,76 +14,25 @@ inputs:
description: 'Whether to run emulator setup steps'
required: false
default: "false"
native_build_hash:
description: 'Hash of the external native tasks file'
required: false
default: "nohash"
already_uploaded_artifacts:
description: 'Whether native build artifacts were already uploaded'
required: false
default: "false"

runs:
using: "composite"
steps:
# Update JS Bundle and Assets Cache
- name: Update JS Bundle and Assets Cache
# JS Bundle and Assets Cache - Save only
- name: Save JS Bundle and Assets Cache
if: always()
uses: actions/cache@v4
uses: actions/cache/save@v4
with:
path: |
android/app/src/main/assets/index.android.bundle
android/app/src/main/res/drawable-*
android/app/src/main/res/raw-*
key: ${{ runner.os }}-js-bundle-${{ hashFiles('package.json', 'yarn.lock') }}-${{ hashFiles('src/**/*.{js,jsx,ts,tsx}', 'lib/**/*.{js,jsx,ts,tsx}', 'modules/**/*.{js,jsx,ts,tsx}', 'App.{js,jsx,ts,tsx}', 'index.{js,jsx,ts,tsx}') }}

# Check if native build directory exists
- name: Check Native Build Directory
id: check-native-build
if: always() && inputs.arch != '' && inputs.already_uploaded_artifacts != 'true'
shell: bash
run: |
if [ -d "android/app/build/intermediates/cmake" ] && [ "$(find android/app/build/intermediates/cmake -type f | wc -l)" -gt 0 ]; then
echo "exists=true" >> $GITHUB_OUTPUT
echo "Native build directory exists and has files."
else
echo "exists=false" >> $GITHUB_OUTPUT
echo "Native build directory does not exist or is empty."
fi

# Upload Native Build Artifacts (only if not already uploaded)
- name: Upload Native Build Artifacts
if: always() && inputs.arch != '' && inputs.already_uploaded_artifacts != 'true' && steps.check-native-build.outputs.exists == 'true'
uses: actions/upload-artifact@v4
with:
name: native-build-cmake-${{ inputs.arch }}-${{ inputs.native_build_hash }}
path: android/app/build/intermediates/cmake
retention-days: 7
if-no-files-found: warn

# Update NDK and CMake Cache
- name: Update NDK and CMake Cache
if: always() && inputs.arch != ''
uses: actions/cache@v4
with:
path: |
.cxx
~/.android/ndk
~/.android/cmake
android/app/.cxx
android/app/build/intermediates/cmake
android/app/build/intermediates/merged_native_libs
android/app/build/intermediates/stripped_native_libs
android/app/build/intermediates/transforms
android/app/build/intermediates/prefab
android/app/build/intermediates/prefab_package
android/app/build/intermediates/prefab_metadata
key: ${{ runner.os }}-cmake-${{ inputs.arch }}-${{ hashFiles('android/app/src/main/jniLibs/**/*.so') }}-${{ hashFiles('android/external-native-tasks.txt') }}

# Update Build Outputs Cache
- name: Update Build Outputs Cache
if: always() && inputs.arch != ''
uses: actions/cache@v4
# Build Outputs Cache - Save only
- name: Save Build Outputs Cache
if: always() && inputs.arch != '' && inputs.optional_cache_key != 'integration-test'
uses: actions/cache/save@v4
with:
path: |
android/app/build/intermediates
Expand All @@ -90,15 +43,51 @@ runs:
android/app/build/intermediates/test_apk
android/app/build/intermediates/test_apk_androidTest
key: ${{ runner.os }}-build-outputs-${{ inputs.arch }}-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}-${{ github.sha }}

# Gradle cache - Save only with architecture in key
- name: Save Gradle Cache
if: always()
uses: actions/cache/save@v4
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
android/.gradle
android/build-cache
key: ${{ runner.os }}-gradle-${{ inputs.arch != '' && format('-{0}', inputs.arch) || '' }}-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}

# Update Android SDK and AVD cache
- name: Update Android SDK and AVD Cache
# Yarn cache - Save only
- name: Save yarn cache
if: always()
uses: actions/cache/save@v4
with:
path: $(yarn config get cacheFolder)
key: yarn-download-cache-${{ hashFiles('yarn.lock') }}

# Yarn install state - Save only
- name: Save yarn install state
if: always()
uses: actions/cache/save@v4
with:
path: .yarn/ci-cache/
key: ${{ runner.os }}-yarn-install-state-cache-${{ hashFiles('yarn.lock', '.yarnrc.yml') }}

# React Native cache - Save only
- name: Save React Native cache
if: always()
uses: actions/cache/save@v4
with:
path: |
node_modules/.cache/react-native
node_modules/.cache/metro
node_modules/.cache/@react-native-community/cli
key: ${{ runner.os }}-react-native-${{ hashFiles('package.json', 'yarn.lock') }}

# Android SDK and AVD cache - Save only
- name: Save Android SDK and AVD Cache
if: always() && inputs.run_emulator_setup == 'true'
uses: actions/cache@v4
uses: actions/cache/save@v4
with:
path: |
~/.android/avd/*
~/.android/adb*
~/.android/sdk/*
~/.android/repositories.cfg
key: ${{ runner.os }}-android-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
~/.android/*
key: ${{ runner.os }}-android-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
Loading
Loading