Skip to content

ci: build speed improvements #76

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 89 commits into from
Apr 22, 2025
Merged

ci: build speed improvements #76

merged 89 commits into from
Apr 22, 2025

Conversation

williscool
Copy link
Owner

@williscool williscool commented Apr 15, 2025

BOTTOM LINE: Cut the build time in half!

about 15 mins down from 30.

main features?

  1. ccache externalNativeBuild(Debug|Release) was very hard to figure how and where to do this. see the story below. long story short checkout scripts/ccachify_native_modules.sh you have to update the android/CMakeLists.txt file in EVERY SINGLE INDIVIDUAL node_modules folder
  2. run instrument tests directly instead of doing connectedAndroidTest which skips over cAT rebuilding the app again for no reason. that alone is like 5 mins off the build

started off trying to figure out how to keep externalNativeBuildDebug from rebuilding on every build

Boy was that a can of worms.

It turns out this is effectively impossible because of the build fingerprinting that is done as a part of the build being the native modules.

I tried everything.

Zips
Tars
Rsync

The only thing that works is rsync -a because it does byte for byte copies. But there's no archive format that will work. They're ever so slight differences which you can check between doing a tar with the options to not do anything any compression and rsync -a

just diff -r untared_dir rsync _a and you'll see ever so slight differences between the .cxx and android/app/build/intermediates folders.

So it just can't be done. Unless you have some place that you can like Mount a disc for bite storage. Which might work but it's probably overkill.

0 so I ended up going with ccache now the builds still happen over and over again but they are still just as fast as if they didn't only off by a couple seconds.

And I almost forgot I was only looking in our project's main Android folder for setting up doing anything related to native modules. The first first archive thing I was describing earlier and then earlier ccache as well.

Well, it turns out any react native modules that you depend on that get auto linked are going to have their own Android folders inside of the node modules folder.

You have to add the ccache calls to the files in there and that's the script in this PR.


again

https://reactnative.dev/docs/build-speed

TODO: figured out all the shit we need to caches is in the node_modules android/.cxx folders
+
+# I feel like the bigggest idiot for not figuring taht out faster

Copy link

Build artifacts for this PR are available:

You can download these artifacts from the "Artifacts" section of the workflow run.

Copy link

Build artifacts for this PR are available:

You can download these artifacts from the "Artifacts" section of the workflow run.

Copy link

github-actions bot commented Apr 15, 2025

Code Coverage Report

Overall Project 25.06%

There is no coverage information present for the Files changed

Copy link

📊 Code Coverage Summary

Type Coverage
Line 13.09
Branch

View detailed coverage report

Copy link

Build artifacts for this PR are available:

You can download these artifacts from the "Artifacts" section of the workflow run.

Copy link

📊 Code Coverage Summary

Type Coverage
Line 13.09
Branch

View detailed coverage report

Copy link

Build artifacts for this PR are available:

You can download these artifacts from the "Artifacts" section of the workflow run.

Copy link

📊 Code Coverage Summary

Type Coverage
Line 13.17
Branch

View detailed coverage report

Copy link

📊 Code Coverage Summary

Type Coverage
Line 25.19
Branch

View detailed coverage report

Copy link

Build artifacts for this PR are available:

You can download these artifacts from the "Artifacts" section of the workflow run.

Copy link

📊 Code Coverage Summary

Type Coverage
Line 13.1
Branch

View detailed coverage report

Copy link

📊 Code Coverage Summary

Type Coverage
Line 1.46
Branch

View detailed coverage report

Copy link

Build artifacts for this PR are available:

You can download these artifacts from the "Artifacts" section of the workflow run.

Copy link

📊 Code Coverage Summary

Type Coverage
Line 13.84
Branch

View detailed coverage report

Copy link

📊 Code Coverage Summary

Type Coverage
Line 1.46
Branch

View detailed coverage report

Copy link

📊 Code Coverage Summary

Type Coverage
Line 1.46
Branch

View detailed coverage report

Copy link

📊 Code Coverage Summary

Type Coverage
Line 1.46
Branch

View detailed coverage report

Copy link

📊 Code Coverage Summary

Type Coverage
Line 1.46
Branch

View detailed coverage report

Copy link

Build artifacts for this PR are available:

You can download these artifacts from the "Artifacts" section of the workflow run.

Copy link

📊 Code Coverage Summary

Type Coverage
Line 13.69
Branch

View detailed coverage report

Copy link

Build artifacts for this PR are available:

You can download these artifacts from the "Artifacts" section of the workflow run.

Copy link

📊 Code Coverage Summary

Type Coverage
Line 13.19
Branch

View detailed coverage report

also add timestampst outputs
Copy link

Build artifacts for this PR are available:

You can download these artifacts from the "Artifacts" section of the workflow run.

Copy link

📊 Code Coverage Summary

Type Coverage
Line 25.46
Branch

View detailed coverage report

Copy link

Build artifacts for this PR are available:

You can download these artifacts from the "Artifacts" section of the workflow run.

Copy link

📊 Code Coverage Summary

Type Coverage
Line 27.24
Branch

View detailed coverage report

Copy link

Build artifacts for this PR are available:

You can download these artifacts from the "Artifacts" section of the workflow run.

@williscool
Copy link
Owner Author

williscool commented Apr 22, 2025

fast follow fixes

  1. use the dorny reporter conclusion thing to fail (our running script only runs we don't want it to fail the build because then we might not get coverage. which we always want)
  2. there should be 3 caches. the instrument tests should have a different cache from the unit tests and from build. they all do different things now and there should be less overlap. that should cut down load times on unit and instrument tests also
  3. fix the android cache. and other caches so we dont have misses anymore

@williscool williscool enabled auto-merge (squash) April 22, 2025 01:26
Copy link

📊 Code Coverage Summary

Type Coverage
Line 25.43
Branch

View detailed coverage report

@williscool williscool force-pushed the ci/try_fix_slowness branch from f321828 to d168acd Compare April 22, 2025 01:51
Copy link

Build artifacts for this PR are available:

You can download these artifacts from the "Artifacts" section of the workflow run.

Copy link

📊 Code Coverage Summary

Type Coverage
Line 25.06
Branch

View detailed coverage report

1 similar comment
Copy link

📊 Code Coverage Summary

Type Coverage
Line 25.06
Branch

View detailed coverage report

we dont need them anymore and some bad outputs in them can break the
test reporter

all we need are the apks
Copy link

Build artifacts for this PR are available:

You can download these artifacts from the "Artifacts" section of the workflow run.

@williscool williscool merged commit 22546b4 into master Apr 22, 2025
9 checks passed
Copy link

📊 Code Coverage Summary

Type Coverage
Line
Branch

View detailed coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant