Skip to content

Handle building arbitrarily large packages #1194

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jrray opened this issue Mar 19, 2025 · 0 comments
Open

Handle building arbitrarily large packages #1194

jrray opened this issue Mar 19, 2025 · 0 comments
Labels
bug Something isn't working

Comments

@jrray
Copy link
Collaborator

jrray commented Mar 19, 2025

When building packages, the writable upperdir backing /spfs is a tmpfs which is limited by the size of the physical RAM of the hardware. This puts a ceiling on the size of packages that can be created.

Side note: users are often confused by the out of disk space error when package builds fail in CI and will open tickets about CI machines being out of disk space when that's not really the case. The actual fix is to find a machine with more RAM, which at some point becomes impossible.

What is needed is a way to opt in for builds to use a given directory on disk to use as the upperdir instead of using a tmpfs directory. The workflow proposed is:

  1. create a new subdirectory under the given path using a uuid/ulid-generated name.
    1. this is to ensure that we start with an empty directory that we own and are free to delete later.
  2. create the "upperdir" and "workdir" directories as subdirectories of 1.
  3. create the spfs runtime using the directories of 2 instead of the normal tmpfs directories.
  4. register 1 with the spfs runtime meta payload in some way.
  5. put the cleanup responsibility onto spfs-monitor which will know about 1 from 4 and make its best effort to delete 1 when cleaning up the runtime.
    1. this parallels the act of unmounting the tmpfs at runtime cleanup

This workflow overlaps a lot with the existing --keep-runtime feature but brings the ability to have more control over where the data will go (instead of it going inside the local spfs repo) and introduces the cleanup concept. Where --keep-runtime is meant for preserving hand-created content under /spfs, this new requirement is for having a larger scratch space for builds but the files are expected to be discarded at the end of the build. It also isn't clear if --keep-runtime works for builds, and spk build is multiple steps that create multiple runtimes.

@jrray jrray added the bug Something isn't working label Mar 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant