"too many open files" sometimes when installing a formula on Linux (same as #9120)

[durka]

brew doctor output

Your system is ready to brew.

Verification

• My "brew doctor output" above says Your system is ready to brew. and am still able to reproduce my issue.
• I ran brew update twice and am still able to reproduce my issue.
• This issue's title and/or description do not reference a single formula e.g. brew install wget. If they do, open an issue at https://github.com/Homebrew/homebrew-core/issues/new/choose instead.

brew config output

HOMEBREW_VERSION: 4.4.19
ORIGIN: https://github.com/Homebrew/brew
HEAD: 23facb219df8615680636ef12103bb3313f44e2a
Last commit: 3 days ago
Branch: stable
Core tap JSON: 30 Jan 21:38 UTC
HOMEBREW_PREFIX: /home/linuxbrew/.linuxbrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_DISPLAY: :1
HOMEBREW_EDITOR: vim
HOMEBREW_MAKE_JOBS: 16
Homebrew Ruby: 3.3.7 => /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/3.3.7/bin/ruby
CPU: 16-core 64-bit skylake
Clang: 19.1.7
Git: 2.43.0 => /bin/git
Curl: 8.5.0 => /bin/curl
Kernel: Linux 6.8.0-52-generic x86_64 GNU/Linux
OS: Ubuntu 24.04.1 LTS (noble)
Host glibc: 2.39
/usr/bin/gcc: 13.3.0
/usr/bin/ruby: 3.2.3
glibc: N/A
gcc@11: N/A
gcc: N/A
xorg: N/A

What were you trying to do (and why)?

I was trying to install the qemu formula.

What happened (include all command output)?

Homebrew downloaded quite a few bottles, and started installing them, but failed along the way with "too many open files".

Now, it says all over this page that my issue will be immediately closed if I can't reproduce it, so maybe you're going to close this, since the issue usually resolves by re-running the same brew install command. Sometimes it gets further, then still fails, and a third try is necessary.

Note that is a duplicate of #9120, but I can't reopen that. It was closed saying only 1 person has ever complained, so let me add my voice as a complainant. This doesn't happen every time, but it does happen frequently.

Following a comment there, I can see that my ulimit is low:

$ ulimit -Hn
1048576
$ ulimit -Sn
1024

This is a standard install of Ubuntu 24.04, nothing spooky.

What did you expect to happen?

The install process should have completed.

Step-by-step reproduction instructions (by running brew commands)

1. `brew install qemu`

[MikeMcQuaid]

Thanks for reopening. #19167 makes me think this may not be Linux or macOS-specific.

This is a standard install of Ubuntu 24.04, nothing spooky.

Just to confirm: you haven't personally changed the ulimit here as far as you're aware?

[durka]

I haven't... it seems like Ubuntu has very low defaults for some reason

…

On Fri, Jan 31, 2025, 9:42 AM Mike McQuaid ***@***.***> wrote: Thanks for reopening. #19167 <#19167> makes me think this may not be Linux or macOS-specific. This is a standard install of Ubuntu 24.04, nothing spooky. Just to confirm: you haven't personally changed the ulimit here as far as you're aware? — Reply to this email directly, view it on GitHub <#19177 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAALPHZ3WJJP2EKB2ULHIZT2NODU5AVCNFSM6AAAAABWGHZPZWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMRXGUYTKMZZGE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[cho-m]

We run CI on Ubuntu 22.04 which should have similar limits.

macOS default is even lower at 256. macOS users usually hit the lock file error message early on rather than failing elsewhere:

brew/Library/Homebrew/lock_file.rb

Lines 31 to 33 in 4372fcf

	rescue Errno::EMFILE
	odie "The maximum number of open files on this system has been reached. " \
	"Use `ulimit -n` to increase this limit."

---

From glance, 1024 per process seems enough for most formulae.

In case of qemu, it has ~164 dependencies, each of which would create 1 lock file. Some extra files will be opened for things like loading Ruby shared libraries and mv/cp commands but shouldn't account for 800 more.

The previous issue analysis would be insufficient here. Even if you double the number for bottle files, it won't be near limit.

---

This is going to be difficult to analyze without further information. If you can get the error output with --verbose --debug, it would help narrow down commands run at time of error.

Really need to figure out what is creating all the open file descriptors. If it is a leak, then looping on lsof -p <pid> should show the list of open files increasing as run continues.

[durka]

We run CI on Ubuntu 22.04 which should have similar limits.

I admit I haven't looked at the CI setup, but is it running using docker? For some reason, the ubuntu:22.04 docker image has a quite high ulimit -Sn:

$ docker run ubuntu:22.04 bash -c 'ulimit -Sn'
1073741816

...however, installing an actual Ubuntu 22.04 in Gnome Boxes shows 1024:

Here is a messy Dockerfile which reliably reproduces the issue: (just run docker build . and then go get a coffee)

FROM ubuntu:22.04

RUN apt update && \
    apt install -y curl git sudo build-essential

RUN useradd -rm -d /home/ubuntu -s /bin/bash -g root -G sudo -u 1001 ubuntu
RUN echo "ubuntu ALL=(ALL) NOPASSWD: ALL" >/etc/sudoers && \
    chmod 0440 /etc/sudoers
USER ubuntu
WORKDIR /home/ubuntu

RUN /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

RUN ulimit -Sn 1024 && \
    eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)" && \
    brew install --verbose --debug qemu

[Bo98]

Thanks, I see an issue relating to patchelf. The reason it doesn't trigger for everyone is because it depends on how often the Ruby GC is kicking in as Ruby is currently cleaning up loose FDs.