Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Packaging: fix build-depends on multiarch debian systems #1324

Merged
merged 1 commit into from
Aug 19, 2024

Conversation

thresheek
Copy link
Member

It's possible to have two versions of the same package installed on debian-based multiarch systems - e.g. i386 alongside amd64. This means that when getting the package status through dpkg-query we'd get a duplicated string:

% dpkg-query -f '$${db:Status-Status}' -W libssl-dev $installed$installed

% dpkg -l | grep libssl-dev
ii libssl-dev:amd64 3.0.11-1deb12u2 amd64 Secure Sockets Layer toolkit - development files
ii libssl-dev:i386 3.0.11-1
deb12u2 i386 Secure Sockets Layer toolkit - development files

The fix is to explicitely check for the main architecture and, in case for noarch (or rather all-arch in debian terms) packages, check for special :all architecture as well.

It's possible to have two versions of the same package installed on
debian-based multiarch systems - e.g. i386 alongside amd64.  This means
that when getting the package status through dpkg-query we'd get a
duplicated string:

% dpkg-query -f '$${db:Status-Status}' -W libssl-dev
$installed$installed

% dpkg -l | grep libssl-dev
ii  libssl-dev:amd64                 3.0.11-1~deb12u2               amd64        Secure Sockets Layer toolkit - development files
ii  libssl-dev:i386                  3.0.11-1~deb12u2               i386         Secure Sockets Layer toolkit - development files

The fix is to explicitely check for the main architecture and, in case
for noarch (or rather all-arch in debian terms) packages, check for
special :all architecture as well.
@ac000
Copy link
Member

ac000 commented Jun 17, 2024

That's true of Fedora et al as well...

@thresheek
Copy link
Member Author

Indeed, it seems the problem is similar on rpm-based:

% fgrep PRETTY /etc/os-release
PRETTY_NAME="Red Hat Enterprise Linux 9.4 (Plow)"
% rpm -qi --whatprovides openssl-devel | grep -E 'Name|Arch'
Name        : openssl-devel
Architecture: i686
Name        : openssl-devel
Architecture: x86_64

However it's not as easy to fix, since architectures are all over the place:

% linux32 rpm --eval '%_arch'
i386
% rpm -qi --provides openssl-devel.i686 | grep ^openssl-devel\(
openssl-devel(x86-32) = 1:3.0.7-27.el9
% linux32 uname -m
i686

And to make things even more complicated, 700ee28 fixed a real issue on Amazon Linux 2023 with curl-minimal providing curl, php8.2-devel providing php-devel, etc, and they don't really provide a CAPABILITY with the architecture bit there:

% rpm -q --provides curl-minimal
curl = 8.2.1-1.amzn2023.0.3
curl-minimal = 8.2.1-1.amzn2023.0.3
curl-minimal(x86-64) = 8.2.1-1.amzn2023.0.3

@ac000
Copy link
Member

ac000 commented Jun 17, 2024

However it's not as easy to fix, since architectures are all over the place:

% linux32 rpm --eval '%_arch'
i386
% rpm -qi --provides openssl-devel.i686 | grep ^openssl-devel\(
openssl-devel(x86-32) = 1:3.0.7-27.el9
% linux32 uname -m
i686

It's been many years since I last needed to run any 32bit stuff (the flash plugin...)

I think part of the issue is that while there is really only x86_64 in terms of AMD/Intel 64bit, there are several subarchs in the 32bit case, (i386, i486, i585 & i686).

From the setarch(8) man-page

The execution domains currently only affects the output of uname -m.
For example, on an AMD64 system, running setarch i386 program will
cause program to see i686 instead of x86_64 as the machine type.

My guess is that rpm --eval '%_arch' is showing the base architecture, i386 in the case for 32bit Intel.

But and as I say it's been a long time since I needed to use 32bit stuff, I think you'll generally only find i686 rpms these days for 32bit Intel... (in Fedora's case anyway...)

(And of course support for i386 was removed from Linux over a decade ago! ;-))

@thresheek
Copy link
Member Author

The problem is I'm a bit lost on how to detect arch and noarch packages, or rather, Provides which generally are not required to specify the architecture (or something similar to architecture).

E.g. if we use a following patch to query for foo(x86-64) when we have a build-dependency on foo:

diff --git a/pkg/rpm/Makefile b/pkg/rpm/Makefile
index 5d28fb2c..e02796de 100644
--- a/pkg/rpm/Makefile
+++ b/pkg/rpm/Makefile
@@ -176,7 +176,11 @@ check-build-depends-%:
        esac ; \
        not_installed= ; \
        for pkg in $${pkgs}; do \
-               rpm -qi --whatprovides $${pkg} >/dev/null 2>&1 ; \
+               rpmarch="$$(uname -m)" ; \
+               case "$${rpmarch}" in \
+                       x86_64) rpmarch="x86-64";; \
+               esac ; \
+               rpm -qi --whatprovides $${pkg}\($${rpmarch}\) >/dev/null 2>&1 ; \
                if [ $$? -ne 0 ]; then \
                        not_installed="$${not_installed} $${pkg}" ; \
                fi ; \

We will be able to detect e.g. openssl-devel.x86_64 vs openssl-devel.i686, but it will fail on noarch packages, like rpmlint. Also, it will fail on Amazon Linux 2023 with curl-minimal that does provide curl, but does not specify the architecture:

% fgrep -i pretty /etc/os-release
PRETTY_NAME="Amazon Linux 2023"
% rpm -q --provides rpmlint
config(rpmlint) = 1.11-19.amzn2023.0.2
rpmlint = 1.11-19.amzn2023.0.2
% rpm -q --provides curl-minimal
curl = 8.2.1-1.amzn2023.0.3
curl-minimal = 8.2.1-1.amzn2023.0.3
curl-minimal(x86-64) = 8.2.1-1.amzn2023.0.3

@ac000
Copy link
Member

ac000 commented Jun 19, 2024

Are you building 32bit packages?

With rpm at least, it defaults to 64bit on x86_64 (as opposed to say defaulting to 32bit on Sparc64).

If you're only building 64bit packages I don't think you really have anything to worry about, I mean whenever I've build rpms in the past I've never needed to worry about such things...

@thresheek
Copy link
Member Author

I'm not building 32 bit packages. The rpm situation described is not something we have on our packages builder machines (contrary to the debian-based ones), but it's still possible for someone who will use our packaging to install both architectures of openssl-devel, which will confuse our check.

@thresheek
Copy link
Member Author

Ping for a review

@callahad callahad self-requested a review July 1, 2024 15:47
@javorszky javorszky self-requested a review July 29, 2024 15:24
@thresheek
Copy link
Member Author

Ping for a review

Copy link
Member

@ac000 ac000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it does the job...

@callahad
Copy link
Collaborator

@thresheek Would you like us to merge, or would you prefer to merge yourself? If we don't hear otherwise, we'll toss it in next Monday.

@callahad callahad added this to the 1.33 milestone Aug 19, 2024
@thresheek thresheek merged commit 5d32e50 into nginx:master Aug 19, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants