Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Targeting ARMv6 produces an ARMv7 binary #792

Closed
reynoldsbd opened this issue Feb 18, 2019 · 7 comments
Closed

Targeting ARMv6 produces an ARMv7 binary #792

reynoldsbd opened this issue Feb 18, 2019 · 7 comments

Comments

@reynoldsbd
Copy link

As with #516 and #578, I am having trouble building an ARMv6 binary (for a Raspberry Pi Zero) when ring is involved. My application builds successfully, but the resulting binary seems to be for ARMv7 rather than ARMv6 (according to readelf -A).

Further details:

  • My Rust toolchain is stable-x86_64-unknown-linux-gnu (1.32), and my target is arm-unknown-linux-gnueabihf
  • I'm using the official (?) Raspberry Pi cross compiler
    • The bin/ directory is in my $PATH
    • C programs compiled by this toolchain work perfectly fine on the Pi Zero
    • I have linker = "arm-linux-gnueabihf-gcc" in my Cargo config
  • With this setup, Rust programs that do not depend on ring can be successfully cross compiled and run on a Pi Zero

According to find target/arm-unknown-linux-gnueabihf/deps -type f | xargs readelf -A, it appears that several - but not all - of ring's native object files are being built for ARMv7, and I think this is causing the final linking of my application to use ARMv7 instead of ARMv6.

@briansmith
Copy link
Owner

Please include some snippets from the build output that show what command line arguments are being passed to the C compiler when compiling C code and when assembling.

@reynoldsbd
Copy link
Author

reynoldsbd commented Feb 19, 2019

Below is trimmed-down output from ring's build. I cut out a few environment variables and a bunch of nearly-identical GCC invocations. Let me know and I'll edit the post with full output.

Based on this output and the results from readelf, I noticed a pattern. Although most objects are being compiled correctly to ARMv6, the following pregenerated assembly files are being compiled to ARMv7:

  • aesv8-armx-linux32.S
  • aes-armv4-linux32.S
  • bsaes-armv7-linux32.S
  • armv4-mont-linux32.S
  • chacha-armv4-linux32.S
  • ghash-armv4-linux32.S
  • sha256-armv4-linux32.S
  • sha512-armv4-linux32.S

It seems strange that GCC would produce ARMv7 objects even when -march=armv6 is specified. Perhaps it is ignoring this flag and choosing a suitable target based on the contents of the assembly file?

CARGO: /home/bobby/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/cargo
CARGO_CFG_DEBUG_ASSERTIONS: 
CARGO_CFG_PROC_MACRO: 
CARGO_CFG_TARGET_ARCH: arm
CARGO_CFG_TARGET_ENDIAN: little
CARGO_CFG_TARGET_ENV: gnu
CARGO_CFG_TARGET_FAMILY: unix
CARGO_CFG_TARGET_OS: linux
CARGO_CFG_TARGET_POINTER_WIDTH: 32
CARGO_CFG_UNIX: 
CARGO_FEATURE_DEFAULT: 1
CARGO_FEATURE_DEV_URANDOM_FALLBACK: 1
CARGO_FEATURE_USE_HEAP: 1
CARGO_HOME: /home/bobby/.cargo
CARGO_MAKEFLAGS: --jobserver-fds=5,15 -j --jobserver-auth=5,15 -j
CARGO_MANIFEST_DIR: /home/bobby/.cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.13.5
CARGO_MANIFEST_LINKS: ring-asm
CARGO_PKG_AUTHORS: Brian Smith <brian@briansmith.org>
CARGO_PKG_DESCRIPTION: Safe, fast, small crypto using Rust.
CARGO_PKG_HOMEPAGE: 
CARGO_PKG_NAME: ring
CARGO_PKG_REPOSITORY: https://github.com/briansmith/ring
CARGO_PKG_VERSION: 0.13.5
CARGO_PKG_VERSION_MAJOR: 0
CARGO_PKG_VERSION_MINOR: 13
CARGO_PKG_VERSION_PATCH: 5
CARGO_PKG_VERSION_PRE: 
DEBUG: true
DISPLAY: localhost:0
HOME: /home/bobby
HOST: x86_64-unknown-linux-gnu
HOSTTYPE: x86_64
LANG: en_US.UTF-8
LD_LIBRARY_PATH: /c/Users/reyno/Source/xc-pi-zero/target/debug/deps:/home/bobby/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib:/home/bobby/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib
LOGNAME: bobby
NUM_JOBS: 4
OPT_LEVEL: 0
OUT_DIR: /c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out
PATH: /home/bobby/Source/rpi-tools/arm-bcm2708/arm-linux-gnueabihf/bin:/home/bobby/.cargo/bin:/home/bobby/.local/opt/go/bin:/home/bobby/.local/opt/riscv64-unknown-elf-gcc-2018.07.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
PROFILE: debug
PWD: /home/bobby/Source/xc-pi-zero
RUSTC: rustc
RUSTC_LINKER: arm-linux-gnueabihf-gcc
RUSTDOC: rustdoc
RUSTUP_HOME: /home/bobby/.rustup
RUSTUP_TOOLCHAIN: stable-x86_64-unknown-linux-gnu
RUST_RECURSION_COUNT: 1
SHELL: /bin/bash
TARGET: arm-unknown-linux-gnueabihf
USER: bobby
_: /home/bobby/.cargo/bin/cargo
OPT_LEVEL = Some("0")
TARGET = Some("arm-unknown-linux-gnueabihf")
HOST = Some("x86_64-unknown-linux-gnu")
CC_arm-unknown-linux-gnueabihf = None
CC_arm_unknown_linux_gnueabihf = None
TARGET_CC = None
CC = None
CROSS_COMPILE = None
CFLAGS_arm-unknown-linux-gnueabihf = None
CFLAGS_arm_unknown_linux_gnueabihf = None
TARGET_CFLAGS = None
CFLAGS = None
DEBUG = Some("true")
running "arm-linux-gnueabihf-gcc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-march=armv6" "-marm" "-mfpu=vfp" "-I" "include" "-Wall" "-Wextra" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-declarations" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-Wno-cast-align" "-fstack-protector" "-g3" "-D_XOPEN_SOURCE=700" "-c" "-o/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/aesv8-armx-linux32.o" "/home/bobby/.cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.13.5/pregenerated/aesv8-armx-linux32.S"
OPT_LEVEL = Some("0")
TARGET = Some("arm-unknown-linux-gnueabihf")
HOST = Some("x86_64-unknown-linux-gnu")
CC_arm-unknown-linux-gnueabihf = None
CC_arm_unknown_linux_gnueabihf = None
TARGET_CC = None
CC = None
CROSS_COMPILE = None
CFLAGS_arm-unknown-linux-gnueabihf = None
CFLAGS_arm_unknown_linux_gnueabihf = None
TARGET_CFLAGS = None
CFLAGS = None
DEBUG = Some("true")
running "arm-linux-gnueabihf-gcc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-march=armv6" "-marm" "-mfpu=vfp" "-I" "include" "-Wall" "-Wextra" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-declarations" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-Wno-cast-align" "-fstack-protector" "-g3" "-D_XOPEN_SOURCE=700" "-c" "-o/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/ghashv8-armx-linux32.o" "/home/bobby/.cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.13.5/pregenerated/ghashv8-armx-linux32.S"

(skipping nearly-identical gcc invocations...)

OPT_LEVEL = Some("0")
TARGET = Some("arm-unknown-linux-gnueabihf")
HOST = Some("x86_64-unknown-linux-gnu")
CC_arm-unknown-linux-gnueabihf = None
CC_arm_unknown_linux_gnueabihf = None
TARGET_CC = None
CC = None
CROSS_COMPILE = None
CFLAGS_arm-unknown-linux-gnueabihf = None
CFLAGS_arm_unknown_linux_gnueabihf = None
TARGET_CFLAGS = None
CFLAGS = None
DEBUG = Some("true")
running "arm-linux-gnueabihf-gcc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-march=armv6" "-marm" "-mfpu=vfp" "-I" "include" "-Wall" "-Wextra" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-declarations" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-Wno-cast-align" "-fstack-protector" "-g3" "-D_XOPEN_SOURCE=700" "-c" "-o/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/x25519-asm-arm.o" "crypto/curve25519/asm/x25519-asm-arm.S"
running: "ar" "crs" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/libring-core.a" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/aesv8-armx-linux32.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/ghashv8-armx-linux32.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/aes-armv4-linux32.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/bsaes-armv7-linux32.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/armv4-mont-linux32.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/chacha-armv4-linux32.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/ecp_nistz256-armv4-linux32.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/ghash-armv4-linux32.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/poly1305-armv4-linux32.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/sha256-armv4-linux32.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/sha512-armv4-linux32.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/aes.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/exponentiation.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/generic.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/montgomery.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/montgomery_inv.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/shift.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/e_aes.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/crypto.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/ecp_nistz.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/ecp_nistz256.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/gfp_p256.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/gfp_p384.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/limbs.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/mem.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/gcm.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/curve25519.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/cpu-arm-linux.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/cpu-arm.o" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/x25519-asm-arm.o"
exit code: 0
cargo:rustc-link-lib=static=ring-core
OPT_LEVEL = Some("0")
TARGET = Some("arm-unknown-linux-gnueabihf")
HOST = Some("x86_64-unknown-linux-gnu")
CC_arm-unknown-linux-gnueabihf = None
CC_arm_unknown_linux_gnueabihf = None
TARGET_CC = None
CC = None
CROSS_COMPILE = None
CFLAGS_arm-unknown-linux-gnueabihf = None
CFLAGS_arm_unknown_linux_gnueabihf = None
TARGET_CFLAGS = None
CFLAGS = None
DEBUG = Some("true")
running "arm-linux-gnueabihf-gcc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-march=armv6" "-marm" "-mfpu=vfp" "-I" "include" "-Wall" "-Wextra" "-std=c1x" "-Wbad-function-cast" "-Wmissing-prototypes" "-Wnested-externs" "-Wstrict-prototypes" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-declarations" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-Wno-cast-align" "-fstack-protector" "-g3" "-D_XOPEN_SOURCE=700" "-c" "-o/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/constant_time_test.o" "crypto/constant_time_test.c"
running: "ar" "crs" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/libring-test.a" "/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out/constant_time_test.o"
exit code: 0
cargo:rustc-link-lib=static=ring-test
cargo:rustc-link-search=native=/c/Users/reyno/Source/xc-pi-zero/target/arm-unknown-linux-gnueabihf/debug/build/ring-27b2bc1ece367170/out

@pietro
Copy link
Contributor

pietro commented Feb 21, 2019

Could be a red herring but it could be something too:

$ git grep \\.arch -- crypto
crypto/chacha/asm/chacha-armv4.pl:.arch  armv7-a
crypto/chacha/asm/chacha-armv4.pl:.arch armv7-a
crypto/fipsmodule/aes/asm/aes-armv4.pl:.arch  armv7-a
crypto/fipsmodule/aes/asm/aesv8-armx.pl:$code.=".arch   armv8-a+crypto\n"                       if ($flavour =~ /64/);
crypto/fipsmodule/aes/asm/aesv8-armx.pl:.arch   armv7-a // don't confuse not-so-latest binutils with argv8 :-)
crypto/fipsmodule/aes/asm/bsaes-armv7.pl:.arch  armv7-a
crypto/fipsmodule/bn/asm/armv4-mont.pl:.arch  armv7-a
crypto/fipsmodule/bn/asm/armv4-mont.pl:.arch    armv7-a
crypto/fipsmodule/modes/asm/ghash-armv4.pl:.arch  armv7-a
crypto/fipsmodule/modes/asm/ghash-armv4.pl:.arch        armv7-a
crypto/fipsmodule/modes/asm/ghashv8-armx.pl:$code.=".arch       armv8-a+crypto\n"       if ($flavour =~ /64/);
crypto/fipsmodule/sha/asm/sha256-armv4.pl:.arch  armv7-a
crypto/fipsmodule/sha/asm/sha256-armv4.pl:.arch armv7-a
crypto/fipsmodule/sha/asm/sha512-armv4.pl:.arch  armv7-a
crypto/fipsmodule/sha/asm/sha512-armv4.pl:.arch armv7-a
crypto/perlasm/arm-xlate.pl:    if ($flavour =~ /linux/)        { ".arch\t".join(',',@_); }

Most of the the .arch armv7-a lines come from 875095a.

@briansmith
Copy link
Owner

I think @pietro's comment probably identified the issue. @reynoldsbd can you revert that commit and see if that fixes the issue for you?

My understanding is that the RPi team recommends a particular ARMv6 toolchain (based on GCC) that, when used, will generate executables that work on every RPi device from Zero to 3. Ideally, when building ring with this toolchain, we'd include the ARMv7 (specifically NEON) optimized code. That way, the code runs at maximum performance on each device based on runtime feature detection. To make that work, we'll need to look more closely at how the OpenSSL (not BoringSSL) project deals with this kind of issue. I think BoringSSL is only interested in ARMv7+ AFAICT.

Now, if you are targetting RPi Zero specifically you probably would rather NOT include the ARMv7 code (especially NEON) because it's just a waste. In order to support that, we'd probably need to modify the assembly modules to compile the ARMv7 code only when #if ARM_MAX_ARCH>=7. It looks like that is actually already done.

So, maybe we just need to put #if __ARM_MAX_ARCH__>=7 around the lines of code added in 875095a.

WDYT?

@briansmith
Copy link
Owner

Now, if you are targetting RPi Zero specifically you probably would rather NOT include the ARMv7 code (especially NEON) because it's just a waste. In order to support that, we'd probably need to modify the assembly modules to compile the ARMv7 code only when #if ARM_MAX_ARCH>=7. It looks like that is actually already done.

Actually, in aes.rs and gcm.rs, we'd need to change the conditions in #[cfg(target_arch = "arm")] to test whatever the equivalent of __ARM_MAX_ARCH__ is in Rust.

@reynoldsbd
Copy link
Author

Good news: after rebooting and setting up a clean build environment, my cross-compiled binaries are now running without crashing on the Pi Zero.

Sorry for the false alarm!

@bigfish24
Copy link

@briansmith I appear to have this same issue - did any changes occur as a result of this original report?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants