Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: don't clobber saved frame pointer in arm64 assembly functions #170

Merged
merged 5 commits into from
Oct 25, 2024

Conversation

nsrip-dd
Copy link
Contributor

@nsrip-dd nsrip-dd commented Oct 23, 2024

The arm64 neon assembly functions in this repository overwrite the frame
pointer saved by their callers, leading to crashes from the Go runtime
execution tracer and profilers which use frame pointer unwinding. For
historical reasons, on arm64 Go functions save the caller's frame
pointer register (x29) one word below their stack frame. See
go.dev/s/regabi#arm64-architecture. The assembly functions here,
translated from C compiler output, save values at the top of their
frame, and overwrite the frame pointer saved by the caller. We can fix
this by decrementing the stack pointer past where that frame pointer is
saved before saving anything on the stack.

Fixed with this sed script on my macos laptop + manual cleanup to match
indentation:

/stp[\t ]*x29/i\
	// The Go ABI saves the frame pointer register one word below the \
	// caller's frame. Make room so we don't overwrite it. Needs to stay \
	// 16-byte aligned \
	SUB $16, RSP

/ldp[\t ]*x29/a\
	// Put the stack pointer back where it was \
	ADD $16, RSP

Ran the script from the root of this repository with

find . -name '*_arm64.s' -exec sed -f fix.sed -i '' {} +

Then manually inspected the assembly for missing SUBs/ADDs at the
beginning of functions and prior to returns.

Fixes #150

The arm64 neon assembly functions in this repository overwrite the frame
pointer saved by their callers, leading to crashes from the Go runtime
execution tracer and profilers which use frame pointer unwinding. For
historical reasons, on arm64 Go functions save the caller's frame
pointer register (x29) one word below their stack frame. See
go.dev/s/regabi#arm64-architecture. The assembly functions here,
translated from C compiler output, save values at the top of their
frame, and overwrite the frame pointer saved by the caller. We can fix
this by decrementing the stack pointer past where that frame pointer is
saved before saving anything on the stack.

Fixed with this sed script on my macos laptop + manual cleanup to match
indentation:

```sed
/stp[\t ]*x29/i\
	// The Go ABI saves the frame pointer register one word below the \
	// caller's frame. Make room so we don't overwrite it. Needs to stay \
	// 16-byte aligned \
	SUB $16, RSP

/ldp[\t ]*x29/a\
	// Put the stack pointer back where it was \
	ADD $16, RSP

```

Ran the script from the root of this repository with

	find . -name '*_arm64.s' -exec sed -f fix.sed -i '' {} +

Then manually inspected the assembly for missing SUBs/ADDs at the
beginning of functions and prior to returns.

Fixes apache#150
@zeroshade
Copy link
Member

@nsrip-dd We do have CI which runs on ARM machines, could we add a test appropriately for at least one or a couple of these which uses tracing as per your original reproducer to validate that this continues to work?

For at least a few of the assembly-backed functions, add unit tests
which check that frame pointer unwinding works after calling them.
@nsrip-dd
Copy link
Contributor Author

@nsrip-dd We do have CI which runs on ARM machines, could we add a test appropriately for at least one or a couple of these which uses tracing as per your original reproducer to validate that this continues to work?

Sorry for the delay. I've added a test which I confirmed fails without my fix. Right now there's only a test for memory.Set. In theory we could test the other functions similarly. But I found it tricky because many of the assembly functions are a few function calls deep from the public API, and it seems like most of the tests in this repo use only the public APIs for the packages they test. Any suggestion on how to proceed? Or is what I have enough for now?

@nsrip-dd nsrip-dd marked this pull request as ready for review October 25, 2024 14:52
@nsrip-dd nsrip-dd requested a review from zeroshade as a code owner October 25, 2024 14:52
Copy link
Member

@zeroshade zeroshade left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM in general, just a couple nit picks

nsrip-dd and others added 2 commits October 25, 2024 11:13
- Add license header to fpunwind_arm64.s
- Add comment explaining the motivation for TestSetNoClobberFramePointer
@zeroshade zeroshade merged commit 19bd313 into apache:main Oct 25, 2024
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Go] arm64 assembly clobbers the saved frame pointer register
2 participants