Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Go] Improve 32-bit support (GOARCH=386) #32

Open
joellubi opened this issue Jul 10, 2024 · 2 comments
Open

[Go] Improve 32-bit support (GOARCH=386) #32

joellubi opened this issue Jul 10, 2024 · 2 comments
Labels
Type: enhancement New feature or request

Comments

@joellubi
Copy link
Member

Describe the enhancement requested

There are several parts of the code that currently fail on 32-bit systems like 386.

  1. One problem is that the refCount int64 we currently use for reference-counted resources requires manual alignment on 32-bit systems when used with atomic.AddInt64. This was raised in [Go] parquet/pqarrow: panic: unaligned 64-bit atomic operation arrow#43186.
  2. Another problem specific to 386 arch is that certain functions with archictecture-specific implementations do not have variants defined for 386, causing them to panic when compiled for that system.

apache/arrow#43206 starts to address these issues for a narrow subset of the codebase. This issue is intended to track the remaining work needed to add full support across the codebase.

Component(s)

Go
@joellubi joellubi added the Type: enhancement New feature or request label Jul 10, 2024
@assignUser assignUser transferred this issue from apache/arrow Aug 30, 2024
@jas4711
Copy link
Contributor

jas4711 commented Feb 11, 2025

Debian supports three 32-bit platforms: i386, armhf, and armel. So we would like to see arrow-go work on 32-bit platforms. On i386, our stripped down build works fine, but there are self-tests failures that only occur on i386 and not amd64:

https://salsa.debian.org/jas/golang-github-apache-arrow-go/-/jobs/7074351

Build log with >1000 FAIL's:

https://salsa.debian.org/jas/golang-github-apache-arrow-go/-/jobs/7074351/artifacts/raw/debian/output/golang-github-apache-arrow-go_18.1.0-1+salsaci+20250211+5_i386.build

For example the first failure:

=== NAME  TestBitmapOps/TestBitmapAnd/aligned/left_offset_3/right_offset_3/out_offset_3
    panic.go:262: test panicked: runtime error: invalid memory address or nil pointer dereference
        goroutine 192 [running]:
        runtime/debug.Stack()
        	/usr/lib/go-1.23/src/runtime/debug/stack.go:26 +0x83
        github.com/stretchr/testify/suite.failOnPanic(0xa8ff408, {0x843d980, 0x8769920})
        	/builds/jas/golang-github-apache-arrow-go/debian/output/source_dir/_build/src/github.com/stretchr/testify/suite/suite.go:89 +0x38
        github.com/stretchr/testify/suite.recoverAndFailOnPanic(0xa8ff408)
        	/builds/jas/golang-github-apache-arrow-go/debian/output/source_dir/_build/src/github.com/stretchr/testify/suite/suite.go:83 +0x3b
        panic({0x843d980, 0x8769920})
        	/usr/lib/go-1.23/src/runtime/panic.go:785 +0x103
        github.com/apache/arrow-go/v18/arrow/bitutil.alignedBitmapOp({0x84a0cc8, 0x84a0ccc, 0x0}, {0xaae59c4, 0x3, 0x3}, {0xaae5a04, 0x3, 0x3}, 0x3, ...)
        	/builds/jas/golang-github-apache-arrow-go/debian/output/source_dir/_build/src/github.com/apache/arrow-go/v18/arrow/bitutil/bitmaps.go:516 +0x624
        github.com/apache/arrow-go/v18/arrow/bitutil.BitmapOp({0x84a0cc8, 0x84a0ccc, 0x0}, {0xaae59c4, 0x3, 0x3}, {0xaae5a04, 0x3, 0x3}, 0x3, ...)
        	/builds/jas/golang-github-apache-arrow-go/debian/output/source_dir/_build/src/github.com/apache/arrow-go/v18/arrow/bitutil/bitmaps.go:545 +0x163
        github.com/apache/arrow-go/v18/arrow/bitutil.BitmapOpAlloc({0x850dde4, 0x8781c20}, {0x84a0cc8, 0x84a0ccc, 0x0}, {0xaae59c4, 0x3, 0x3}, {0xaae5a04, 0x3, ...}, ...)
        	/builds/jas/golang-github-apache-arrow-go/debian/output/source_dir/_build/src/github.com/apache/arrow-go/v18/arrow/bitutil/bitmaps.go:555 +0x16f
        github.com/apache/arrow-go/v18/arrow/bitutil.BitmapAndAlloc({0x850dde4, 0x8781c20}, {0xaae59c4, 0x3, 0x3}, {0xaae5a04, 0x3, 0x3}, 0x3, 0x3, ...)
        	/builds/jas/golang-github-apache-arrow-go/debian/output/source_dir/_build/src/github.com/apache/arrow-go/v18/arrow/bitutil/bitmaps.go:568 +0xca
        github.com/apache/arrow-go/v18/arrow/bitutil_test.(*BitmapOpSuite).testAligned.func1.1.1()
        	/builds/jas/golang-github-apache-arrow-go/debian/output/source_dir/_build/src/github.com/apache/arrow-go/v18/arrow/bitutil/bitmaps_test.go:411 +0x28a
        github.com/stretchr/testify/suite.(*Suite).Run.func1(0xa8ff408)
        	/builds/jas/golang-github-apache-arrow-go/debian/output/source_dir/_build/src/github.com/stretchr/testify/suite/suite.go:115 +0x141
        testing.tRunner(0xa8ff408, 0xa87a200)
        	/usr/lib/go-1.23/src/testing/testing.go:1690 +0x119
        created by testing.(*T).Run in goroutine 191
        	/usr/lib/go-1.23/src/testing/testing.go:1743 +0x3d1
...
--- FAIL: TestBitmapOps (1.05s)
    --- FAIL: TestBitmapOps/TestBitmapAnd (0.51s)
        --- FAIL: TestBitmapOps/TestBitmapAnd/aligned (0.03s)
...
            --- FAIL: TestBitmapOps/TestBitmapAnd/aligned/left_offset_3 (0.01s)
                --- FAIL: TestBitmapOps/TestBitmapAnd/aligned/left_offset_3/right_offset_3 (0.00s)
                    --- FAIL: TestBitmapOps/TestBitmapAnd/aligned/left_offset_3/right_offset_3/out_offset_3 (0.00s)
                        --- PASS: TestBitmapOps/TestBitmapAnd/aligned/left_offset_3/right_offset_3/out_offset_3/zero-length (0.00s)
                    --- FAIL: TestBitmapOps/TestBitmapAnd/aligned/left_offset_3/right_offset_3/out_offset_19 (0.00s)
                        --- PASS: TestBitmapOps/TestBitmapAnd/aligned/left_offset_3/right_offset_3/out_offset_19/zero-length (0.00s)
                    --- FAIL: TestBitmapOps/TestBitmapAnd/aligned/left_offset_3/right_offset_3/out_offset_27 (0.00s)
                        --- PASS: TestBitmapOps/TestBitmapAnd/aligned/left_offset_3/right_offset_3/out_offset_27/zero-length (0.00s

Any ideas?

@zeroshade
Copy link
Member

I must have made a mistake on some of the build constraints. Let me take a look and do some testing for 32-bit arch and get back to you.

@zeroshade zeroshade mentioned this issue Feb 11, 2025
Merged
zeroshade added a commit that referenced this issue Feb 12, 2025
@zeroshade
fix(arrow/bitutil): fix bitmap ops on 32-bit platforms (#277)
1d3100c 
### Rationale for this change
Fixes the broken 32-bit arch tests identified by #32 

### What changes are included in this PR?
Defining the `opAligned` pure go fallback methods for bitmap operations.

### Are these changes tested?
Yes, it makes the tests succeed properly on `GOARCH=386`

### Are there any user-facing changes?
No.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Type: enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jas4711 @zeroshade @joellubi