[Java] DenseUnionWriter#setPosition fails with NullPointerException

[asfimport]

The writer always iterates through all BaseWriters, and an array of 128 BaseWriters is allocated. So if you do not have 128 typeIds and do not touch all of them, setPosition will give you an exception.

Reporter: David Li / @lidavidm

_{Note: This issue was originally created as ARROW-8665. Please see the migration documentation for further details.}

[asfimport]

Liya Fan / @liyafan82:
@lidavidm Sorry for the problem. I will provide a patch for it.

[asfimport]

David Li / @lidavidm:
No worries! I ran into this while trying to find a way to get the validity buffer to be resized when trying to work around ARROW-8666. But thanks to your suggestion there I should be set for our use case.

[asfimport]

Todd Farmer / @toddfarmer:
This issue was last updated over 90 days ago, which may be an indication it is no longer being actively worked. To better reflect the current state, the issue is being unassigned. Please feel free to re-take assignment of the issue if it is being actively worked, or if you plan to start that work soon.

[nbauernfeind]

This issue still persists.

Notably:

  @Override
  public void setPosition(int index) {
    super.setPosition(index);
    for (BaseWriter writer : writers) {
      writer.setPosition(index);
    }
  }

But, writers is an array of length 128 and each writer is lazily initialized - in that every writer is null.

The result is that it is impossible to write Lists of Unions using this library. Noting that UnionListWriter#startList invokes setPosition:

  @Override
  public void startList() {
    vector.startNewValue(idx());
    writer.setPosition(vector.getOffsetBuffer().getInt((idx() + 1L) * OFFSET_WIDTH));
    listStarted = true;
  }

[lidavidm]

@jbonofre @kou I think there are a lot of little bugs like this that Nate has been reporting or re-raising...maybe let's organize them with a label/project and see if we can tackle them over time? At least I don't want to lose track of the and Nate has been doing us a favor showing where we've overlooked things 😅

[jbonofre]

I think it's OK to re-add step by step this kind of issues. On some other ASF projects, we started with the issues specific for the first release (after a move from a repo to another). Then, we can add the "old" issues. I think it's ok 😄 (a bit of work/script).

[lidavidm]

I'm just going to start labeling with help-wanted (and good-first-issue if appropriate)

[jbonofre]

I'm just going to start labeling with help-wanted (and good-first-issue if appropriate)

It sounds good. I will help on that front 😄

[kou]

+1