merging requires sky FIBERFLUX* (but shouldn't)

[sbailey]

Something in the formats update PR #228 ended upon making FIBERFLUX* a required column for input skies, which may not be the case for off-the-footprint sky files.

Review the chain for what columns are required and and make it as minimalist as possible. The intension is that fiberassign requires only the minimal columns that are actually necessary for fiberassign, and beyond that it propagates the superset of columns that it finds in any of its input target files, regardless of what those might be.

Context: target files can genuinely have different columns than sky files (e.g. PRIORITY and SHAPE* parameters), but they have to be merged into a common table for fiberassign targets. An additional complication is that some columns have to be renamed for ICS (e.g. RA/DEC -> TARGET_RA, TARGET_DEC).

Thanks for @Srheft for reporting this problem.

[sbailey]

This is currently blocking 19.9 integration testing. We either need to add FIBERFLUX_IVAR_G/R/Z to the skies output by desitarget.mocks or we need to make fiberassign not require it. Adding this ticket to the 19.9 project for now as a reminder.

[sbailey]

Update: it turns out the problem is with the SKY_MONITOR HDU rather than merging sky targets with science targets in the FIBERASSIGN HDU. In the case of SKY_MONITOR HDU, we have a datamodel interface with ICS to follow that specifies including FIBERFLUX_IVAR_G/R/Z. It may actually be optional, but we shouldn't arbitrarily remove it without coordinating.

Since these are passed to platemaker for assignment, the list of required columns for FIBERASSIGN also applies here:

TARGETID
PETAL_LOC
DEVICE_LOC
FIBERSTATUS
TARGET_RA
TARGET_DEC
FIBER

@dkirkby can you confirm which (if any) of these additional columns in SKY_MONITOR are used by the ETC code?

FIBERFLUX_G
FIBERFLUX_R
FIBERFLUX_Z
FIBERFLUX_IVAR_G
FIBERFLUX_IVAR_R
FIBERFLUX_IVAR_Z
FIBERASSIGN_X
FIBERASSIGN_Y
LOCATION
NUMTARGET
BRICKID
BRICK_OBJID
FA_TARGET
FA_TYPE
BRICKNAME
PRIORITY
SUBPRIORITY