support GAIA standard stars

[sbailey]

Apologies if I'm behind on a conversation about this, but I don't see an issue on it.

In desihub/desitarget#669 @geordie666 added three new standard star classes:

    - [GAIA_STD_FAINT,      33, "Standard stars for dark/gray conditions",   {obsconditions: DARK|GRAY}]
    - [GAIA_STD_WD,         34, "White Dwarf stars",                         {obsconditions: DARK|GRAY|BRIGHT}]
    - [GAIA_STD_BRIGHT,     35, "Standard stars for BRIGHT conditions",      {obsconditions: BRIGHT}]

He notes:

Crucially, these are added to the mws_mask (and sv1_mws_mask), not to desi_mask:

https://github.com/desihub/desitarget/blob/22229d03115ec1804fa6e5464d221c1bb97bf22d/py/desitarget/sv1/data/sv1_targetmask.yaml#L140-L142

These are analogues of the already-defined standards, but based only on Gaia to facilitate observations outside of the Legacy Surveys imaging footprint

We should have support for these in fiberassign too, but I think they should be treated as lower-priority standards that only get used if we can't fulfill our budget with regular standards (analogous to SUPP_SKY vs. SKY). However, it looks like fiberassign doesn't yet have support for two tiers of standards which makes the implementation a bit messier.

Alternatively, maybe (?) these could be just as good as normal standards in the main footprint. @geordie666 do you or others have a study of the overlap between these GAIA_STD_* targets within the main footprint and a recommendation of whether they can be treated on-par with normal standards or if they need to be treated only as backup standards? Again apologies if this has already been hashed out on list or telecons and I missed it.

[geordie666]

I think the strong preference would be to only use the GAIA_STD* standards if there aren't enough of the normal standards. So, a two-tier system.

The Gaia standards don't incorporate information from the Legacy Surveys. The Legacy Surveys imaging is useful for setting more precise color constraints on the standard stars.

[araichoor]

as @sbailey says, I think the fiberassign code cannot assign fibers to standard with different priorities, i.e. it treats standards as a separate class, with no notion of priority; I could try to locate the relevant lines in the code; though, I currently don't feel like touching the "core" c++ code...

[araichoor]

@sbailey @geordie666:
As a follow-up of this morning call, I confirm that those GAIA_STD targets live only in the backup catalogs (/global/cfs/cdirs/desi/target/catalogs/gaiadr2/0.57.0/targets/sv3/resolve/backup), i.e. there are none in the targets folders of the bright/dark programs (/global/cfs/cdirs/desi/target/catalogs/dr9/0.57.0/targets/sv3/resolve/{bright,dark}).

Currently fba_launch only reads for each program the dedicated desitarget folder.
However, I don t currently see anything preventing to always add a reading of the backup target folder, select GAIA_STD, and add those in the target list; so that shouldn t be a blocking factor here.

[geordie666]

Sorry, I totally missed this issue!

I think the "regular" standards are significantly superior to the Gaia-only standards, because they use Legacy Surveys colors to help with the selection box.

I would have no problem "topping-up" standard stars by using the Gaia-only standards at lower priority, but I think we should always try to reach full coverage with the regular standards and only add the Gaia-only standards as a last resort. In general, less development work has gone into the Gaia-only standards.

As with all other Gaia-only targets, the Gaia-only standards will only appear in the BACKUP files (what used to be called the "supp" files).

[araichoor]

yes, the idea was to use gaia standards in case of a lack of dr9 standards, ie have a two-priority approach; but that requires surgery in the c++ code.

[tskisner]

Referencing your "surgery in the C++ code", do we really need to define a new fiberassign target type? Why not just add the Gaia-only targets at lower priority than any normal standards? That way they will only be used to meet the standards budget if there are no regular standards available.

[araichoor]

If I m correct, standards have no priority (priority=-1), they are dealt with in a special way:
https://github.com/desihub/desitarget/blob/073e430c0c148756c0d2e79fcb28cf0b2c4ab589/py/desitarget/sv3/data/sv3_targetmask.yaml#L231-L236
https://github.com/desihub/desitarget/blob/073e430c0c148756c0d2e79fcb28cf0b2c4ab589/py/desitarget/sv3/data/sv3_targetmask.yaml#L285-L288

[tskisner]

In fiberassign, the target table is loaded and any priority values are simply used. So if the standards catalog has a priority column then it already works with fiberassign

…

On May 6, 2021, at 2:32 PM, araichoor ***@***.***> wrote:  If I m correct, standards have no priority (priority=-1), they are dealt with in a special way: https://github.com/desihub/desitarget/blob/073e430c0c148756c0d2e79fcb28cf0b2c4ab589/py/desitarget/sv3/data/sv3_targetmask.yaml#L231-L236 https://github.com/desihub/desitarget/blob/073e430c0c148756c0d2e79fcb28cf0b2c4ab589/py/desitarget/sv3/data/sv3_targetmask.yaml#L285-L288 — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[geordie666]

So if I change the Gaia-only standards to have a priority of -2, fiberassign would understand that the Gaia-only standards should be assigned after the regular standards (which have a priority of -1)?

[tskisner]

Ok, I see the problem now.

When loading target tables, fiberassign uses the PRIORITY column if it exists, else it uses PRIORITY_INIT, or if neither column exists it sets priority to zero:

fiberassign/py/fiberassign/targets.py

Line 642 in a0cc61b

if "PRIORITY" in tgdata.dtype.fields:

Later, all target (including standards) are assigned according to their "total_priority":

fiberassign/src/targets.cpp

Line 104 in a0cc61b

return (double)(priority * 100 + obsremain) + subpriority;

SO: if you assign normal standards a priority (any integer) and assign Gaia-only targets a lower integer priority, then during the assignment to unassigned fibers, the normal standards will be assigned first.

However, the complication comes when bumping science targets to place standards. This happens in inverse order of science target priority (which is what we want), and the priority of standards is only considered if there are multiple standards available to the fibers we are bumping science from.

If you want to "bump science targets for normal standards up to some limit" and then "bump science targets for Gaia standards up to some limit", then we are in exactly the same situation we faced with "SKY" and "SUPP_SKY". And of course if we start creating sky targets on the fly it removes the need for all that previous work!

So, before doing this same kind of additional target class for "SUPP_STANDARDS", let's make sure we really need it. It is not impossible, just somewhat tedious...