Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Should METAR parsing have more datetime business logic added? #2707

Open
dopplershift opened this issue Oct 3, 2022 Discussed in #2706 · 1 comment
Open

Should METAR parsing have more datetime business logic added? #2707

dopplershift opened this issue Oct 3, 2022 Discussed in #2706 · 1 comment
Labels
Area: IO Pertains to reading data Type: Enhancement Enhancement to existing functionality

Comments

@dopplershift
Copy link
Member

dopplershift commented Oct 3, 2022

Discussed in #2706

Originally posted by akrherz October 3, 2022
Whilst considering the PR for #2701, I realized that there are some other painful edge cases in MetPy + METAR parsing. Presently, MetPy requires the user to either:

  1. Apriori know the year and month of the METAR / METAR file being passed to MetPy or
  2. Let MetPy guess the year and month based on the calendar UTC date.

The issue is that neither of those will work for the situation of a user processing a stream of observations from a system like NOAAPort / LDM IDD. They will have situations where the METAR day, WMO header date, and calendar date may not match.

Should all this business logic / boilerplate be shunted to every user attempting to parse METARs with MetPy?

In python-metar we do have business logic that attempts to help the user out by using the current utcnow() value to more accurately guess a timestamp when no month/year is provided to the parsing library.

Should a similar approach be added to MetPy?

(cc @akrherz)

@dopplershift dopplershift added Type: Enhancement Enhancement to existing functionality Area: IO Pertains to reading data labels Oct 3, 2022
@dopplershift
Copy link
Member Author

So first, as identified in #1256, we should try to actually parse full METAR products, so that we can grab the correct date from the product header. Right now we have a kind of hack that just joins lines it thinks are from METARs (to handle wrapping) and discards everything else:

MetPy/src/metpy/io/metar.py

Lines 363 to 377 in ca26129

def full_metars(x, prefix=' '):
tmp = []
for i in x:
# Skip any blank lines
if not i.strip():
continue
# No prefix signals a new report, so yield
if not i.startswith(prefix) and tmp:
yield ' '.join(tmp)
tmp = []
tmp.append(i.strip())
# Handle any leftovers
if tmp:
yield ' '.join(tmp)

After that, having better heuristics for guessing the appropriate date by combining "now" and what's in the data would be good. I'm not in love with python-metar's code--there has to be a better way using timedelta rather than that decision tree of manual wrapping? (I could very well be wrong, though.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: IO Pertains to reading data Type: Enhancement Enhancement to existing functionality
Projects
Status: No status
Development

No branches or pull requests

1 participant