Making .gz Files Useful For MetPy Operations

[kylejgillett]

Hello folks!

I am attempting to get ahold of ACARs data from the NWS but the data files that I have found are all .gz files which are very difficult to deal with. Any ideas? getting them into a manageable netCDF file and then into a pandas dataframe would be the best option.

The data I am attempting to use is here: ([https://madis-data.ncep.noaa.gov/madisPublic1/data/point/acars/netcdf/])

Thanks!

[DanielAdriaansen]

I think these files are just compressed using "gzip". On Linux, you can just do "gunzip filename.nc.gz" to get the regular netcdf file back.

[kylejgillett]

Any solutions for Windows?

[DanielAdriaansen]

Maybe Peazip?

https://peazip.github.io/

[kylejgillett]

I will try that, thanks!

[dopplershift]

Given that they're netCDF, I think the xarray solution from @akrherz is the right way. For future reference though, the Python standard library has the gzip module that should work regardless of platform to open the file. So you can use it to get a "file-like object" that lets you e.g. call read():

import gzip

fileobj = gzip.GzipFile(filename)

While many Python libraries work with file-like objects, netcdf4-python does not (since it uses netcdf-c under the hood). But you can pass it a memory buffer to use:

from netCDF4 import Dataset

buffer = fileobj.read()
nc = Dataset('myinmemoryfile.nc', memory=buffer)

[akrherz]

I have not tested this on windows, but xarray may be able to load it in one step

import xarray as xr
# wget https://madis-data.ncep.noaa.gov/madisPublic1/data/point/acars/netcdf/20220920_0000.gz
nc = xr.open_dataset('20220920_0000.gz')

[kylejgillett]

Okay so this did work. Now I have to figure out how to utilize this massive mess of data 🙃 lol. Thank you!