Reading and processing weather model data

Doing data science on atmospheric model output data involves using gridded data files in 4 dimensions (i.e., 3 spatial dimensions + time), mainly using one of two binary formats: grib (if you work mostly in operational meteorology) or netcdf (the preferred format from people doing climate science). I personally prefer netcdf, which I find more user friendly (see this excellent description of the format by Rohith Teja) . Netcdf files are self-describing, while grib files needs a table to be able to decipher what is inside the file, and different weather centers use different tables (although there is a well defined standars produced by the WMO). Also, netcdf follow a hierarchical format. A nice discussion of the netcdf vs grib format can be found in this blog post.

Most of the raw model output from NWP models comes in grib format (except for some open source models like WRF, that output model data in netcdf).Depending on where you work, you might end up using either a locally developed grib or netcdf reader, written my some local researcher years ago. I tend to prefer or one of the many open source libraries read atmospheric data.
In all my years working in NWP I’ve used either Fortran (old days, don’t do that anymore!), Python, NCL, R or even Julia.
Some of the tools that I have used in the past to read grib data:

Grads is a very old and pretty simple tool to read grib files, but I believe it is still widely used. It can read read grib or netcdf data.
NCL, the NCAR language is a pretty useful scripting language, now moving to the GeoCAT library, NCL is still widely used. The NCL email list is still pretty active (and friendly!), so it is probably still widely used.

In the python ecosystem there is wide variety of libraries that allow you to read grib or netcdf files:
iris is a pretty cool library developed at the met office. I only used it once about 3-4 years ago, mainly because at that time it still seemed to depend on python 2.7. Probably not an issue now.

Probably the best known and useful library to read grib data is the eccodes package, which includes both command line and python bindings.

Leave a Reply

Your email address will not be published. Required fields are marked *