NDFD Overview - conrad-blucher-institute/semaphore GitHub Wiki
NDFD stands for National Digital Forecast Database and is a database of forecast data maintained by the National Weather Service. The database contains a suite of gridded forecasts for standard meteorological and oceanic weather elements such as wind speed and direction, air temperature, and wave height.
The database is updated twice per hour at 25 and 55 minutes after the hour (*1).
The data from the generated forecasts are available through 5 different vehicles each one described in more detailed below.
- as GRIB2 encoded file that are downloadable via HTTPS and FTPS
- via an operational SOAP web service that provides coarse-resolution for original NDFD elements
- via an experimental SOAP web service that provides fine-resolution for both original and new NDFD elements
- via an operational REST web service that provides coarse-resolution for original NDFD elements
- via an experimental REST web service that provides fine-resolution for both original and new NDFD elements
NDFD GRIB2 files are organized by geographical sectors. For the sectors relevant to South Texas see below (for up-to-date sector info see NDFD geographical sectors. Each geographical sector has a dedicated folder (and thus URL) for the GRIB2 files.
- Southern plains includes all of Texa (AR.splains) - URL segment : 'DF.gr2/DC.ndfd/AR.splains/'
- Full CONUS (AR.conus) - URL segment: 'DF.gr2/DC.ndfd/AR.conus/'
Each NDFD GRIB2 file contains the predictions for a single weather element for a given geographical sector. For example, the wind speed predictions are contained in a file named 'ds.wspd.bin'. NDFD GRIB2 files are further organized in folder based on lead times (and possibly other things? need to research more)(*1). Most of the regions and weather elements, including the Southern Plains and basic weather elements such as air temp and wind data follow the below organization:
- VP.001-003: contains files with lead times up to 54 or 72 hours (seem to vary depending on time of day and/or weather element)
- VP.004-007: contains files with lead times past what the previous folder contains and typically up to 156 hours
- VP.008-450: contains files with long range forecast typically 12 days and/or more.
Files are named base on the weather element they provide predictions for and have a '.bin' extension. For example wind speed predictions are in a file named 'ds.wspd.bin'.
Note that not all regions support all 3 folders. The Southern Plains regions does support all three. For a description and additional information of all weather elements, NDFD files and their contents and grids and sectors, see NDFDSpreadsheet.xls. To download the latest spreadsheet go to NDFD Information Links and scroll down to the NDFD Products and status section to see the link to download the spreadsheet.
NDFD GRIB2 files are available through both https and ftp at the following base URL (for operational weather elements)(*2):
To obtain a file, we then construct the URL following this template:
<base url><geagraphical sector><folder name for targeted lead times><weather element file name>
For example, to download the wind speed direction for the southern plains for lead times from 1 hour up to 54 or 72 hours, use the following link
ftp://tgftp.nws.noaa.gov/SL.us008001/ST.opnl/DF.gr2/DC.ndfd/AR.splains/VP.001-003/ds.wspd.bin
Note that the server is typically locked down when files are being uploaded so it is a good idea to schedule download enough minutes away from the top and bottom of the hour.
Files are updated on the FTP server regularly but the schedule seem to vary by variable and lead time. Here's what we have witnessed/experienced so far.
- For predictions from 1 to 55 hours (provides predictions at 3 hours interval, so lead times are 1h, 4h, 7h, etc.): file is uploaded every hour, about 10 minutes before the hour.
- For predictions from 66 to 156 hours (provides predictions at 6 hours interval, so lead times are 66h, 72h, 78h, etc.): file is uploaded every ? hours, 5 minutes passed the bottom of the hour.
Getting data out of the files with wgrib2. Because of the way the files are organized on the ndfd server, each file contains predictions for only one series (e.g., air temp, wind dir, etc.). So it's not necessary to ask for the variable or restrict the search to that variable but it is good practice to do so. So getting the data out of an air temp prediction file would look like this:
wgrib2 ds.temp2.bin -match ':TMP' -T -ftime -var -lon -97.31286 27.46767
-T gets you the prediction time and -ftime gets you the lead time (the example is for the sbib location - or close to it). The above command would get you results like the below:
~/ndfd_downloads$ wgrib2 ds.temp1.bin -match ':TMP' -T -ftime -var -lon -97.31286 27.46767
1:80:D=20241106170000:1 hour fcst:TMP:lon=262.685591,lat=27.451409,val=298.1
2:48268:D=20241106170000:4 hour fcst:TMP:lon=262.685591,lat=27.451409,val=298.7
3:96417:D=20241106170000:7 hour fcst:TMP:lon=262.685591,lat=27.451409,val=297.6
4:144232:D=20241106170000:10 hour fcst:TMP:lon=262.685591,lat=27.451409,val=297.6
5:190215:D=20241106170000:13 hour fcst:TMP:lon=262.685591,lat=27.451409,val=297.6
6:235625:D=20241106170000:16 hour fcst:TMP:lon=262.685591,lat=27.451409,val=297
7:280801:D=20241106170000:19 hour fcst:TMP:lon=262.685591,lat=27.451409,val=297
8:325262:D=20241106170000:22 hour fcst:TMP:lon=262.685591,lat=27.451409,val=299.3
9:370213:D=20241106170000:25 hour fcst:TMP:lon=262.685591,lat=27.451409,val=300.9
` ...
(*1) As far as we are able to tell, NDFD generates forecasts with different offsets during the first half of the hour compared to during the second half of the hour:
-
first half of the hour:
- reference time is at the top of the hour (e.g, 1500)
- time offset unit is hours, starts at 1 hour and is in 1 hour increments. so 1 hour, 2 hours, 3 hours, etc.
-
second half of the hour:
- reference time is at the bottom of the hour (e.g, 1530)
- time offset unit is minutes, starts at 30 min (so the forecast is for the top of the hour) and is in 60 minutes increments. so 30 min, 90 min, 150 min, etc.
_(*2) There is also an experimental server that is not included in this documentation