Jeff,
I am doing a project which considers wind effects on aircrafts and the wind data used are from GRIB files. I have planned to use the GRIB files from NWS/NCEP FTP Servers rather than from Canadian Meteorological Center (CMC). I now have 5 questions concerning the download of GRIB files, and I hope you can answer them.
1. On PCGRIDDS32 home page, you have listed several hyperlinks for downloading under two headings, namely NWS GRIB and WAFS GRIB, what is the difference between NWS GRIB and WAFS GRIB ?
2. If my project involves consideration of wind effect on airplanes, am I right to use WAFS GRIB because WAFS is a program developed by ICAO and WMO to improve the quality and consistency of enroute guidance provided for international aircraft operations?
3. The directory or file structures for the GRIB FTP servers are so complicated for an amateur like me, can you tell me where I should download GRIB files that are relevant to my project?
4. Suppose my aircraft is going to fly off at 3 UTC and I want to forecast the wind effects on the aircraft at 3 UTC, am I right to go to the directory SL.us0080001/ST.opnl/MT.gfs_CY.00 and go to RD.{YYYYMMDD}/PT.grid_DF.gr1 and then download an octant file with forecast hour equal to 003, for instance, fh.003_tl.press_ar.octanti ?
5. Why the GRIB files in the directory MT.gfs_CY.00_o are not the latest and has reference date which is the past one, such as this one RD.20050901 while the GRIB files in MT.gfs_CY.00 are the latest and has up-to-date reference date such as this one RD.20060108 ? (remarks: I browsed the FTP server on Jan 08 2006) If the files in MT.gfs_CY.00_o are not referenced as the current day, how come you still recommend people to download GRIB files from MT.gfs_CY.{CC}_o on PCGRIDDS32 home page? For example, you recommend ftp://weather.noaa.gov/SL.us0080001/ST. ... rid_DF.gr1
***Questions on downloading of GRIB files****
Hi David,
WOW!!...All the questions
I'm a bit busy right now but I will address these later this week. I have updated the PCGRIDDS32 Home Page with some new information concerning the GRIB file information. Check it out & see if it answers any of your questions. Also, the WAFS grid description is described in the GRIB documentation which is included in the PCGRIDDS32/DOC directory.
I'll be back...
Jeff Krob
PCGRIDDS32 System Developer
WOW!!...All the questions
I'm a bit busy right now but I will address these later this week. I have updated the PCGRIDDS32 Home Page with some new information concerning the GRIB file information. Check it out & see if it answers any of your questions. Also, the WAFS grid description is described in the GRIB documentation which is included in the PCGRIDDS32/DOC directory.I'll be back...
Jeff Krob
PCGRIDDS32 System Developer
Hi Jeff,
Regarding Question 1, I have read the GRIB documentation which is included in the PCGRIDDS32/DOC directory, and understood that:
NWS GRIB files are non-WAFS GRIB data files and data for the entire grid are stored together as a single group in NWS GRIB data files. No pre-processing of NWS GRIB data is necessary before the data are converted to PCGRIDDS32 format.
While
Data for global grids are stored in WAFS data. This data must be preprocessed to create an inventory and list of model runs available for conversion from GRIB to PCGRIDDS32 format.
But my problem is that I only know entire grid are stored together as a single group in NWS GRIB data files while data for global grids are stored in WAFS data but do not understand what exactly these descriptions mean and what exactly the difference between NWS GRIB and WAFS data is. And my further problem is how to distinguish which file in the FTP server is NWS GRIB or WAFS GRIB.
Regarding Question 4's hypothesized case, is it true that there are several alternatives to downloading an octant file with forecast hour equal to 003 ? For example I can go to directory SL.us0080001/ST.opnl/MT.gfs_CY.18 and go to R D.{YYYYMMDD}/PT.grid_DF.gr1 and then download an octant file with forecast hour equal to 009, for instance, fh.009_tl.press_ar.octanti. (Remarks: I know I need to download the octant file that covers the areas of the flight)
Regarding Question 1, I have read the GRIB documentation which is included in the PCGRIDDS32/DOC directory, and understood that:
NWS GRIB files are non-WAFS GRIB data files and data for the entire grid are stored together as a single group in NWS GRIB data files. No pre-processing of NWS GRIB data is necessary before the data are converted to PCGRIDDS32 format.
While
Data for global grids are stored in WAFS data. This data must be preprocessed to create an inventory and list of model runs available for conversion from GRIB to PCGRIDDS32 format.
But my problem is that I only know entire grid are stored together as a single group in NWS GRIB data files while data for global grids are stored in WAFS data but do not understand what exactly these descriptions mean and what exactly the difference between NWS GRIB and WAFS data is. And my further problem is how to distinguish which file in the FTP server is NWS GRIB or WAFS GRIB.
Regarding Question 4's hypothesized case, is it true that there are several alternatives to downloading an octant file with forecast hour equal to 003 ? For example I can go to directory SL.us0080001/ST.opnl/MT.gfs_CY.18 and go to R D.{YYYYMMDD}/PT.grid_DF.gr1 and then download an octant file with forecast hour equal to 009, for instance, fh.009_tl.press_ar.octanti. (Remarks: I know I need to download the octant file that covers the areas of the flight)
David,
I think the following link will give you the information you are looking for.
http://www.nco.ncep.noaa.gov/pmb/docs/o ... l#table-a2
Jeff, I hope you will forgive me for butting in. Just thought I might help out a little here.
Kevin
I think the following link will give you the information you are looking for.
http://www.nco.ncep.noaa.gov/pmb/docs/o ... l#table-a2
Jeff, I hope you will forgive me for butting in. Just thought I might help out a little here.
Kevin
Dear Kevin,
Thank you very much for your nice diagram to make me clear which file is NWS and WAFS files.
But I have to say sorry that I have read the web-page which your hyperlink points to and found it difficult to understand the content and difficult to apply the content to my project. Actually I am a university student and a newbie to the world of GRIB data, this may prohibit my ability to understand your recommended web-page.
Dear Jeff,
I think it is important to receive different opinions from as many people as possible regarding a subject so that the knowledge obtained can be more objective. So I hope there is someone else like you can answer all of my questions (if possible) even though Kevin has helped me a lot by answering some of my questions.
Dear other forum users,
You are welcomed to express opinions on my questions if you know the answers.
Finally, thank you for your response.
Thank you very much for your nice diagram to make me clear which file is NWS and WAFS files.
But I have to say sorry that I have read the web-page which your hyperlink points to and found it difficult to understand the content and difficult to apply the content to my project. Actually I am a university student and a newbie to the world of GRIB data, this may prohibit my ability to understand your recommended web-page.
Dear Jeff,
I think it is important to receive different opinions from as many people as possible regarding a subject so that the knowledge obtained can be more objective. So I hope there is someone else like you can answer all of my questions (if possible) even though Kevin has helped me a lot by answering some of my questions.
Dear other forum users,
You are welcomed to express opinions on my questions if you know the answers.
Finally, thank you for your response.
David,
OK – here we go…
>Jeff,
>I am doing a project which considers wind effects on aircrafts and the >aind data used are from GRIB files. I have planned to use the GRIB files >from NWS/NCEP FTP Servers rather than from Canadian Meteorological >Center (CMC). I now have 5 questions concerning the download of GRIB >files, and I hope you can answer them.
I’ll do my best
>1. On PCGRIDDS32 home page, you have listed several hyperlinks for >downloading under two headings, namely NWS GRIB and WAFS GRIB, >what is the difference between NWS GRIB and WAFS GRIB ?
The World Area Forecast System (WAFS) was designed to disseminate global GRIB information to the third-world regions via a point-to-point communications. The WAFS GRIB products are taken from the GFS model and the global coverage is broken up into 8 sectors or “octants”; 4 in the northern hemisphere & 4 in southern hemisphere. The octants are identified by the letters I-P. Also, the data resolution is “thinned” from 1 x 1 degree in the GFS native grids to 1.25 x 1.25 degrees for WAFS and there are fewer vertical layers and fewer parameters. All this is to reduce the volume of data to be transmitted. For more info on WAFS, go to:
http://www.weather.gov/tg/iscscvr.html
The reason PCGRIDDS32 treats the WAFS GRIB files differently is because the ingest routine for the WAFS files, GRIB2PCG32.EXE can “glue” the individual, adjacent octants together to cover a larger area for a single file.
The GFS native GRIB files are 1 x 1 degree, global coverage in one file with many more vertical layers & more parameters so the files are larger, but you get more data and sice there are no sectors which need to be joined with other sectors, that data goes in the NWS directory along with all other GRIB-type files for processing.
In my previous message to you I recommended you read up on the GRIB standard where the WAFS (and all other grid domains) are described in detail. I think you actually read the PCGRIDDS User Guide which shows “how” to process the data but does not actually describe the GRIB data. I thought I included the GRIB documentation along with everything else. The /PCGRIDDS32/DOC file “Sect1.pdf” explains the GRIB grid domains and, yes, I know, I really should rename them
>2. If my project involves consideration of wind effect on airplanes, am I >right to use WAFS GRIB because WAFS is a program developed by ICAO >and WMO to improve the quality and consistency of enroute guidance >provided for international aircraft operations?
Yes, the WAFS system was designed to do that but also with the limitation of less data to work with & required to transfer. However, like what Kevin showed in his graphic, the fh.HHHH_tl.press_gr.onedeg GRIB files contain much more data in more detailed forecast hours and for a longer forecast period but take longer to download. The trade off is yours to decide.
>3. The directory or file structures for the GRIB FTP servers are so >complicated for an amateur like me, can you tell me where I should >download GRIB files that are relevant to my project?
The link I provided in my previous post to the SOO/STC Web site should show what the individual GRIB files contain and what their file names are to know what to download.
>4. Suppose my aircraft is going to fly off at 3 UTC and I want to forecast >the wind effects on the aircraft at 3 UTC, am I right to go to the directory >SL.us0080001/ST.opnl/MT.gfs_CY.00 and go to >RD.{YYYYMMDD}/PT.grid_DF.gr1 and then download an octant file with >forecast hour equal to 003, for instance, fh.003_tl.press_ar.octanti ?
Not neccesarily – the fh.003 is signifying 3 hours into the forecast period from when the model was initialized. If you want 03 UTC, you would want the fh.003 file from the 00 UTC model run. The fh.003 file from the 12 UTC model run would equate to 15 UTC & so on.
>5. Why the GRIB files in the directory MT.gfs_CY.00_o are not the latest >and has reference date which is the past one, such as this one >RD.20050901 while the GRIB files in MT.gfs_CY.00 are the latest and has >up-to-date reference date such as this one RD.20060108 ? (remarks: I >browsed the FTP server on Jan 08 2006) If the files in MT.gfs_CY.00_o >are not referenced as the current day, how come you still recommend >people to download GRIB files from MT.gfs_CY.{CC}_o on PCGRIDDS32 >home page? For example, you recommend >ftp://weather.noaa.gov/SL.us0080001/ST. ... rid_DF.gr1
Yea…I know. NCEP went & changed things on me after I posted that reference. It seems the MT.gfs_CY.{CC}_o directories have been abandoned in place and all data is now going to the MT.gfs_CY.{CC} directories. I have updated the PCGRIDDS32 Home page to reflect that & sorry for any confusion.
I guess the bottom line from my perspective is that it doesn’t really matter which model or GRIB file you choose. What appears to matter the most is:
1 - does the GRIB file cover the geographical are you wish to look at (& in your case, does it matter “where” the plane is flying…just as long as it is in the air?)
2 – does the GRIB file you are analyzing have sufficient horizontal and vertical data resolution and do the forecast hour time-steps progress in a small enough increment to resolve what you wish to diagnose.
Generally, the larger the areal coverage of the model, the more course the data resolution and forecast time steps are and the smaller the coverage, the finer the detail of data and time steps. The choice is yours…choose wisely (from Indiana Jones).
I hope all you questions were answered in a clear manner and I did not do too much to create more questions.
Good luck,
Jeff Krob
PCGRIDDS32 System Developer
PS - Thanks Kevin for the graphical help. Everyone seems to be better at that than me
OK – here we go…
>Jeff,
>I am doing a project which considers wind effects on aircrafts and the >aind data used are from GRIB files. I have planned to use the GRIB files >from NWS/NCEP FTP Servers rather than from Canadian Meteorological >Center (CMC). I now have 5 questions concerning the download of GRIB >files, and I hope you can answer them.
I’ll do my best
>1. On PCGRIDDS32 home page, you have listed several hyperlinks for >downloading under two headings, namely NWS GRIB and WAFS GRIB, >what is the difference between NWS GRIB and WAFS GRIB ?
The World Area Forecast System (WAFS) was designed to disseminate global GRIB information to the third-world regions via a point-to-point communications. The WAFS GRIB products are taken from the GFS model and the global coverage is broken up into 8 sectors or “octants”; 4 in the northern hemisphere & 4 in southern hemisphere. The octants are identified by the letters I-P. Also, the data resolution is “thinned” from 1 x 1 degree in the GFS native grids to 1.25 x 1.25 degrees for WAFS and there are fewer vertical layers and fewer parameters. All this is to reduce the volume of data to be transmitted. For more info on WAFS, go to:
http://www.weather.gov/tg/iscscvr.html
The reason PCGRIDDS32 treats the WAFS GRIB files differently is because the ingest routine for the WAFS files, GRIB2PCG32.EXE can “glue” the individual, adjacent octants together to cover a larger area for a single file.
The GFS native GRIB files are 1 x 1 degree, global coverage in one file with many more vertical layers & more parameters so the files are larger, but you get more data and sice there are no sectors which need to be joined with other sectors, that data goes in the NWS directory along with all other GRIB-type files for processing.
In my previous message to you I recommended you read up on the GRIB standard where the WAFS (and all other grid domains) are described in detail. I think you actually read the PCGRIDDS User Guide which shows “how” to process the data but does not actually describe the GRIB data. I thought I included the GRIB documentation along with everything else. The /PCGRIDDS32/DOC file “Sect1.pdf” explains the GRIB grid domains and, yes, I know, I really should rename them
>2. If my project involves consideration of wind effect on airplanes, am I >right to use WAFS GRIB because WAFS is a program developed by ICAO >and WMO to improve the quality and consistency of enroute guidance >provided for international aircraft operations?
Yes, the WAFS system was designed to do that but also with the limitation of less data to work with & required to transfer. However, like what Kevin showed in his graphic, the fh.HHHH_tl.press_gr.onedeg GRIB files contain much more data in more detailed forecast hours and for a longer forecast period but take longer to download. The trade off is yours to decide.
>3. The directory or file structures for the GRIB FTP servers are so >complicated for an amateur like me, can you tell me where I should >download GRIB files that are relevant to my project?
The link I provided in my previous post to the SOO/STC Web site should show what the individual GRIB files contain and what their file names are to know what to download.
>4. Suppose my aircraft is going to fly off at 3 UTC and I want to forecast >the wind effects on the aircraft at 3 UTC, am I right to go to the directory >SL.us0080001/ST.opnl/MT.gfs_CY.00 and go to >RD.{YYYYMMDD}/PT.grid_DF.gr1 and then download an octant file with >forecast hour equal to 003, for instance, fh.003_tl.press_ar.octanti ?
Not neccesarily – the fh.003 is signifying 3 hours into the forecast period from when the model was initialized. If you want 03 UTC, you would want the fh.003 file from the 00 UTC model run. The fh.003 file from the 12 UTC model run would equate to 15 UTC & so on.
>5. Why the GRIB files in the directory MT.gfs_CY.00_o are not the latest >and has reference date which is the past one, such as this one >RD.20050901 while the GRIB files in MT.gfs_CY.00 are the latest and has >up-to-date reference date such as this one RD.20060108 ? (remarks: I >browsed the FTP server on Jan 08 2006) If the files in MT.gfs_CY.00_o >are not referenced as the current day, how come you still recommend >people to download GRIB files from MT.gfs_CY.{CC}_o on PCGRIDDS32 >home page? For example, you recommend >ftp://weather.noaa.gov/SL.us0080001/ST. ... rid_DF.gr1
Yea…I know. NCEP went & changed things on me after I posted that reference. It seems the MT.gfs_CY.{CC}_o directories have been abandoned in place and all data is now going to the MT.gfs_CY.{CC} directories. I have updated the PCGRIDDS32 Home page to reflect that & sorry for any confusion.
I guess the bottom line from my perspective is that it doesn’t really matter which model or GRIB file you choose. What appears to matter the most is:
1 - does the GRIB file cover the geographical are you wish to look at (& in your case, does it matter “where” the plane is flying…just as long as it is in the air?)
2 – does the GRIB file you are analyzing have sufficient horizontal and vertical data resolution and do the forecast hour time-steps progress in a small enough increment to resolve what you wish to diagnose.
Generally, the larger the areal coverage of the model, the more course the data resolution and forecast time steps are and the smaller the coverage, the finer the detail of data and time steps. The choice is yours…choose wisely (from Indiana Jones).
I hope all you questions were answered in a clear manner and I did not do too much to create more questions.
Good luck,
Jeff Krob
PCGRIDDS32 System Developer
PS - Thanks Kevin for the graphical help. Everyone seems to be better at that than me
Jeff,
I have the 6th question and hope you don’t mind answering it.
6th Question:
At 14 UTC, I wanted to download GRIB files for 15 UTC and went to MT.gfs_CY.12 directory and found that fh.003 files had not been released yet. Later, I found that the fh.003 file was actually released at 15:40 UTC. So I want to ask, first, am I right to go to MT.gfs_CY.06 directory and download the fh.009 file as a substitute? Second, am I right to comment that if I could have downloaded the fh.003 file from the 12 UTC model run, the forecast data in the fh.003 file would have been more accurate than that in the fh.009 file from the 06 UTC model run because the fh.003 file data were forecasted from a shorter time ago?
I have the 6th question and hope you don’t mind answering it.
6th Question:
At 14 UTC, I wanted to download GRIB files for 15 UTC and went to MT.gfs_CY.12 directory and found that fh.003 files had not been released yet. Later, I found that the fh.003 file was actually released at 15:40 UTC. So I want to ask, first, am I right to go to MT.gfs_CY.06 directory and download the fh.009 file as a substitute? Second, am I right to comment that if I could have downloaded the fh.003 file from the 12 UTC model run, the forecast data in the fh.003 file would have been more accurate than that in the fh.009 file from the 06 UTC model run because the fh.003 file data were forecasted from a shorter time ago?
David,
If you go to the NCEP Central Operations Web page which show the model production status:
http://www.nco.ncep.noaa.gov/pmb/nwprod/prodstat/
You will see, for the GFS, actual model calculation does not start until around 1 hr. 15-20 min after the quoted initialization time and the forecast products are not complete for another 90 min. Since NCEP runs so many different models for different situations it figures it can start the medium range models (15 day forecasts) a little late in the cycle to let the shorter range models (1-3 day) start first. So, yes, if you require the 15 UTZ forecast time "at" 15 UTC, you may need to use the 06 UTC model run's 09 HR GRIB file. Or, if you are analysing flights over North America, you could use the 12 UTZ runs of the NAM or RUC models. They are completed earlier in the processing cycle.
Good luck,
Jeff
If you go to the NCEP Central Operations Web page which show the model production status:
http://www.nco.ncep.noaa.gov/pmb/nwprod/prodstat/
You will see, for the GFS, actual model calculation does not start until around 1 hr. 15-20 min after the quoted initialization time and the forecast products are not complete for another 90 min. Since NCEP runs so many different models for different situations it figures it can start the medium range models (15 day forecasts) a little late in the cycle to let the shorter range models (1-3 day) start first. So, yes, if you require the 15 UTZ forecast time "at" 15 UTC, you may need to use the 06 UTC model run's 09 HR GRIB file. Or, if you are analysing flights over North America, you could use the 12 UTZ runs of the NAM or RUC models. They are completed earlier in the processing cycle.
Good luck,
Jeff