MADIS - selecting a smaller set of data

Wally Mayo · Post by **Wally Mayo** » Sat Dec 09, 2006 2:59 pm

For those of you who like or use the MADIS set of data, here is a way to really save time and file size factors.
Who needs the whole HUGE data set at many megabytes?

If you want just one state, or a box defined by lat/long, then go to
http://www.madis-fsl.org/public/sfcdump.html
Choose your parameters, then SUBMIT. You will get a password/signon, just hit cancel ... it will say Authorization required, but now copy the URL in the address bar at the top of your browser. Then go to FUSION, choose Internet, Retrieve data, and ADD a site, putting the URL in appropriately, naming what you want, such as "MADIS for KY," or "MADIS for local area."

This is wonderful, for you get a nice small file, much quicker, and MUCH quicker to ingest!

Also, it allows you to choose the data set to plot. I choose to NOT use the NWS data, since it is already plotted in another step in map creation, and I don't want the doubling of plots, which is messy.

Here is a map doing this, and it's all there.
http://www.weatherroanoke.com/area.html

Now, someone may be able to figure out the full public access of data from
http://www.madis-fsl.org/
without the password stuff, but it doesn't matter for this purpose.

Wally Mayo · Post by **Wally Mayo** » Sun Dec 17, 2006 2:08 am

Well, it isn't EXACTLY as simple as the above, but it is still possible, IF you change the header or first part right before "recwin" to

http://www.madis-fsl.org/cgi-bin/simple ... &minfwd=0&

Leave the rest as you got it above.

jaytee · Post by **jaytee** » Wed Dec 20, 2006 4:48 pm

I can get logged onto MADIS and get a text file with lots of OBs when I choose 20060102_1200 for GA, but nothing seems to really import into Fusion when I put that url into add site and try to retreive.

Any clue what I might be doing wrong?

Wally Mayo · Post by **Wally Mayo** » Wed Dec 20, 2006 5:55 pm

Ok, what is the URL line you get? I'll check it over.

It should look something like this for GA:
http://www.madis-fsl.org/cgi-bin/simple ... el=1&xml=0

But note, I have had to use "RETRIEVE DATA" from internet to get the data to disk, or plot directly. So far, I have not been able to get it to work in a script. Next, I'm going to try to use the incoming.asc to import in a script.

Weasel · Post by **Weasel** » Thu Dec 21, 2006 4:48 am

Are you sure that last link doesn't work all on it's own?

I just clicked on it in firefox and it brought up a couple hundred obs, so it must be a subset:)

Wally Mayo · Post by **Wally Mayo** » Thu Dec 21, 2006 2:13 pm

Ooops, I didn't narrow the data set down. I like to create the URL so it does NOT get the regular METARs from NOAA. There is a reason for this. If you use the MADIS for everything, incl. NOAA, then you won't get the complete data like clouds, pressure, etc. So, I import the NOAA metars like normal, then go get the MADIS data with the data set excluding NOAA metars, or you will get DOUBLE plots, and a mess.

I should've put this URL in the above post.
http://www.madis-fsl.org/cgi-bin/simple ... d=WXforYou

I don't know why it gets it 4 times, but it works. Even so, the file for just the area is GREATLY smaller than the entire MADIS dataset.

Weasel · Post by **Weasel** » Sat Dec 23, 2006 9:18 pm

I don't think it is grabbing it four times. Each section amounts to a different weather element observation. Wind, temperatures, etc.

Thanks for dissecting the link for us, Wally!

Pat