Global Surface Archives

Global Surface Archives is the single largest collection of hourly and special METAR and SYNOP weather observations in existence. Using any off-the-shelf viewer like Digital Atmosphere, you can plot historical charts for anywhere in the world for any hour, even with ship reports (after 1973).

You don't get complex XML or JSON records, or proprietary binary files. All data is in plaintext (SYNOP, METAR, and CSV) and is designed for maximum portability. These standard formats are familiar to almost all meteorologists and can be plotted with a variety of off-the-shelf software tools or simply opened in any text editor (e.g. Notepad).

For detailed historical maps, researchers traditionally had to rely on the NCEP Reanalysis series, which is a binary dataset requiring specialized viewers and is mostly limited to 250 km resolution. These are not always adequate for case studies or mesoscale reconstructions. In Global Surface Archives, you have all of the actual observations and can build the most demanding mesoscale analyses for years gone by, whether you're looking at 2012's Hurricane Sandy or the 1984 Ivanovo tornado in the former USSR.

Global Surface Archives represents so much processing work that it took 1 month of continuous processing time to complete our original 2010 release on an Intel Core i7 machine.

Details

■ Current period of record: January 1, 1930 to 2024 and then every six months onward. Latest data on request.
■ Available formats: METAR and SYNOP ordered by date-time. Volumes 2 and 3 contain data sorted by station in SYNOP, METAR, and tabular (CSV) format.
■ Displayable as maps?: Yes - Volume 1 can be plotted with Digital Atmosphere, GEMPAK, or other display software.
■ Displayable as text?: Yes - Volumes 1 and 2 are in text format.
■ Displayable as a spreadsheet?: Yes - Volume 3 is in CSV format (Excel, OpenOffice, etc). One file per station year.
■ Operating systems: All, including Windows and Linux. This is a data package.
■ Media: This product is delivered on a portable USB external hard drive (normally 500 GB) that is yours to keep.
■ Size: ~150 GB compressed (apprx 1 terabyte if fully uncompressed)
■ Author: Developed and produced by Tim Vasquez / Weather Graphics

Global Upper Archives (radiosonde data)

Add the new Global Upper Archives dataset! Click here for more information.

Information on Lifetime Updates

If you have Lifetime Updates or are thinking about purchasing it, please read this important information:

When will there be an update?
Updates are every six months: in January and July.

What's the procedure for Lifetime Updates?
Updates are placed on our website for download. The link will be sent to you when you place the order. These are new files that you can unzip and use as-is or merge with your existing product.

Can I get a special update to make all the data current through today?
Yes. If you want the data brought current through the time of purchase, this is available for a $95 service charge either at the time of purchase or any date after your purchase. You do not have to have a Lifetime Updates package to request this. Click here to place this in your cart. Allow anywhere from 2 to 7 days to build the data.

What if I don't want to download the Lifetime Updates patches? What if I want them on a USB drive?
If you prefer physical media or can't download the updates, we will be glad to provide a fully updated Global Archives package on a USB drive at your expense for the actual costs of shipping and the drive. Please contact us to work out an arrangement.

Terms and conditions.
This program is intended to allow customers to receive updates to Global Archives, providing we are continuing to maintain it, at no cost except for the actual costs incurred by Weather Graphics, typically raw materials and postage. The guarantee provided to you is to provide you a method of getting updates as we continue to sell that product without having to re-purchase it. Terms and conditions are subject to change.

Technical aspects of Global Surface Archives

Contents

Global Surface Archives is a ultra-massive collection of worldwide "hourly" and "synoptic" weather observations in two of the most universal formats for weather data distribution. The size of this dataset is approximately 109.5 GB (~500 GB uncompressed).

Documentation

Refer to the documentation (590 KB, PDF) for details about the Volume 3 comma-delimited data and for other information about this dataset.

Identifiers

METAR data is indexed according to standard ICAO codes. For all observations before 2010, identifiers have been corrected to the locations used in 2010, for example, a 1975 observation from Hanoi Gia Lam, which was transmitted as VWHN during the North Vietnamese regime, now appears as VVGL, same as the current identifier. We've taken care of all the hard work for you. SYNOP data is indexed according to standard 5-digit World Meteorological Organization station numbers; since these do not change they are provided in their original assignment as received.

Period of record and coverage

Datasets start in 1930, though a few records exist for Finland and France going back to 1920 and were included. The data is sufficiently dense to allow synoptic maps of the US in the late 1930s and for parts of the US and Europe in the 1940s. It may be impractical to construct satisfactory maps of various areas until the 1940s and 1950s.

Also, much of the older data is reassembled from "packed" records. Before the 1980s mass storage space was extremely expensive and archiving data in original form was considered wasteful. All METAR data is also standardized to the post-1996 standard, which is generally considered to be a bonus but does cause some degradation of sky condition data, which was encoded in a different system before then (e.g. 7CI250 vs. 250 BKN). There is a known reduction in observations between 1969 and 1972 due to data losses at the telecommunications centers and results in data not being available in some countries; the data loss episode peaked at 1972 before recovering rapidly in 1973. Observations subject to WMO Resolution 40 restrictions (generally 3-hour SYNOP from certain countries) are not included.

Inventory

A basic inventory of the dataset is provided here. These ZIPs contain a large text file that lists every file in the set along with its size. THE FOLLOWING MAY NOT APPLY TO NEWER RELEASES; this update was prepared in January 2012 and has not been updated, and is useful mostly for pre-2012 data.
lib001.zip - Volume 1 (METAR/SYNOP by date/time)
lib002.zip - Volume 2 (METAR/SYNOP by station)
lib003.zip - Volume 3 (tables by station)

Archive volumes

This dataset is made up of three different volumes. Different volumes contain the exact same information as other volumes, but are sorted or formatted different ways.

Volume 3. Observations are in tabular (CSV) format. Each file contains data for a unique station. The CSV format is decoded and can be easily opened in Excel or OpenOffice. The station files are also split into years, since the row count would otherwise overload most spreadsheet programs. A filename might appear as RJTT-1997.csv, which would contain all observations for Haneda Airport in Tokyo for 1997. Volume 3 contains land stations only (no ships). Click here for a sample of Volume 3 data.

Maps plotted with Global Surface Archives

All of these map samples were generated with Digital Atmosphere. You can use other software such as GEMPAK and MetPy that handles METAR data.

An incredible cold wave in South America (17 July 2010, 2000 Brasilia Time -1) (click for full version). An amazing example of a cold outbreak making it to within 400 miles of the equator (see west side of Amazon basin). The town of Tarauaca reported 13°C (56°F) just after dusk. Rio Branco at 10°S reported a daytime high of 14°C (57°F), while Corumba at 19°S never rose above 9°C (48°F). This map is in degrees Celsius; American readers can see temperatures in Fahrenheit by clicking here.

The January 1987 cold wave hits Great Britain (12 January 1987, 1800 UTC) (click for full version). Winter sweeps westward across much of central and western Europe, producing what was arguably some of the coldest weather since the 19th century. Evening temperatures as seen here were well below freezing. The daytime maximum was -9.1°C (16°F) in the southern suburb of London of Warlingham, and the island of Jersey off England's southern coast recorded a high of -6°C (20°F)! This map is in degrees Celsius; American readers can see temperatures in Fahrenheit by clicking here.

The Chernobyl disaster is depicted on this map valid 1200 UTC on 26 April 1986 (click for full version). The dataset shows that all of Ukraine under the influence of a 1034 mb anticyclone over the White Sea, which was producing easterly wind components across far western Russia, Ukraine, and Belarus. This pattern was partly responsible for advecting some of the contamination northwestward into the Baltic region, contributing to widespread alarm throughout western Europe.

A record heat wave in England (28 June 1976, 1800 UTC) (click for large version) which brought temperatures well above 30°C throughout much of England. Temperatures reached 35.6°C (96.1°F) in Southampton, the highest June temperature recorded in the UK. Unprecedented drought conditions led Parliament to pass the Drought Act, giving local authorities the power to control the use of water in their jurisdictions. The northeasterly flow into Great Britain is somewhat ironic since it is this pattern which, in the winter, is associated with cold waves. However, in this instance the trajectory off the European continent is bringing warm, dry air rather than cold, dry air.

The lowest temperature to ever occur on Earth (21 July 1983, 0000 UTC) (click for large version) occurred at Vostok, Antarctica. There's not much to see on this map because the station density in Antarctica has always been historically sparse, but there it is. Vostok had started out at -60°C (-76°F) on July 7th, then two weeks of clear nights in the "perpetual night" of the Antarctic winter allowed heat to radiate away before the site finally reached -89.2°C (-128.5°F) as shown here. That's a 16-knot wind. We tried to calculate wind chill with the NWS/SRH wind chill calculator, but at those readings it just says "Undefined".

One of the coldest days on record in India (9 January 2011, 1730 Indian Standard Time) (click for large version). Widespread, persistent fog in the northern valley regions of India led to record low temperatures. Delhi's maximum on this day was 11 degrees Celsius, the coldest on record since 1969. This map is in degrees Celsius; American readers can see temperatures in Fahrenheit by clicking here.

---