Surface Airways Station Data

This file discusses the processing programs, and data formats of the Surface Airways Data set.

Unless otherwise noted:

source code and makefiles are in /nfs/plane/usr1/arkansas/rawdata/source
executables are in /nfs/plane/usr1/arkansas/rawdata
scripts are in /nfs/plane/usr1/arkansas/rawdata, and should be edited for each processing task

File Contents

PreProcessing Format Conversions
Data Processing Programs
Utility Programs
- Converting from Binary to ASCII
- Plotting Time Series
Data Formats

PREPROCESSING FORMAT CONVERSIONS:

No preprocessing format conversions are necessary at this time.

PROCESSING SURFACE AIRWAYS STATION DATA

Padding Missing Data Records:

In order to successfully fill missing data, gaps in the raw data file must be filled with blank records. The program Pad.SAWD adds blank records to the input file, so that the output file is continuous. The current version of Pad.SAWD will read through a binary NCDC hourly export format data file, containing multiple stations with record types TMPD, WIND, PRES, TSKC, and RHUM, and produce separate files for each station, placing them into separate directories for future processing. Outfiles have the file name extension ".pad".

Filling Missing Data:

Once the data files are continuous, the program Estimate.SAWD is able to estimate the missing data using the nearest station, or previous values. This program operates on a number of files using an infofile.

This program reads files in the raw data format, and writes out files in filled binary format. Output files have the extension ".out".

Calculating Long and Short Wave Radiation:

Long and short wave radiation can be calculated on a site by site basis, by this program, or for gridded values using the interpolation program. This program processes all stations listed in an infofile, and requires access to the RHUM, TSKC, and TMPD data directories. It will create separate files for long wave (LWRD), and short wave (SWRD) radiation, and puts them into appropriate subdirectories.

At this time the long and short wave radiation is calculated when the gridded files are read into the model.

Interpolating Program:

The program interp, creates gridded data files of all of the surface airways data (RHUM,TSKC,TMPD,WIND,PRES). It will create a gridded data set at a user defined resolution.

The following files must be present in the directory from which you run interp:

information files for the 5 data types:
- info.press
- info.rhum
- info.tmpd
- info.tskc
- info.wind
ASCII data files for each data type, and each station, created using Read.SAWD.
A mask file, called "mask1".

UTILITY PROGRAMS

Converting from Binary to ASCII:

The output data from Estimate.SAWD program is in a binary format. This can be converted to the ASCII format used by interp by running the program Read.SAWD. This program also uses an information file to process a list of site files. Input files have the extension ".out", and output files have the extension ".asc".

Plotting a Time Series:

The program create_ts, creates a time series from a fixed binary file. The resulting ASCII data file can be plotted to check for stray values.

DATA FORMATS

EarthInfo Binary Export Files (Raw Data Files):

The following format is used by the EarthInfo surface airways for hourly binary data export:

Field Name	Description	Offset	Length	Type
state	P. O. Code	0	2	char
wban	WBAN #	2	4	long
name	Station Name	6	24	char
par_code	Parameter Code	30	4	char
month	Month number	34	2	char
day	Day	36	2	char
year	Year	38	4	char
hr_val	Hourly Values	42	4x24	long
hr_mflg	Hourly Measurements	138	24	char
hr_qflg	Hourly Quality flags	162	24	char
	Total bytes per record	186

Parameters normally used with the VIC hydrologic model:

Parameter	Name	Units
PRES	Station Pressure	inches of Hg
RHUM	Relative Humidity	percent
TMPD	Dry Bulb Temperature	degrees F
TSKC	Total Sky Cover	percent
WIND	Wind	360 degrees in knots

Information Files:

General Information Files:

These files contain position and elevation information for individual surface airways stations. All files listed in the infofile should be present in the run directory. The file format is:

nstations = x

startdate = month/day/year

enddate   = month/day/year

parameter = parameter-code


filename       latitude     longitude     elevation
---------------------------------------------------
infilename1    stat1.lat    stat1.lon     stat1.ele
     .              .            .              .
     .              .            .              .
infilenamex    statx.lat    statx.lon     statx.ele

Note: latitude and longitude are in decimal degrees, not degree.minutes, as provided by the CDROMs. North and east are positive values, south and west are negative (eg. 45 degrees 50 minutes West, is entered as -45). Elevations are in integer meters.

The possible parameter-codes are:

RHUM - relative humidity
WIND - wind speed
PRES - station pressure
TMPD - dry bulb temperature
TSKC - total sky and total opaque sky cover

Interpolation Information Files:

In order for the interpolation program to run correctly, there must be a special information file for each data type (RHUM,TMPD,TSKC,WIND,PRES), in the current directory. Each of these files should be in the following format.

Each line contains data for a different station
type		name			offset	size
----------------------------------------------------------------------
char		Filename		0	27
char		"station name:"		27	13
char		Station Name		40	20
float		latitude		60	7(.2)
float		longitude		67	8(.2)

Filled Data Files:

Filled Data Files: Binary

For each station a binary file with 'nhourly' values. The file will have the same name as the input file, with the extension '.out' added. The format of each file is the following:

First a 68 byte header, with:
Description            length           type
-------------------------------------------------------------------
state                     2             char
station name             24             char
wban code                 4             long
latitude                  4             float
longitude                 4             float
elevation                 2             short int
parameter                 4             char
units                     4             char
time interval             4             char
starting date of file:
  month                   2             short int
  day                     2             short int
  year                    2             short int
ending date of file:
  month                   2             short int
  day                     2             short int
  year                    2             short int
-------------------------------------------------------------------
Total                         62

Followed by 54 byte records with the following format:
Description            length           type
-------------------------------------------------------------------
date:
  month                   2             short int
  day                     2             short int
  year                    2             short int
I_PER_DAY times the following two values
nhourly value             4             float
quality flags             2             short int
-------------------------------------------------------------------
Total                 6+5*I_PER_DAY

If the quality flag <= nhourly, then it indicates how many values during the interval were averaged to calculate this value. QUALITY FLAGS 97, 98, and 99 ARE ADDED TO INDICATE A MISSING VALUE ESTIMATED BY THIS PROGRAM. A 97-flag indicates that the value is estimated using information from the same station, i.e. previous and next n-hourly time interval. A 98-flag indicates that the value is estimated using the nearest station for which this value is not missing. A 99-value means that the previous value is repeated.

Filled Data Files: ASCII

The ASCII filled data format is used by the interpolation program, and is extracted from the binary files using the program Read.SAWD. The file format is as follows:

Header:

state       : e.g.(TX)
station name: e.g.(LONGVIEW WSMO           )
wban        : e.g.(3951)
latitude    : e.g.(32.350  )
longitude   : e.g.(94.650  )
elevation   : e.g.(95    )
parameter   : e.g.(RHUM)
units       : e.g.(%   )
interval    : e.g.(1 HR)
start date  : e.g.(01/01/1972)
end date    : e.g.(12/31/1991)

Each day is a record of:

Description            length           type
-------------------------------------------------------------------
month			  2		char
"\"			  1		char
day			  2		char
"\"			  1		char
year			  4		char

24 Hourly Records per Day:

Description            length           type
-------------------------------------------------------------------
data flag		  4		int
data value		  8(.2)		float

Total Daily Record Length:	298

Gridded Data Files:

Each file contains the time step data for a grid box.

Each line is in the following format:

Description            length           type
-------------------------------------------------------------------
time step		  2		char
"/"			  1		char
month			  2		char
"\"			  1		char
day			  2		char
"\"			  1		char
year			  4		char
TMPD			  8(.2)		float
blank			  2		char
PRES			  8(.2)		float
blank			  2		char
WIND			  8(.2)		float
blank			  2		char
RHUM			  8(.2)		float
blank			  2		char
TSKC			  8(.2)		float
blank			  2		char

Return to Data Types Menu
Return to Work Links Page

Hydrology Homepage / University of Washington / hydro@hydro.washington.edu