Dennis Lettenmaier: Current Research, Land Surface Hydrology Group
HOME
PEOPLE
RESEARCH
PUBLICATIONS
MODELS
DATA
LINKS

Getting Started with Computing

Computing in the Hydrology Research group is done primarily on the UNIX operating system (FreeBSD). UNIX can be daunting to learn, but it is a powerful and flexible computing platform. To get started you might try reading O'Reilly's "Learning the UNIX Operating System." by Jerry Peek. It is available at the University bookstore.

There are a few low power computers with Microsoft Windows installed for preparing documents using Office software. Office software is also available under UNIX.

A listing of available computer systems is here.

 

Research Computing

The large majority of processor time and disk space is used for research compute jobs. It is important since all resources are shared that you understand how to do this efficiently and in such a way that others can also do their work.

Following are a couple of points to assist you:

1. CPU & MEMORY

Keep an eye on your processes including how much CPU and memory they are using up. The "top" command is useful for this. It displays the processes which are using the most system resources, and how much CPU and memory they are using. Keep in mind that "top" itself consumes resources and should only be used to check on your process, not run constantly.

Most systems are dual processors, so for these machines a steady load of 2 is okay. This means that on average 2 jobs are on the processors. When the load goes higher, a great deal of efficiency and compute cycles are lost to switching jobs on and off the processor.

2. DISK STORAGE

Keep an eye on your disk space usage. Given the large models that the group works with, it is quite easy to consume vast amounts of disk space. The "du" utility is very useful for this. It can give a disk usage summary for files and directories. It is best to use the -k flag always, as some versions of UNIX do not report usage in kilobytes. Backup tapes, a CD-write and DVD-write drive are all made available for archiving files and removing them from active disk space. A guideline for how to lay out your files and directories as well as backup and restore considerations is available here.

3. NETWORK

Systems are interconnected by 100Mbps (megabits/second) shared links. This works out to a theoretical maximum of about 13 Megabytes/second. In reality it is much less. We regularly deal with files from several Megabytes to more than a Gigabyte. This means: although it is possible, DO NOT RUN COMPUTE JOBS OVER THE NETWORK. All files for the process should be local to the machine. The capability of running jobs with remotely located files will be removed at some time in the future.

To access other machines on the network, we have a couple of methods available. The primary method is SSH Secure Shell which allows secure encrypted access to other machines. From a Windows machine you can log in to a UNIX machine, or you can transfer files using SSH Secure FTP. From one UNIX machine to another you can log in and transfer files using ssh and scp or sftp. We also have nfs automount between UNIX machines allowing you to easily copy files between machines. As mentioned above DO NOT use this service to run compute jobs over the network. Lastly files can be accessed through SAMBA or Windows shared disks. This is the least reliable, least secure and least recommended method.

4. SOFTWARE

We use a large variety of free UNIX software to do our work. There is also a limited amount of commercial software installed. In general, you can request that any free software package be installed and it will be installed quickly and available on all of our UNIX systems from our central server. Purchases of commercial software must be approved by Dennis and will take longer to aquire and install.

Other Computing

1. WINDOWS SYSTEMS

We have a limited number of Windows PCs available for shared use. As noted in the backups document do not keep ANY files on the Windows PCs that are of value to you. These systems are not backed up at all. These systems are a shared resource for working on papers and spreadsheets that require the Microsoft platform. Be aware that at certain times during the school year the systems will be very busy with people writing term papers etc. Please be considerate of others.

On the UNIX systems we have StarOffice available which has some compatibility with Microsoft Office formatted documents. The compatability is not always perfect, but it is useful if you need to create a document quickly that will be viewable on a Windows system.

2. SCANNING

We have a document feed flatbed scanner available. It can be used to convert papers into .pdf format. Instructions for using it are availabe here. It is attached to one of the general use Windows PCs, but priority should be given to people who need to use the scanner.

Computing Services

1. EMAIL

Our email server, mail.hydro.washington.edu is available for you to send and receive email. Your login (given when you arrived) is your email address. For example, to log in I use penglish. My email address is penglish@hydro.washington.edu

To read and send email, log into the system mail.hydro.washington.edu otherwise known as dynamic.hydro.washington.edu then run "pine."

Secure IMAP is also available (eg: for Netscape Mail), but is strongly recommended against due to the amount of trouble it has given us in the past. Pine is very simple and reliable by comparison..

Email can be used to transfer files, but it it is very inefficient method of doing so, and transferring large files could potentially disrupt the service for everyone. For this reason there is a 5 megabyte cap on email attachments.

2. FTP

For you
Ftp service is available to allow us to share files with the world including models, datasets, presentations and documents. Ftp also allows collaborators to upload files anonymously. This way they do not all have to have user accounts on our systems or have to email us large files.

To put a file on the ftp server, you can simply use SSH to securely copy it. Do:

scp filename.txt ftp.hydro.washington.edu:/home/ftp/pub/yourusername

This puts the file in your personal area. To put a file in a general area, so that it can be linked from the website and be persistant after you graduate, do

scp filename.txt ftp.hydro.washington.edu:/home/ftp/pub/HYDRO/data/datasetname OR

scp filename.txt ftp.hydro.washington.edu:/home/ftp/pub/HYDRO/model/modelname

To collect a file uploaded to the ftp server, copy it from the ftp incoming directory:

scp ftp.hydro.washington.edu:/home/ftp/incoming/filename.txt .

Note that this will require your collaborator to tell you what she called the file when it was uploaded.

For collaborators
For collaborators - instruct them to upload their files to our server. To do this have them log in to our server, ftp.hydro.washington.edu with the username anonymous and their email address as a password using an ftp client for example wu-ftp. Then have them change to the "incoming" directory and upload their file(s). Here is a copy of the instructions for Windows and UNIX that you can copy and paste into an email to your collaborators.

The files are uploaded to /home/ftp/incoming on ftp.hydro.washington.edu (currently ontario). You should be able to copy them from there, but due to security restrictions YOU CANNOT LIST THE DIRECTORY. This is why your collaborator must tell you the names of the files that they upload, per the instructions you send to them. When you are done, be sure to remove the files from the FTP server or request that Paul do it.

3. WWW

We have a www site for providing information about the hydrology group and links to our datasets etc. We do not use the www site to provide data downloads because the space available on the www server is much more limited than on the ftp server. To create a professional web page as part of the group, copy someone else's web page, edit it and then tell the System admin and he will link it in for you, leaving you with the permissions to edit it in the future.


University of Washington Hydrology Group
Wilson Ceramic Laboratory
Box 352700
University of Washington
Seattle, WA 98195-2700

hydro@hydro.washington.edu
ph. 206.685.1796