 |
|||||||
|
|||||||
 |
Backup Information
Following is the disk backup policy as approved by Dennis. Please read and make sure that you understand the implications for your work. If you have any questions, feel free to contact Joanna.Our backups go back for 4 years with the NETWORKER-DLT backup system. As of September 2005, they are archived using the Bacula backup system. As we decided at the beginning, the tape management software recycles the backup catalogs after 1 year (catalogs grow by in GB per year). We can still retrieve data from tapes older than 1 year but it requires quite a bit more time/effort.
(1) Incremental: Recycle after 1 year Requires 1-2 x 100/250GB per month
(2) Differential Monthly: Recycle after 1 year Requires 1-2 x 100/250GB eight months per year
(3) Full 1st & 3rd Quarter: Keep for good. Requires 4 x 100/250GB twice yearly
(4) Differential 2nd & 4th Quarter: Keep for good. Requires 2 x 100/250GB twice yearly
(5) Bacula system (based on its capacity, and the time that it
takes to back up our amount of data) should be used to back up
- user personal files
- programs
- documents
- results from analysis of fully-processed data
All UNIX systems /usr1 and /home is backed up using Bacula. On the computing
cluster, /export is backed up, but not /export/raid which is a raid drive.
(6) Fully-processed data files should be backed up on DLT or AIT-3 as TAR files
and NOT as Bacula files (for reason given above regarding old
tapes). If possible, these data files should be kept on RAID
systems. If not, such directories should be excluded from the Bacula
backup -- let Joanna know if you have such directories..
(7) Raw input/output data files should be kept on RAIDs.
(8) The volume of data on RAIDs well exceeds the capacity of our AIT-3
drive, hence the RAIDs are not backed up. This is not an ideal
situation by any means. There are higher capacity tape-drive
systems (7 or 22 tape libraries) and it
may be worth considering.
(9) Shared computers running Microsoft Windows are not backed up. All of your files
should be stored under UNIX. If you need help mapping a UNIX drive under Windows
just ask.
If you need a DLT or AIT-3 tape (for data backup or distribution), let me know the volume of the data so that I can make sure that we have enough media.
How To Organize Your Data
In light of the disk backup policy, you may be wondering how to organize your data. Here is the suggested method:
1: jet:/home (up to 500MB controlled by quota)
Email
Documents (such as Windows documents, spreadsheets etc)
Uncompiled program source code
Scripts
2: /usr1 (up to drive_space/X where X is the number of people sharing /usr1, uncontrolled)
Overflow from (1)
Compiled programs, object files, libraries, archive files (.exe, .o, .a, .so, etc)
Fully processed results of output stored here before removal to tape, cd or publication
3: /raid (up to drive_space/X where X is the number of people sharing /raid, uncontrolled)
Raw input and output files
Output files prior to post-processing
If you have trouble structuring directories and symlinks, feel free to ask for help.
Use the "du" command to monitor your disk usage (hint du -sk will give you a summary in kilobytes). Use the "quota"
command to monitor your quota status. Manpages are available for both of these commands.
One Time Archive Information
As users leave and projects are completed, data are either published, put on the ftp site, or saved to tape for permanent storage. If a user leaves and they do not clean out their data it must be approved by Dennis to be archived, since other users may be depending on that data. Following is a list of directories archived for tracking purposes.
Directory Date Archived Archive Volume(tar) Archived As
electric:/home/jamesv 2/1/2002 Archive 001 (DLTIV) jamesv
electric:/home/tony " " tony
logic:/home/jamesv " " logic_jamesv
plane2:/opt/island " " island
plane2:/opt/esri/arcdoc_71 " " arcdoc_71
plane2:/opt/esri/arcexe_72 " " arcexe_72
plane2:/opt/esri/samples71 " " samples71
ephemeral:/home/jamesv " " ephemeral_jamesv
ephemeral:/home/tempes " " ephemeral_tempes
ephemeral:/home/tempmh " " ephemeral_tempmh
ephemeral:/home/tmp " " ephemeral_tmp
gen:/usr1/cherkaue 5/10/2004 Archive 003 (AIT3) gen/usr1/cherkaue
ontario:/usr1/cherkaue " " ontario/usr1/cherkaue
ontario:/home/tempcm " " ontario_backup/home/tempcm
ontario:/home/httpd " " ontario_backup/home/httpd
sphere:/raid/sansone 5/11/2004 " sphere/raid/sansone
sphere:/usr1/tempgd 5/14/2004 " sphere/usr1/tempgd
CHARON (retired Sun System) 5/23/2004 " CHARON/*
HYDRO4 (retired Sun system) 5/23/2004 " HYDRO4/*
METER (retired HP Unix system) 5/23/2004 " METER/*
NORWAY (retired Sun system) 5/23/2004 " NORWAY/*
ONTARIO (believed redundant temporary backup of ontario (freebsd web/ftp server), backed up
just in case) 5/23/2004 " ONTARIO/*
PLANE (retired Sun system) 5/23/2004 " PLANE/*
dynamic:/home/geraghti 5/15/2004 " dynamic/home/geraghti
ftp.ce:/pub/lettenma (old one-time backup of Dennis's ftp.ce directory. Backed up just in case
5/23/2004 " ftp.ce/lettenma
ontario:/home/ftp/pub/papers/wrs 5/23/2004 " ontario/home/ftp/pub/papers/wrs
FLOURIC:/raid/archive/LDAS
graph-raid-aww-ldas_crb-output 2/17/2006 archive1 (USB disk) full name
graph-raid-peterson-LDAS_output " " "
logic-raid-millard-LDAS-gulf_basin " " "
meteor-raid-aww-LDAS " " "
sphere-raid-jenny-ldas-LDAS_output " " "
sphere-raid-edm-LDAS_output " archive2 (USB disk) "
sphere-raid-niklas-colo-vic " " "
thermal-raid-edm-LDAS_output " " "
thermal-raid-millard-LDAS_ohio_out " " "
FLOURIC:/raid/archive
totape/HYDRO1 " " "
totape/HYDRO1/user/reiner " " "
totape/METER2/ " " "
totape/METER2/users/hamleaf " " "
totape/METER2/users/jamesv " " "
totape/METER2/users/kyle " " "
totape/METER2/users/lettenmaier " " "
totape/METER2/users/uhillard " " "
totape/METER2/users/yichu " " "
totape/Peterson/ " " "
totape/electric/usr1/LDAS " " "
totape/electric/usr1/davidb " " "
totape/electric/usr1/tempcm " " "
totape/electric/usr1/uhillard " " "
totape/gen/raid/kbergen " " "
totape/gen/usr1/archana " " "
totape/gen/usr1/kbergen " " "
totape/graph/raid/jslee " " "
totape/graph/usr1/jslee " " "
totape/graph/usr1/montana " " "
totape/logic/raid/pxs " " "
totape/logic/raid/rkbaum " " "
totape/logic/raid/vista " " "
totape/logic/raid/wenge " " "
dynamic/home/archana " archive3 (USB disk) "
dynamic/home/colleen " " "
dynamic/home/comanor " " "
dynamic/home/daniel " " "
dynamic/home/fmunoz (snapshot from Feb 06. restored to sphere/raid)
" " "
dynamic/home/gopi " " "
dynamic/home/jtpayne " " "
dynamic/home/kbergen " " "
dynamic/home/marketa " " "
dynamic/home/olivier " " "
dynamic/home/park " " "
dynamic/home/prodigy " " "
dynamic/home/sridhar " " "
dynamic/home/tempgd " " "
dynamic/home/uhillard " " "
ephemeral/usr1/jichen " " "
ephemeral/usr1/pgraham " " "
graph/raid/lxb " " "
graph/raid/marketa " " "
graph/raid/peterson " " "
graph/usr1/ " " "
thermal/raid/park " " "
thermal/usr1/park " " "
jhm-expire " " "
|
University
of Washington Hydrology Group hydro@hydro.washington.edu
|
