1. Introduction

The HFS Backup service at Oxford uses TSM client-server software to provide an up-to-date and complete backup of the current local filestore of your machine. The following steps assume that you have already registered for the service and obtained, installed and configured the TSM client software on your computer.

The procedure starts with the making of an initial backup of all the files on the local machine (i.e. the client machine) and subsequently making regular backups reflecting any changes to the filestore. This initial backup must be performed manually and must complete before running automatic scheduled backups. Subsequent backups can also be performed manually but our recommendation is to use the automated scheduled backup facilities and to supplement these with manual backups when required.

There are some general considerations regarding Backup and Restore that apply to all TSM client users, irrespective of their client platform. Please therefore read the next three sections ( 2 - 4 ) before reading the platform specific pages on how to use the TSM software.

2. File versions & types of manual backup

On the HFS at Oxford, the TSM server is configured to keep up to two copies of each file; a copy of the latest or current version and a copy of the previous version (if it existed). If a third (or subsequent) copy is made of the file, at the next backup the oldest copy held is deleted, the current one held becomes the oldest and the very latest becomes the current copy. If the file is deleted on the client machine some rules come into play saying how long the two copies will be retained on the HFS server. This depends on whether the client is a desktop/laptop or server - please see our section on data retention. In TSM terminology, the backup copy of the current version of a file is called the active version; the backup of the previous version is called the inactive version.

There are up to four types of manual backup available, some depending on the client operating system; the screen shot below is for the Windows client but is similar to the TSM interface on other client platforms :

  • Incremental (complete) - also known as a full incremental backup. In a full incremental backup the server and client compare notes about which files are on the client and which backups are on the HFS to allow a full reconciliation to take place. The server passes full details of the client's filestore to the client and the client compares this with the current filestore; any new or changed files are then backed up. A file is considered to have changed if any of the following properties have changed: size, permissions, owner, modification date/time. The client tells the server which files have been deleted since the last backup; the server then marks the backup copies for deletion. They are not deleted immediately but become inactive and are retained for the period specified for desktop/laptop and server backups (see our section on data retention), whereafter they are deleted from the server and can no longer be restored. On NT, 2000 and XP machines this reconciliation is potentially different if the Journal Engine Service is running on the local partition. In this case the client records changes to files in a local journal database and uses this to determine which files are eligible for backup. This is useful on drive partitions with folders holding very large numbers of files, only a small percentage of which change. A full incremental backup is the normal mode for a weekly backup.
  • Incremental (date only) - also known as partial incremental backup. In a partial incremental backup the server passes to the client the date of the last backup of it's filestore partitions. The client then decides which files to backup - essentially all files with a modification date later than this. New files on the client which have a modification date earlier than the date of the last backup of the partition - for example recently installed software files - will not be backed-up by a partial incremental. Similarly, files deleted from the client since the last backup will not be marked inactive at the server. Note that a partial incremental only updates the last backup date for a partition if it completes a backup of the entire partition. If a partial incremental is performed on only part of a filesystem partition, the date of the last backup is not updated and the next partial incremental will back up these files again. An incremental by date backup is much quicker than a full incremental and is the recommended mode for extra backups carried out between the weekly full incremental backup.
  • Incremental (without journal). This only applies to NT, 2000 and XP clients with the Journal Engine Service switched on. In this case the client performs a standard full incremental backup as above, and bypasses the journal database. This is useful to resync the journal and the server when, for example, the journal is corrupt or the Journal Engine Service is stopped or started for any reason.
  • Always backup - also known as selective backup. In a selective backup the client backs up the files and directories chosen, unconditionally. This means that all file(s) will be backed up irrespective of whether they have changed since the last backup. This can potentially be very useful if you want to be sure that you have a backup of a particular set of files that you are working on. The disadvantage is that by running an 'Always' backup on a file which is already backed up is that it results in holding two identical versions of that file. In contrast, full and partial incremental backups only backup a file again if it has changed, thereby ensuring that the active and inactive versions are different. This is not a backup mode that we would unreservedly recommend.

3. The initial backup

Your initial backup of a machine newly registered to the HFS should be all of the following:

  • A manual backup not a scheduled backup.
  • A backup of local disk drives and partitions only, not of network drives or of removable - floppy, cdrom - drives.
  • A backup of less than the daily limit (100GB for desktops/laptops; 200GB for servers; 300GB for large servers).

Your initial manual backup will essentially be a full backup of your entire local filestore. With disk capacity in a typical personal computer ranging from 10GB to 200GB and with those in departmental and college servers often being even larger, this can mean that your initial backup requirements are very large. This is particularly so if you have installed a large amount of software and/or data onto the machine prior to this backup. Network transfer rates vary: approximate maximum speeds are 30GB/hour on a 100Mb network and 300GB/hour on a 1Gb one, but such a speed can be reduced greatly by other factors such as other traffic on your local network and the speed of your machine's disk(s). Please try to estimate and allow sufficient time for this initial backup to complete while you are in attendance.

By default TSM will back up all the files on your machine, though there are a few exceptions - for these please see the HFS Policy Pages.

If the total amount of data does not exceed 100GB (for desktops/laptops), 200GB (for servers) or 300GB (for large servers), please go right ahead and start the first backup.

If the total amount of data to be backed up will exceed the daily limit then it will need to be staged (i.e. spread over a number of days). This is not difficult but some consideration needs to be given as to how best to proceed. If you are unfamiliar with the HFS backup service, it may be best to contact us at hfs@ox.ac.uk in order to discuss the initial backup. There are some examples shown below for guidance:

  • If you have several (small) disk partitions then backup one (or a few) partitions each day - as illustrated in the section titled 'Backing up your local disks' in the guide below appropriate for your client platform.
  • If you have one or more very large disk drives, then the initial backup can be split by selecting individual folders for backup. This is illustrated in the section titles 'Backing up selected files & directories' in the guide appropriate to your client platform.
  • If you have one or more very large drives and are unable to manually split the backup into separate 'chunks', just backup the entire disk. The account will be locked when it exceeds the daily limit (see above) and may be unlocked as detailed in the paragraph below. Repeating this process will eventually backup the entire disk - that is, the backups start from where they were locked, they do not restart from the beginning.

In cases where the initial backup of a single large partition is being done in stages, it is very important that a manual full incremental backup is run over the entire partition before including this partition in the scheduled backup. That is, once the partition has been backed-up in stages, it should then be manually backed-up in it's entirety. This ensures that the backup has a completion date which is then used for processing of subsequent backups.

We have procedures in place that detect when a client has sent data in excess of the daily limit in a working day. The backup session will automatically be cancelled and the client account locked. Newly registered clients (those registered within the last 14 days) will be automatically unlocked to allow completion of the initial backup; this is done at 10:00 am the next day (2 pm for those clients on the server service). This is effective for the first 14 days after registering the account with the HFS service. When the next backup session is started, the backup carries on from the point at which it was stopped. The whole process usually takes just a few days and once completed, subsequent sessions will only backup files which are new or have changed since the last time. After the first 14 days since registration the lockout is effective until you contact us.

4. Restoring your data - considerations

4.1. Stop any backups while restoring.

If you are restoring any sizeable amount of data, you should disable any backups (scheduled or manual) from running until at least the restore has completed. Running a restore in conjunction with a backup will cause the two processes to slow considerably as they contend for your local machine's cpu and I/O bandwidth. Halting the backups also avoids the possible confusion of restored files being backed up and overwriting a previous version of a file backup. Once complete you should examine the files restored before reenabling any scheduled backups again.

Clearly certain circumstances which precede a large partition or system restoration cannot be planned. However, if you are planning a system or hardware upgrade which will leave your data at risk, there are methods by which we can minimise the number of tape volumes upon which the data is held. Please contact us in advance of such an operation at hfs@ox.ac.uk giving as much notice as possible of the planned upgrade and we will do what we can to help. Similarly, please let us know if you suffer a major disk failure and while you are waiting for a replacement, we may be able to help reduce the time needed for restore.

4.2. It may be quicker to restore from local software installation media

If you have the installation media for your software applications to hand, then it may well be quicker to re-install from these than restore from the HFS.

4.3. "I can't see old and/or deleted versions of a file in the restore window !"

You have just stumbled upon the difference between active and inactive files. An active file is a file for which a backup exists and the file currently resides on the local machine. An inactive file is a file for which a backup exists but which is no longer resident on the local machine. If you have overwritten or deleted a file or folder, and subsequently run a full incremental backup, TSM will mark the file or folder as inactive. At a subsequent restore, TSM displays by default, only active files in the Restore window. To display and restore your inactive files and folders using the TSM GUI, open the Restore window, select the View menu and check the [Display active/inactive files] option. Using the Command Line Interface add the -inactive option to your query. Please note that inactive objects are only kept on the TSM server - and hence can only be displayed - for a certain length of time after becoming inactive according to the backup policy. This policy information is available in our section on data retention.

4.4. Restoring complete partitions or filesystems

Restoring a complete data partition, with the exception of the 'root' or system partition, is as simple as restoring a single file or folder. Please see the sections headed 'Restoring Local Partitions' in the appropriate interface guide in the next section.

Restoration of the complete 'system' partition - i.e. the root partition on Linux machines, or the C: partition on Windows holds with it a number of complicating factors, not least the number of open system files that can or cannot be overwritten. Specific procedures need to be adopted for such a process to be successful. Necessarily, such procedures will be operating system (OS) platform specific - and will vary between OS releases. If you find yourself in a position of having to restore a system partition please contact us at hfs@ox.ac.uk for more help and advice.

It should be understood that the HFS/TSM does not support a full network re-install capability and that a combination of re-installation of the OS, basic network configuration, followed by installation and configuration of the TSM client will be required before restoring any necessary files from the HFS. Another consideration is that the hardware base may change following a major problem and/or the OS requirements may change. For all of these reasons we would recommend contacting us in these circumstances.

5. Using TSM Backup & Restore on your client platform

For simplicity, usage of the TSM backup client software is grouped according to interface type and function. Please note that for Netware users the sole interface is the Command Line Client (CLI), there is no GUI interface available. Please also note that the GUI may be very slow when trying to display the contents of a very large and/or deeply-nested directory structure. In such circumstances the Command Line Interface may be quicker and more convenient to use. Each link below gives an introduction how to Backup and Restore files using that interface: