Optimizing Backups With Rsync
Recently, we changed some of our systems to use Linux rsync to backup to a removable disk, instead of to tape. We have been very pleased with the improvements. We have reduced the time to backup dramatically and improved the value of each backup.
We scripted it to use a simple versioning system, so there are multiple complete backups being stored. These are used in a round-robin schedule, with the oldest being updated every night.
Once a full backup is completed, rsync only backs up the changes that were made. The beauty of rsync is its delta algorithm for incremental backups. It bases what has changed since the backup that is being updated was changed, not since the last time rsync ran.
In a simple example, say you do a full backup to Backup-A. Then the next night, you do a full backup to Backup-B. The next night, you do a backup to Backup-A, again.
With many backup systems, the backup on the third night, to Backup-A, will only pick up the changes made since Backup-B was done. In other words, they only track changes since the last time that a backup was done. This means that Backup-A can be missing changes that are in Backup-B, but not changed since.
With rsync, on the third night, rsync determines what was changed since the original Backup-A was created, and saves all of those updates. Therefore, each backup is complete with all changes at the time the backup was made.
In our case, the initial full backups took about 55 minutes. Our tape backup was taking up to 3 times as long. When the round-robin scheduling comes back to these rsync backups, the updates are only taking about 5 minutes to complete. With the tape every backup was a new full cycle. If the rsync backups are needed to restore our systems we now have the improved speed of the external USB 3.0 drive over the tape transport.
We also put the external drive in an NAS enclosure and added RAID-0 for more complete protection of our backupd.
If you would like to have your Linux server configured to use rsync, contact us at CCS Retail Systems, Inc. support.
Dave.