Disaster Recovery Is Not Just a Technology Issue

A basic disaster recovery plan for small business (and some larger ones) consists basically of redundancy in the server (RAID, redundant power) and a nightly backup. The theory is that server redundancy protects you most of the time and if that happens to fail, then you get your data from backup. We recently had a server failure just like that but with a twist.

A RAID 5 array failed in a server and took out information that was business critical. The failure occured on an Thursday evening. Reviewing the backup log, the backup Thursday night appeared to complete successfully, but in fact it hadn't. The RAID 5 array failed partway through the backup. This leaves us restoring to Wednesday nights backup. All of Thursday's data changes were lost.

Here is where the twist comes in. This organization scans in historical documents and then shreds them afterwards. After the document has been shredded there is no backup copy except on the computer system. This is one of the databases that was lost.

Because the document is shredded before a backup is taken, there is small window where loss of data is an issue due to only a single failure. To protect that data, shredding needs to be delayed at least one day to ensure there is a good backup. And depending on how paranoid you are, maybe two.

In this case, no documents were processed in the lost window, but it could easily have happened.