One of my first tasks when I was hired ten years ago was to investigate the creation of a disaster recovery site for our mainframe computer systems. I had already had some experiences with disasters and recovery. Here are some examples of a few that we included in our plans.

Major Earthquake – Those of us who live in California understand earthquakes. I’ve personally been through at least five significant quakes (6.8 or greater) without suffering any damage at all. In many people’s mind, a major earthquake is the disaster scenario.

One day the “big one” will come (in California) and who knows what will happen at that time. In fact, my boss and I were able to convince the CEO of our company to create a “hot site” (a duplicate site which is already ready to take over in the event of a disaster) because of a recent significant earthquake.

One of the first things that we did is contact Caltech (the experts on earthquakes) to commission a study to determine where we should place our disaster site. The primary criteria was that the site be relatively close (within 50 miles) but on a different geologic plate so the earthquake would not flatten both locations.

As we studied the possibility of this disaster, we realized that the building and computers might emerge from the earthquake entirely intact, but the infrastructure (power, phone lines and so forth) might be destroyed. In addition, a major earthquake is a unique disaster because it’s more likely that your people will be in complete shock and more interested in their families and homes than in restoring your computer operations.

The thing to do here is be sure you’ve got the infrastructure issues covered cold. This includes phones, power and the network. Make sure you have a disaster site (or very good backups kept off-site) ready to go. Rehearse your disaster plan, and make sure your people know what to do.

Minor Earthquake – A minor earthquake might be easy to survive (we’ve been through several of them with no issues) and it might introduce some interesting quirks on its own. The power might be out, phone lines might be down and take weeks to repair, and the general infrastructure (roads, food shipments and so on) might be disabled. In addition, earthquakes tend to put people into a state of shock, so it might be difficult to get people to recover and get back to work.

Biological Event – When envelopes fill of anthrax started appearing on the news, we were suddenly confronted with a new type of disaster. What if a biological attack or event occurred in our building? What if the receptionist opened an envelope contaminated with anthrax? We would then be confronted with a unique situation. The building would be sealed and off-limited for days, weeks or even months; and we would not be allowed back in under any conditions for any reason for that time.

This situation is probably one of the worst disaster scenarios of all. The old building and equipment is intact but completely unreachable. Tantalizing and frustrating. Sigh. What to do?

Hope and pray that you’ve got an excellent disaster recovery plan and a very competent team, that’s what you do. In this case, you’d better have a hot site or, at the very least some excellent backups. Not only that, you’re business continuity plan had better be totally finished and rehearsed. You see, in this case you will not even be able to look through the ruble or burned building for papers, disks, CDs or anything else. Nothing, absolutely nothing, will be available for your use for a long, long time. In fact, if any of your people are in the building you may find them unavailable (as in quarantined) as well.

Lightning – One day a few years ago, when I was just beginning in the computer field, I was working late, way past midnight. I was just sitting in the computer room (freezing my butt off), listening to one of the heaviest rain storms that I’d heard in a long time. It was kind of eerie, listening to the rain drops on the roof and the thunder in the distance.

Suddenly, the room lit up and I was blinded for a few seconds. I instinctively pulled back from the keyboard, and I believe that saved my life. I felt heat on my face and body, and when I opened my eyes the computer room was dark.

I soon learned that lightning has struck the power pole just outside the building. The computer simply melted down – no circuit breaker in existence could have protected it. I was lucky to be alive – wow, what a rush that was!

Unfortunately, this company did NOT have a disaster recovery plan. We had to purchase a new computer system and build it back up from scratch. Fortunately, we did keep backups off-site, and within a week or so we were back up and running.

And that’s the reason why, now, I have a complete belief and focus on disaster recovery – if you have a plan and have prepared well, you will recover just fine (even with the unexpected bumps and such). If you don’t, then you basically leave it up to fate or whatever else you believe in. Personally, I would rather be in control of the situation. I find that makes things much easier.

richardlowejr