10 things you need in your DR plan

 By Harvey Betan

There are many items that are required in a Disaster Recovery (D/R) Plan, some are location specific but some are more generic. This article will touch on items critical for the success of the disaster recovery plan. Organizations may differ on the priority or of these items, but they must exist within the DR plan.

The most critical issue in the Plan is that it is current, and that a copy exits at the recovery site. You do not want to go through a recovery process with an outdated plan. The plan should detail the latest hardware and software configurations. Also the plan must be clearly laid out and easy to follow. A plan should not rival “War and Peace” in length or complexity. It should be concise with easy to follow bullet points. Though plans are written by individuals who anticipate their presence during the recovery, this cannot always be guaranteed. There may be issues that do not allow the plan’s author to be present during the recovery. Plans should be written so that someone with a similar skill level can accurately follow all steps in the plan. Omit any shorthand or technical jargon from instructions. It is important to realize that DR plans will probably be exercised under extreme conditions of stress and timing. When reviewing and editing the plan ask yourself “If I am trying to go through this at 2 AM after a full work day will I be able to understand what needs to be done?”

It is imperative that any DR plan have an accurate communication or call list. Some organizations utilize third party vendor products for this, regardless, the list must be current. The list should have designated backups for each key individual and multiple contact information for them as well. If you are using a call tree, make sure that you have a loop back so that the last person on the list will confirm that the call was made.  Someone should be designated as the communication list manager to monitor responses and contact backup staff as necessary.

You must have and be able to work off a detailed script.  There is no guarantee that the script will be followed by the same person who wrote it. For this and other reasons the script should have a formal review process by other members of the team. For best results use bullet points and easy to follow steps. When in a real recovery scenario, there will be intense pressure and many things going on at the same time. Add to this the possibility at it is to be executed at 2 AM the likelihood of confusion is high. Whatever can be done in the plan to make the steps easier to follow will go a long way.  If at all possible, try to anticipate errors and include remediation steps. For example: “Connect network cable to PC.” If network not found, first check if PC Ethernet connection shows signal. Something as simple as that, when you are under extreme stress and fatigue goes a long way. Within the script keep things simple. Avoid using terms that may not be understood when extreme fatigue hits. If you must use technical terms that may not be understood, add a glossary of terms.   

Once you have a detailed plan, you need to test and retest the plan. It is possible to test separate portions of the plan on their own, but make sure the whole plan is tested at least once a year OR if a major change takes place. Exercising or testing the plan often will make staff familiar with the plan. Try to exercise the plan with different staff members if you can. DO NOT assume anything in the plan, not even that everyone can make it to the recovery location at the correct time.  When testing the plan try to anticipate unusual conditions. What would the team do if a drive or a technical component fails?

Each member of the team should be familiar with their defined role. Additionally the backup members must be familiar with their roles. If a team member whose primary role is applications has a backup role as a telecommunications resource, make sure they know what that role entails.

This next item may sound odd as a “Must Have” but believe me it is of utmost importance sometime. Have a list of restaurants and 24 hour supply delivery resources at the recovery site. You will likely spend many hours at a recovery site and will need to replenish supplies. You do not want to start searching for places when every minute and resource counts. You may very well need to be at the recovery site for longer than 24 hours at a time.

Part of the Disaster Recovery plan must include an application list. Each entry in the list should have the application name as the technical staff identifies it, the name the business side recognizes and any technical details such as server name etc. Along with the technical items, include the application owner, their full contact information and backup contacts.

There should be an accurate, that is, up to date, network diagram of the entire network as well as the recovery site. Each node on the switch and panels should have some means of identification. This will ease any issues with following cables and wires.  

The DR Plan should contain an easy to follow map and directions of how to reach the recovery area. Secondary directions could be included as well, if the main route is congested or impassable for whatever reason. If there may be an issue with parking, you should specify additional parking facilities in the vicinity.

Additional documentation that is helpful within a DR plan is a list of vendor contacts and any applicable insurance documentation such as policy numbers. Another item that is helpful is a list of all the hardware and software licenses you may have.

Last week’s Quiz Question

What was the only major Revolutionary War Battle fought on Vermont soil? 

Answer: The Battle of Hubbardton (http://en.wikipedia.org/wiki/Battle_of_Hubbardton)

Winner: Andrey N. Chernyaev

This week’s Quiz Question

The first guiding value of Norwich University is that “We are men and women of honor and integrity. We shall not tolerate those who ________________.

Current competition standings:

Andrey N. Chernyaev:  5 wins Matt Bambrick: 3 wins

Dianne Tarpy: 2 wins

Sam Moore

Autumn Crossett

Gil Varney, Jr.

Glen Calvo

Thomas Reardon

Sherryl Fraser

Srinivas Chandrasekar

Marc Ariano

Linda Rosa

Joanna D'Aquanni

Bill Lampe

Srinivas Bedre Christian Sandy Joseph Puchalski Scott Madden Ken Desforges

Is the Future of Business Continuity Broad or Deep?

John Orlando

From DRJ—Summer 2010

Nat Forbes created quite a stir recently with a thought-provoking blog post at calamityprevention.com called “Is the BCM Profession a Dead-End?” He begins with a dark but somewhat funny characterization of the business continuity professional as a lonely, underappreciated, underfunded, corporate pariah begging for scraps at the edge of the organizational hierarchy. Worse yet, the professional faces no discernable career track, and no light at the end of the tunnel.

According to Forbes, the problem is that the BC professional cannot convince upper management of his value because his job is to prepare for the unlikely horrific disaster. As he puts it, “It’s just difficult to chart an upward career path for a guy whose job is to plan for the Apocalypse if asteroids never smash into the planet.”

But Forbes suggests a way out of this dead-end road. He urges the BC professional -- and by implication the profession -- to expand his duties into neighboring fields from risk management (including financial risk management) to corporate security, disaster recovery, human resources, facilities, and corporate communications. His reasoning is that the corporate world already recognizes the value of these periphery fields. If the BC professional is to establish his own value, he should adopt these duties as his own.

This is not a new suggestion. A number of commentators have predicted, or encouraged, the convergence of business continuity with fields that it intersects. For instance, ASIS International has talked about the convergence of business continuity with information security and physical security.

The Answer?
While convergence between these fields makes sense on a rhetorical level, does it make sense on an organizational level? Yes, the umbrella term “risk management” can be divided into the subfields of financial risk and physical risk, but do those two fields really share much in common beyond the similar title? Financial risk management seems very different from physical risk management. Are the duties and skill sets of the CFO who watches the various markets to manage risk and determine where best place to park an organization’s assets really so similar to those of the business continuity folks who prepare to relocate functions in the event of a disaster? The former is an accounting function, while the latter requires a variety of skills relating to organizing people and systems.

No doubt the business continuity professional is well-served to understand periphery fields such as physical security and the like. It is always helpful to understand the duties of the people with whom you must work to do your own job. In that sense, it is helpful for the business continuity professional to understand the functions of all jobs within an organization, since she could be called upon to find ways to reproduce those jobs at another location. We study the fields of risk management, IT continuity, and others because that knowledge is an important part of a business continuity professional’s job.

Understanding another’s job is different from actually doing it, or as Forbes says, “to merge responsibilities” with another job. Corporate structure is determined by a lot of factors, both good and bad, and working with another does not imply the need to merge with their function. Forbes defends the merging of functions on grounds that business continuity intersects with other parts of the organization. But you can say the same about a lot of functions with an organization, such as accounting. All parts of the organization are affected by accounting, but this does not mean that accounting needs to be merged into other functions.

Deeper, not Broader
Most importantly, it’s not clear that the best strategy out of the profession’s conundrum is to try to establish its value by branching out horizontally. Forbes is asking the business continuity professional to justify her value on grounds that she can do someone else’s job, but this opens her up to the reply “that’s great, but we already have someone else doing those jobs.”

It’s normally not prudent to stake your value on grounds you can do someone else’s job. Rather, it’s best to demonstrate that only you can do your own job. The medical profession did not establish its importance by encouraging doctors to add plumbing to their skill-set. They demonstrated their value by demonstrating that they had knowledge of medicine beyond that of anyone else.

In other words, the business continuity profession is better served by deepening, rather than broadening, his or her knowledge. Any profession is better served by cutting a unique niche that is not in other’s competency, than by attempting to insert oneself into another’s business.

As David Lindstedt, director of enterprise continuity at Ohio State University, notes, the profession needs research to legitimate itself and establish its value. Too much of business continuity practice is intuition driven, rather than evidence driven. While it certainly makes intuitive sense that organizations with a business continuity plan will fare better than one without a plan during a disaster, has that ever been tested and proven? Many people quote a statistic, which varies according to the author, that “X” percent of companies without a business continuity plan fail after a disaster.

But as Lindstedt points out, even if true – and it turns out that it is hard to track these statistics back to a reliable origin – that would not tell us anything about the value of such plans until we compare the survival rates of companies with a business continuity plan to those without a plan. It might be the case that companies without a business continuity plan fare just as well as those with one.

Of course, I personally believe that having a business continuity plan is better than not having one, but convincing a skeptical upper management of this fact is going to require more than my asserting – in pretty forceful and unambiguous terms – that I think it’s true.

The problem is that studies up to now lack the foundational scientific principle of a control group. Without a control group very little can be concluded from a study. Until the late 1800’s, doctors were using the accepted practice of bloodletting to treat most diseases. In the world view of the time, it made more intuitive sense to think that disease was caused by bad spirits than the new-fangled theory of “germs” so small you couldn’t see them. It wasn’t until someone actually compared the recovery rates of those who were treated with bloodletting to those who were not treated with this practice they discovered, much to their surprise, that bloodletting did not actually make recovery more likely. Who would have guessed?

A while ago people predicted that insurance companies would require companies to have business continuity plans. This has not materialized because insurance companies are driven by actuarial tables – evidence of loss based on different factors. There is simply no evidence that business continuity plans lower loss. Again, I think (or hope) they do, but that’s not enough to convince the insurance companies.

The business continuity profession needs to deepen, rather than broaden, its knowledge base to establish its value. This means comparing the results of different business continuity techniques. For instance, there is currently a debate in some circles over whether a risk analysis should come before or after a BIA. Has anybody compared the two methods to determine which yields better results?

Research will also help produce a common set of definitions, which is sorely needed in the profession, and a common body of knowledge. Note that a common body of knowledge does not mean that everyone is forced to accept principles without question. Quite the opposite. Scientific progress can only be made when there is an accepted way of doing things that can be subjected to rigorous testing against alternatives. In other words, a common body of knowledge serves as a foundation against which people can test new ideas.

Right now too many commentators act as if they were the first person to have ever spoken on their chosen topic. There is very little sense of challenging, or adding to, a common understanding in most business continuity articles. The profession carries the air of a variety of commentators with different perspectives, rather than an organized development of knowledge. While it might sound combative, and unfortunately research can become that, knowledge can only advance when one researcher argues for or against the results of others.

Colleges and universities are in the process of fostering research to advance the field. The goal is to provide more of an organized structure to research into the field so that it advances in a way that builds up prior understanding. Some of this research will undoubtedly support commonly held beliefs, thus providing credence to what professionals are already doing. That is important to moving the field forward. But much of the research will likely challenge current orthodox.

These counter-intuitive results will be the most interesting and possibly the most important to a field. A business continuity professional who can say to a prospective employer “while it might seem the practice you are doing is best for the organization, research actually demonstrates that this other practice generates better results in the long run.” That person presents themselves as having knowledge that others do not have precisely because it is not intuitively obvious.

Nat Forbes may have accurately painted a gloomy view of the business continuity professional’s life – though I’m sure many people will disagree. We all want to elevate the profession’s status, but the light at the end of the tunnel is more likely to be found by deepening, rather than broadening, the profession’s knowledge. To be clear, it is important for the business continuity professional to have an understanding of periphery fields to do his or her job well. But the profession will only establish its value by pursuing a deeper understanding of the business continuity practice.

Last week’s quiz question
Question: What is Cadet Tracey L. Jones’ claim to fame at Norwich University?
Answer: First Norwich University Rhodes Scholar in 1995.
Winner: Ken Desforges

This week’s Quiz Question
What was the only major Revolutionary War Battle fought on Vermont soil?

Current competition standings:

Andrey N. Chernyaev: 4 wins
Matt Bambrick: 3 wins
Dianne Tarpy: 2 wins
Sam Moore
Autumn Crossett
Gil Varney, Jr.
Glen Calvo
Thomas Reardon
Sherryl Fraser
Srinivas Chandrasekar
Marc Ariano
Linda Rosa
Joanna D'Aquanni
Bill Lampe
Srinivas Bedre
Christian Sandy
Joseph Puchalski
Scott Madden
Ken Desforges