Tag Archives: business continuity planning

Avoiding Disasters: The Value of Continuity Planning

Server room represented by several server racks with strong dramatic light.The recent technical problems with the Delta Airlines network got me thinking about the value of business continuity planning. We teach an AIM short course dedicated to business continuity and disaster recovery planning and stress the importance of thinking through all potential scenarios. Consider this a friendly reminder to update and test your plan to make sure it is still valid. Has anything changed since your last test and could it halt your business? What is the worst-case scenario and how will you deal with it?

Delta

Delta is just the latest example of a sophisticated network of hardware and applications that failed and caused disruption to a business. In the case of Delta, a power control module failed in their technology command center in Atlanta. The universal power supply kicked in but not before some applications went offline. The real trouble began when the applications came back up but not in the right sequence. Consider application A that requires data from a database to process information to send to application B. If application B comes up before Application A, it will be looking for input that does not exist and will go into fault mode. In the same vein, if application A comes up before the database is online, it will be looking for data that does not yet exist and will fault.

Any of these scenarios will affect business operations such as ticketing, reservation and flight scheduling processes. Once flights are canceled due to lack of valid information, then the crew in San Francisco cannot get to Atlanta to start work and even more flights are canceled or delayed. In this case, it took four days before flights were fully restored. That is a lot of lost revenue and goodwill just because one power control module failed in a data center.

Disaster Recovery Planning

Information systems and networks are complex and getting more so all the time. In order to develop a plan to cover a potential interruption consider the following steps:

  • Map out your environment. Understand what systems you have, their operating systems, how they are dependent on each other, and how they are connected to each other via the network. Is it critical that all these elements come up in sequence? This map will be crucial in the event you need to rebuild your systems after a disaster.
  • Understand risks and create a plan. Understand your risk for each system and application. A small application that only runs once a month may not need attention whereas a customer order fulfillment application that runs 24/7 should be able to failover without interruption. Create a plan to keep the environment running or to restore it quickly.
  • Test the plan. This may be the most important part of the process. Testing the plan on a regular basis ensures that you have accounted for any changes to the environment and ensures that all people are up to date on their part in the event of a problem. Periodic testing also keeps the plan active and not something that becomes “shelfware.”

Thoughts

Businesses increasingly rely on sophisticated technology in order to sell product, service customers and communicate with partners. Any break in that technology can have a real impact on revenue and the long-term viability of the business. Have you tested your business continuity plan lately?

Author Kelly BrownAbout Kelly Brown

Kelly Brown is an IT professional and assistant professor of practice for the UO Applied Information Management Master’s Degree Program. He writes about IT and business topics that keep him up at night.

Optimism Bias: How Your Half Full Glass Leaves You Vulnerable

Photo of a woman holding a large happy face sign in front of her face.There is a cognitive phenomenon called the optimism bias that leads humans to think that the likelihood of a negative event is lower than it really is. This is great when we are battling the day-to-day stresses of our lives but not so good when trying to plan for unexpected risks. In this blog post I will explore how the optimism bias can affect risk management.

Tigger or Eeyore

In a recent article for the Nonprofit Risk Management Center, Erin Gloeckner describes different personality types as Tiggers, or people who are always positive and bouncy, and Eeyores, those who are always down and negative. In reality, most of us fall somewhere in between but tend to have an optimism bias. As I have mentioned in previous posts, I tend to be an uber-Tigger and that can get me into trouble when determining the likelihood of failure or disaster. I was once asked to develop potential disaster scenarios for a project so that I could mitigate any risks associated with those scenarios. Try as I might, I could not come up with any realistic scenarios that involved failures. I realized my own bias towards optimism and asked for help from a project member I knew had a negative bias. That person was able to develop many different disaster scenarios and we created risk mitigation plans to counter each of them. True to my world, none of those scenarios ever happened but we were prepared nonetheless.

Business Planning

When managing information, it’s important to have a realistic sense of security. Start with preparing honest answers to these questions:

  • What are the chances of a security breach that leads to leaked confidential information?
  • What are the chances of a natural disaster that affects the operations of my organization?
  • What are the chances that I will lose a key person in my organization, at least temporarily?

It is important to have plans in place to counter the various threats that can happen in the course of doing business. Storms don’t stay away forever and key people don’t stay in one position their whole life. We can lessen the impact of these events by planning for them.

Personal Planning

I have talked about this in past blog posts but I think it is also important to evaluate potential risks in our personal lives. Ask yourself:

  • What are the chances that I could lose my current job?
  • What are the chances that I could suffer health problems?
  • What are the chances that a natural disaster could affect me or my family?

While it is not good to dwell on these scenarios to the point of distraction, it will give you peace of mind to know that you have planned to mitigate risks. These mitigation strategies should include making sure your skills and education are up to date and that you are exercising in order to fend off avoidable health problems. Set aside money to counter any unforeseen financial problems. Just as you plan for business disruptions, you can also plan for personal issues. These plans can help you sleep at night and be a Tigger all day.

Thoughts

If you are interested in learning more about the optimism bias, there is an excellent 2012 TED talk by Talit Sharot that covers the topic. Whether you tend to be an Eeyore or a Tigger, it is important to recognize your biases as you make plans for your business and your life. Do you already know your personal biases? Let me know your thoughts.

Author Kelly BrownAbout Kelly Brown

Kelly Brown is an IT professional and assistant professor of practice for the UO Applied Information Management Master’s Degree Program. He writes about IT and business topics that keep him up at night.

All In The Preparation

Photograph of hiking and camping equipment.Last week I went backpacking in the high Cascades of Washington state. As I hiked I thought about the preparations I made to get to this beautiful place and how they helped me enjoy the moment. I had prepared physically and mentally and I had brought the right gear. At the end of the day, however, I realized that I brought too much food and my legs were very tired by the time I got back to the trailhead. In one instance I prepared too much and in another I did not prepare enough. In this post I will focus on the value of proper preparation for everyday activities and for life.

On The Run

A number of years ago I ran the Portland Marathon for the first time. I prepared by running smaller distances such as five and ten kilometers but I did not know how long a marathon would take. I had a goal of finishing in four hours or less. I came in at four hours and twenty minutes. In contrast, world-class athletes run it in two hours and fifteen minutes. In other words, they had finished, gone home, eaten lunch and taken a nap by the time I finished. I ran it again the next year just to see if I could meet my original goal. That year I came in at four hours and ten minutes and that is the last time I ran marathon. While I was prepared to finish the race I was not committed enough to put in the preparation necessary to meet my goal.

Business Continuity

In our Business Continuity and Disaster Recovery Planning short course we talk a lot about preparing for a possible business interruption or disaster. Each business needs to understand their tolerance for risk and how prepared they need to be. A hospital, for example, needs to be prepared for any risk since lives might be in danger in a disaster. By contrast, a taco stand only risks losing a small amount of revenue in the event of a disruption so the continuity preparation is not as great. There are different levels of preparation and it is important to understand what level is needed in various scenarios.

For Life

One of the goals of the AIM program is to prepare students to meet the challenges of tomorrow, next year, and for life. Our curriculum is broad, but also deep in areas in which students need experience to perform their daily tasks and work with others to accomplish their goals. AIM students are preparing now for future challenges and opportunities.

Thoughts

In reflecting on these ideas I have come to realize that there are degrees of preparation and I need to understand how much is needed and how much I am willing to invest. Sometimes I prepare too little and expect a different outcome and sometimes I prepare too much and end up having to carry a heavier pack. Are you prepared enough for opportunities coming your way? Is there anything more you need to add to make sure you are ready? Let me know your thoughts.

Author Kelly BrownAbout Kelly Brown

Kelly Brown is an IT professional and assistant professor of practice for the UO Applied Information Management Master’s Degree Program. He writes about IT and business topics that keep him up at night.

All Shook Up: What Happens To Your Data When The Earth Moves?

Illustration of seismometer graph, indicating a period of seismic activity.On the morning of July 4 there was a magnitude 4.2 earthquake just east of Eugene. The jolt shook everyone, but there was no significant damage to homes and businesses or the road system. The earthquake temporarily rattled nerves and it has renewed the conversation about “the big one” here in the Pacific Northwest. The Cascadia subduction zone, running off the west coast between California and Vancouver, BC is overdue for what is expected to be an 8.0–9.0 magnitude earthquake and accompanying tsunami. While it is important that we plan ahead to protect our homes and infrastructure from earthquakes, I wondered what we are doing to protect our digital assets. I decided to do some research.

Data Centers In The Northwest

There are several data centers in the Pacific Northwest, primarily because of inexpensive power, relatively cool weather, abundant water, and a talented workforce. These data centers are operated by companies such as Google, Amazon, and Facebook. They were all built after we began to emphasize earthquake ready infrastructure.

New buildings are designed to withstand at least some lateral movement due to seismic activity. They are secured to the foundation and multistory buildings are heavily braced but they can still sway up to a certain amount to counteract the effects of an earthquake. The new data center for Oregon Health Sciences University (OHSU) for example is designed as a geodesic dome to “…provide superior resistance to seismic events.” While doing this research I came across an inventive solution in Japan that floats a home or building on air during an earthquake and then returns it to the foundation after the event.

Inside The Data Center

Servers inside the data center are often housed in seismic frame cabinets, which are anchored to the building but still allow for a minimal amount of movement. This keeps the server rack from falling over or dancing across the floor. Another option for flexibility is a product called ISO-Base, a two-part device that uses an isolation base. The bottom of the base is bolted to the floor and the top is bolted to the bottom of the server rack or cabinet. There is significant flexibility between the two levels of the base so in an earthquake the cabinet has controlled movement within the confines of the base. This means that cabling has to also be flexible.

Components that are seismic rated, including the backup generator outside, are tested on a shake table. This is a platform that simulates an earthquake and can test buildings or components to make sure they can withstand seismic force. The largest shake table is outside of Kobe, Japan and measures 65 x 49 feet and can hold structures weighing 2.5 million pounds. The shake table test is part of a seismic certification process for equipment, including computer infrastructure and components.

Thoughts

Computer centers in earthquake prone areas of the country have secured your cloud data as part of their business continuity plans. They employ several products and techniques to secure facilities, equipment, and data in the event the earth moves under their feet.

In a future blog post I will talk about products that let us secure equipment and data in our home office.

Author Kelly BrownAbout Kelly Brown

Kelly Brown is an IT professional and assistant professor of practice for the UO Applied Information Management Master’s Degree Program. He writes about IT and business topics that keep him up at night.