A Report from the Front Lines It was Monday, August 29, 2005. Katrina hit between the Louisiana/Mississippi border. I live in central Mississippi, about 150 miles from the Gulf coast. When Katrina passed through, the eye was about 20 miles east of me and sustained winds were about 80 miles an hour. This was much more severe than with Camille. Our power went out at about 1 p.m. and cell phone service stopped soon after. With the media focus on the calamity that was New Orleans, great recovery stories were happening outside the national spotlight. Some of these recoveries were the product of good planning, while others were the result of quick thinking, brought about by good planning. In the aftermath of Katrina, I would like to share with you some important lessons learned. 1. It Is All About The Planning I was concerned with the recoverability of clients and my family. There was no power or cell phone service. Many had lost both their land and cell phone service and could not contact anyone. Some people could not call me, but I could occasionally get through to them. Others could call me, but I could not call them. Worst of all, there was no gasoline available. Once the roads were cleared, I would not dare leave the house; I did not want to waste gas. What little gas that was available needed to go to the emergency workers. I would work from my father-in-law’s house (one of only 3% that had power) and manage the client recoveries from there. I was unable to be with my clients, yet they all worked through their recoveries with calmness and creativity. How could that be? I was expected to manage the recovery process. Why did they not need me? Do they need me? The answer is that we had planned well. We had practiced our plans. My job was done before the storm – we plan, we practice. Like a coach in sports, once the players take the field, it is their show! Don’t get hung up on your role during recovery. It’s all about the planning. Your job is to make sure everyone is prepared. 2. Worst Case Is Not Enough Worst case planning was our best way of preparing. What was the worst tornado in history? Let’s research it and prepare for one of that scope. The same held true for hurricanes. We had always planned for a hurricane like Camille in 1969. Camille was the second most powerful hurricane to hit the United States, with a wind speed of 190 miles per hour when it hit the Mississippi coast. Damage was severe on the gulf coast, moderate about 60 miles in and caused very little damage in central Mississippi. With that in mind, some recovery strategies included central Mississippi-based businesses having primary recovery facilities in Hattiesburg branch offices, 90 miles south of the home office and 60 miles from the coast. Other strategies included recovery facilities in Jackson, 80 miles north of the home office. Based on Camille, this was a sound strategy. A tornado, flood, fire or hurricane would not severely affect both locations based on worst case history. Widescale outages would not impact both locations. We were set. Katrina hit slower than Camille, but retained strength much longer. Damage in Hattiesburg was massive. All phone service was out, land lines as well as cell phones. Power was out. Generators were running, but some of them burned out after 96 hours of use. The county’s sewage system was out. There was no way for fuel to get in since the fuel terminal that supplied most of Mississippi was severely damaged. Homes in central Mississippi suffered, but not at the same level of those in south Mississippi or the coast. We had 97% power outages, phone outages, cell phone outages and large numbers of downed trees. The trees damaged homes. Roofs were not torn off; they were just hit. Instead of looking at Hattiesburg, we should have looked to Memphis for our strategy. We needed to plan for zero phone service – and have satellite phones in place. We needed to plan for a loss of internet service and had a satellite service available as a backup. We needed to plan for an outage from our long distance provider and had a second provider available. We have now done all of these. Historical worst case is not enough. Plan for a little more. 3. Business Process Planning Was Critical The storm hit August 29. End of month processing was impacted including payroll, taxes, booking sales, etc. With a clear understanding of what was important and what was not, confusion was avoided. The business units knew what needed to be done immediately and what could wait. When work could be performed either manually or with available workstations, the users knew what to expect. We knew what had to be done at different times of the month. We had practiced working short shifts – one or two hours at a workstation before giving it up to the next user. Documentation was saved from manual processing so it could be entered into the system when it was available. Notes were kept so that a proper post-mortem could be conducted. Some teams worked from home. They had practiced this for several years – not just stating that they could work from home, but proving they can quarterly. Policies and controls were also in place to create as secure an environment as possible. 4. If You Really Want To Know What Is Going On, Don’t Watch Cable News Power was restored to our house (and my office) eight days after the storm. Cable service was restored three weeks after the storm. I did get to watch the national news for the week we spent at my father-in-law’s house. I was unable to watch more than 10 minutes at a time. All of the second guessing and sensationalism did not give the outside world the view of what was happening in Mississippi. Acts of kindness, sharing, teamwork and sacrifice were not shown on the national news. Fortunately for us, we could watch the local news and read the local papers. It hurt me that the rest of the country could not. They would have understood what was happening in Mississippi and would have been astonished by the stories. They would have learned of local charities that needed help. During the disaster, the local television stations and newspapers did a fantastic job covering the stories that needed to be covered. They focused on what happened, what the response was, and what was needed to continue to make progress. There was not time for the gotcha game or pointing fingers. The local media, from the coast to Jackson, never lost contact with the public. They were there and brought the news home. As a planner, what does this mean? When disaster strikes, if you really want to know what is going on, find the local news. Thanks to the Internet, you can view the web sites of local television stations and newspapers. Do a search on the town name and “daily newspaper” or “television station”. Follow the links and learn. You will gain insight that should help your own recovery planning efforts. It was easy to see that some of the lessons of Katrina were successfully put into action less than four weeks later when Rita hit the Texas/Louisiana coast. As for us, September 25th was a day of torrential rains and tornados. We were ready.
The four lessons discussed in this article barely scratch the surface of all of the lessons learned. I hope that the four discussed in this article may not have been as obvious as others and will help your planning efforts. About the Author Gerry Printz, CBCP, CISA, CISSP is the President of AMSADOR, Ltd., a consulting firm dedicated to business continuity planning and information security consulting. He can be reached by telephone at 601-824-4011 or by email at gprintz@amsador.com |