Disaster Movies

In every disaster movie you’ve ever seen, there’s a key transitional moment when the characters change from believing that they can go back to the way things were, to accepting that the old world is gone. Up until this point they’ve been fighting a losing battle, frantically trying to shovel back the tide, and it’s only when they accept their new reality that they’re able to find a solution.

I like to think of this when facing huge problems. The question isn’t whether something will need to be addressed, but when, and how much will it cost to fix it now vs. waiting for later. There comes a point when you’re not going to have a choice – your servers are melting into the ground, you can’t scale any more, and something fundamentally has to change. Your old model won’t work anymore, so you’re going to have to rethink your architecture, your processes, and/or your business plan.

Sometimes this is because things are going well – your site traffic has grown so dramatically that simply adding more hardware (servers, RAM, cores, SSDs, etc) isn’t going to cut it. Sometimes it’s because things are going terribly wrong – no one’s buying your product and you desperately need to pivot.

The Plot

You know that you’re getting close. You do. You know that there are things that need to get done, changes that need to be made if disaster is to be averted. And you care, you want to do them, you even think they’d be exciting to do. You talk to your boss and co-workers about the danger, even obsess about it, and think about what you would do if only you could. You might even complain that no one else is doing anything about the problem, and bemoan the complacency.

But you do nothing.

It’s easy to do nothing. Even when it’s the most dangerous thing in the world, it feels safe to continue with the status quo. And let’s be fair, you have plenty of other work to do, and there’s a reason why “shooting the messenger” is an expression. But it keeps getting worse, and the pressure continues to build, and then one day, out of nowhere, there’s an initiative from somewhere else in the organization. And you think, hey, I thought of that months ago! And you watch from the sidelines while someone else tries to save the day.

The Method

In a previous post, I talked about how hard it is to know which of all the chronic problems you face is the one that most needs attention, and how being able to do so is one of the most important skills you can develop. Good employees tidy up around the edges. Stars figure out what critically needs to get done, and how to do it – both technically and organizationally.

The first step is to understand the problem. It’s not necessarily hard to identify a problem (“boy, our code repository sucks”), but understanding the history, root cause, and options for mitigation, can take time and research. Don’t just stop with understanding it inside your head – write it all down. This serves several purposes – when you write something down, it’s more obvious when you’re missing key pieces of information. Also, it serves as a reference that you can continue to add to as you learn new things. And lastly, it’s a great resource when you get to the point of communicating your findings and suggesting a plan of action.

Which brings us to the second step – coming up with a plan. You’ve done your research, you’ve come up with the best option(s), so get it all down on paper. Plans should be as concrete as possible, ideally with phases, timeframes, deliverables, and resource allocation (this sounds like a lot, but doesn’t have to be – the goal is to be concrete, not verbose). The more concrete, the more useful. Your plan will be a strawman – though the ideas in it should be sound, it’s likely going to get torn to pieces when you share it with your boss and/or colleagues. That’s expected, so don’t be surprised or discomfited, and definitely don’t get emotionally attached – your goal is to find a solution, not to demand that people use your solution. Similarly to the initial research step, writing down and presenting a plan serves several goals. First, it helps you refine your ideas, and gives you an opportunity to see where the weak points are. Next, it provides a starting point for the conversation. Uncomfortably large problems can be very difficult to even start thinking about, so giving people an anchoring device – even to attack – can be very helpful. Lastly, it helps establish you as a contributor. You may not end up doing the work, but people will remember that you were involved from the beginning.

Now, of course, the actual work begins. You’ve researched the problem, you know the various options, you’ve been involved in the planning from the very beginning – at this point, no matter what your official role, you’re an active part of the solution. This isn’t the movies – sometimes it doesn’t work. Sometimes feelings get hurt, toes get stepped on, and Nausicaans stab you through the heart in a bar brawl. But when disaster’s about to strike and you see the opportunity, to some extent the choice becomes existential. What kind of person do you want to be? The kind that stands up to make a change, or the kind that keeps his/her head down, erects a “somebody else’s problem” field, and hopes the issue magically goes away?

There is one important caveat. Before you go ahead with any of this, there’s a critical step that you need to accomplish. Call it step zero. Without this, you won’t have credibility, and even the best researched, most painstakingly detailed, most likely-to-succeed plan will be ignored or dismissed. As I’ve mentioned previously, to be a star you first have to meet baseline expectations. Sure, there are exceptions to every rule – no one will question your credibility if you’re pulling them from a burning building – but when it comes to big hairy problems, you need to have built a positive reputation before people are going to pay attention to your ideas. The landscape is littered with the bleached bones of well-meaning people who blew off assigned tasks, failed to earn street cred, and were ignored when they tried to participate on bigger, more difficult problems.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s