OKRs for Research, QA, Operations, and Support

“‘Tain’t what you do, it’s the way that you do it.” – Ella Fitzgerald

Teams involved in scientific research & development (R&D), operations, quality assurance (QA), and support often struggle to adopt OKRs. I recently had the privilege of introducing OKRs to a group of physicists, mathematicians, and computer scientists. They were using machine learning and artificial intelligence to performing complex quantitative modelling and scientific research activities as well as providing operational & production support for those activities.

I’ve also introduced OKRs to production support engineering teams at the BBC who were responsible for keeping BBC television on the air.

The researchers asked “How can we predict when we will make some breakthrough discovery?” and the support & operations engineers remarked, “We can’t predict what will happen in 3 hours, let alone in 3 months!”

So how can these teams use OKRs?

Use OKRs in research & development, operations, and support teams to help them improve how they deliver, not to predict what they will deliver.

OKRs can be very valuable to these types of teams, but the practice requires thinking in a more abstract way and remembering that OKRs differ from initiatives or tactics. I encourage teams to think not about what they will deliver, but how they will deliver it. In other words, use the OKRs to measure and improve your ways of working. This sounds counterintuitive, particularly to someone focused on delivery, but it embraces the fact that research & support teams exist to embrace and profit from uncertainty, not in spite of it. But they can reduce and even eliminate uncertainty in may areas of their work.

Don’t try to predict the unpredictable; instead, optimise what teams can change to improve their ability to deliver anywhere.

Research & development (R&D)

To say that it’s difficult for a research team to predict what they will discover and when is an understatement. If they could predict this, they wouldn’t be doing research, it would be business as usual.

Yet, if we look at how a team makes those discoveries, we can see plenty of “business as usual” work which can be improved. At a high-level, scientific research (or the scientific method) can be broken down into:

• Forming questions
• Making observations (collecting data)
• Forming hypotheses
• Conducting experiments
• Analysing the data
• Sharing results

Each of these is a repeatable process which can be improved. Can we measure and improve the quality of our questions? How can we improve the speed with which we gather data? And the quality and quantity of the data we collect? Can we improve our analysis practices to make them faster, clearer, or more “correct”? Can we make the running of experiments simpler, faster, and more consistent? Can we improve how we share our results with those who can use the findings?

Look at the questions above and think about how you could form each into a key result. Lead time to perform an experiment, number of experiments completed, time between completing analysis and publishing results, etc.

When setting OKRs for research teams

• Don’t try to predict what breakthroughs you’ll make or when
• Do commit to improving how you’ll make any breakthrough

Operational support teams

Similar to research teams, QA & operational support teams exist to handle the unpredicted. Predictable, consistent issues will (ideally) be analysed, understood, and permanently resolved via product or process improvements so that they never happen again. Everything else has to be handled as it happens. As such, we wouldn’t want to set OKRs around the frequency or severity of production issues (for example). However, if we look at our standard practices for dealing with these types of issues, we’ll see a lot of repeatable, predictable activity which can be improved.

How efficient is our triage process? Are our monitoring tools adequate? What about our communication processes? Do we have the right capacity and skills in the team?

Consider the above questions as objectives (e.g. “Have the right monitoring in place.”) and think about Key Results to demonstrate that we’ve reached the objective (e.g. “100% of production issues are detected and reported by us before our clients.” or “Our average time to resolution will fall from 6 hours to 30 minutes.”)

When setting OKRs for QA, operations, & support teams:

• Don’t try to predict what issues you’ll find and solve
• Do commit to improving how you’ll find and solve any issue
• Use SLA’s as a starting point.

Consider the following Key Results:

• Respond to any issue within 15min
• Run our full regression suite in under 5 minutes
• Achieve an internal Net Promoter Score (NPS) of 85

In summary

Research, operations, and support teams (in contrast to product development, or sales, for example) exist to discover or deal with the unknown and the unpredictable. OKRs in general, should focus not on outputs but on outcomes; on impacts rather than tactics. This is even more true of research, operations, QA, and support.

Help these teams to decompose what they do into repeatable steps and then find metrics (and key results) which demonstrate that their day-to-day capabilities are improving. By following this path, you’ll enable these teams to set meaningful OKRs improve how they work so that they can become more effective at everything else they do, including solving production issues or coming up with scientific breakthroughs.