IT wants to deliver quality service, which requires accurately measuring and presenting your service performance. Choosing the right approach ensures that you’ll be able to quickly focus executives and managers on the information they need to make agile decisions.

If you’ve been working with IT metrics for a while, you’ve seen many standard KPIs that all use the same measurement, approach: Averages.

Mean Time to Repair, Mean Time to Resolve, Mean Time between Failures – all are calculated using averages. But this isn’t the only way you could measure these outcomes, so why do it this way? Why not use, for example, Medians?

You might be doing this because this is the way you’ve always done it, or because your tool doesn’t give you any options. But let’s supposed you’re not satisfied with doing things the way they’ve always been done – why do we use averages instead of medians?

 So, what is the difference between Averages and Medians?

Overall, both averages and medians are looking at what falls in the middle of a range of numbers 

If you cast your mind back to school, you’ll remember that an average is calculated by adding up all the values and dividing by the number of values.

For example, to average 1, 2, 3, 3, 4, you would add the numbers up (1 + 2 + 3 + 3 + 4 = 13) and then divide by the number of values (5) to get a result of 2.6.

On the other hand, to get the median of 1, 2, 3, 3, 4, you would put the number of values in order and take the middle value to get a result of 3.

Even from that little example, you can see that the results are similar but not the same.

The reason is that averages tend to be more sensitive to outliers. Medians, meanwhile, are limited to either the numbers in your set, or (if your set of data is odd) halfway between two numbers. Let’s take a look at two examples that illustrate the impact.

Here’s one example, based on the number of times incidents have been reassigned:

You can see a clear divergence in the reassignment average and median.

A major reason for the difference is the 12 incidents that were reassigned 100 times. Although it’s only a few edge cases (compared with over 200 that were only reassigned once), it has a big impact on the average. Take a look at the same report, but with those 12 incidents excluded:

You can see that by dropping the outlier, the average moves a lot without the median moving much – and now the average looks a lot more like the median!

Okay so they’re not the same – why does an IT team care?

Clearly if these two measures give you different results, you want to think twice about which one you pick!

Let’s start by looking at the differences and what that might mean in the data.

Use Medians to Limit the Impact of Outliers

When you have a big set of data that may have a lot of outliers, you can use a median to get a steadier idea of the data. For example, Service Level Agreements may often have a lot of data, and there may be a few SLAs that stay open for a long time, disrupting your results.

Suppose you have an organization that is measuring resolution of service delivery. Our target is to have the typical service resolved within 14 days.

Here’s a chart with the average and median resolution times (the dark dotted line across the center is our target):

You can see that the Average resolution time has a lot more spikes and troughs where a few outages may have deeply impacted the numbers. The Median is a much more consistent measure of what the typical person is experiencing.

What does that tell you? It tells you that your typical resolution experience is actually on-track! If you tackle the few big outliers, you could make a big impact in your number! On the other hand, if the median was above the target, you’d want to make decisions about how to speed delivery time in your typical process.

(Good news though: the colored, dotted lines into the future show that both measures are trending to improve).

Use Averages Where Fine Distinctions are Useful

On the other hand, sometimes the sensitivity of an average is useful. One common example, which also speaks to service quality, is service desk survey results – typically ranking service on a scale of 1-5 or 1-10.

Take the following set of survey results:

Here, because the values are whole numbers, the median is going to tend to be a whole number – and in this case, it’s pretty likely to be 3. This is a case where you want the outliers to have a bit of an effect, and you know it won’t be disproportionate because your range is limited from 1 to 5!

The Right Measurement for the Task at Hand!

As you can see, the choice of which measurement can really impact what you see when you analyze these important service quality metrics. Both averages and medians have their place, so make sure to think carefully about which approach provides your executives and managers with the most accurate answers to the question they’re asking.