The Right Service Quality Measure: Average or Median?

Posted on by

IT wants to deliver quality service, which requires accurately measuring and presenting your service performance. Choosing the right approach ensures that you’ll be able to quickly focus executives and managers on the information they need to make agile decisions.

If you’ve been working with IT metrics for a while, you’ve seen many standard KPIs that all use the same measurement, approach: Averages.

Mean Time to Repair, Mean Time to Resolve, Mean Time between Failures – all are calculated using averages. But this isn’t the only way you could measure these outcomes, so why do it this way? Why not use, for example, Medians?

You might be doing this because this is the way you’ve always done it, or because your tool doesn’t give you any options. But let’s supposed you’re not satisfied with doing things the way they’ve always been done – why do we use averages instead of medians?

 So, what is the difference between Averages and Medians?

Overall, both averages and medians are looking at what falls in the middle of a range of numbers 

If you cast your mind back to school, you’ll remember that an average is calculated by adding up all the values and dividing by the number of values.

For example, to average 1, 2, 3, 3, 4, you would add the numbers up (1 + 2 + 3 + 3 + 4 = 13) and then divide by the number of values (5) to get a result of 2.6.

On the other hand, to get the median of 1, 2, 3, 3, 4, you would put the number of values in order and take the middle value to get a result of 3.

Even from that little example, you can see that the results are similar but not the same.

The reason is that averages tend to be more sensitive to outliers. Medians, meanwhile, are limited to either the numbers in your set, or (if your set of data is odd) halfway between two numbers. Let’s take a look at two examples that illustrate the impact.

Here’s one example, based on the number of times incidents have been reassigned:


You can see a clear divergence in the reassignment average and median.

A major reason for the difference is the 12 incidents that were reassigned 100 times. Although it’s only a few edge cases (compared with over 200 that were only reassigned once), it has a big impact on the average. Take a look at the same report, but with those 12 incidents excluded:

You can see that by dropping the outlier, the average moves a lot without the median moving much – and now the average looks a lot more like the median!

Okay so they’re not the same – why does an IT team care?

Clearly if these two measures give you different results, you want to think twice about which one you pick!

Let’s start by looking at the differences and what that might mean in the data.

Use Medians to Limit the Impact of Outliers

When you have a big set of data that may have a lot of outliers, you can use a median to get a steadier idea of the data. For example, Service Level Agreements may often have a lot of data, and there may be a few SLAs that stay open for a long time, disrupting your results.

Suppose you have an organization that is measuring resolution of service delivery. Our target is to have the typical service resolved within 14 days.

Here’s a chart with the average and median resolution times (the dark dotted line across the center is our target):

You can see that the Average resolution time has a lot more spikes and troughs where a few outages may have deeply impacted the numbers. The Median is a much more consistent measure of what the typical person is experiencing.

What does that tell you? It tells you that your typical resolution experience is actually on-track! If you tackle the few big outliers, you could make a big impact in your number! On the other hand, if the median was above the target, you’d want to make decisions about how to speed delivery time in your typical process.

(Good news though: the colored, dotted lines into the future show that both measures are trending to improve).

Use Averages Where Fine Distinctions are Useful

On the other hand, sometimes the sensitivity of an average is useful. One common example, which also speaks to service quality, is service desk survey results – typically ranking service on a scale of 1-5 or 1-10.

Take the following set of survey results:

Here, because the values are whole numbers, the median is going to tend to be a whole number – and in this case, it’s pretty likely to be 3. This is a case where you want the outliers to have a bit of an effect, and you know it won’t be disproportionate because your range is limited from 1 to 5!

The Right Measurement for the Task at Hand!

As you can see, the choice of which measurement can really impact what you see when you analyze these important service quality metrics. Both averages and medians have their place, so make sure to think carefully about which approach provides your executives and managers with the most accurate answers to the question they’re asking.

Aging Reports: Not Just For ServiceNow Incidents

Posted on by

The Power of Aging Reports

One of the most common reports we see operational teams depending on is an aging report. It’s no surprise: an aging report is a powerful way to focus the team on improving resolution times.

This classic example of an incident aging report shows the view of a manager whose team who needs to focus on their most stale incidents:

Users who are logged in can drill through to take action on these incidents, driving them to resolution. (Bonus technique: that report is contextually filtered so it shows each manager their own direct reports!)

But for this post, we’ll be examining some other contexts to apply aging reports.

First: the Ease of Aging Reports in Explore Analytics

If you’re thinking about making more aging reports, take a moment to think about how you build the reports.

Because ServiceNow doesn’t capture an “age” field, some organizations are using a code + customization based approach that has to be applied to each table. Others are dumping the data to Excel on a regular basis and building reports there.

Instead, Explore Analytics gives you the ability to build the report using a simple interface:

You just select “Treat Date as Age” on the “created” date-time, and fill in the groupings!

If you want to learn how to build an aging report for ServiceNow, check out this video!

Use Case 1: Other Task-Based Processes

This is the low-hanging fruit, but any task within ServiceNow (or outside of it!) is ripe for the same analysis.

HR Case Management, Customer Service Management, Facilities Management, or your own custom processes — anywhere you have a time-stamp representing when it was opened, you can apply this same approach.

Here’s an example of an HR Case aging breakdown:

(Bonus technique: use flexible, multi-level pivots to replace multiple reports — here, by allowing us to easily compare branches within the same report. Also: the “Branch” column is just a relabel within the report for Location.City — present fields in the way that is easiest for the viewer to understand!)

This highlights the importance of using a flexible tool like Explore Analytics to build the reports, rather than manual techniques like Excel or customizing ServiceNow for every table you want to build an aging report against. You want a tool in your toolkit that lets you apply these solutions over and over!

Use Case 2: Managing Contract Expirations or Other Due Dates

Instead of looking at the time since a task was created, you can also use the same approach to measure how long until a deadline. The methodology is the same, but gives you the ability to look at asset contract expirations, or other deadlines:

Use Case 3: Keeping CMDB and Knowledge Up to Date

Another variation is reporting on the age since a record was Updated instead of when it was created. That indicates whether your data is fresh, which is incredibly important for processes like asset management, CMDB, and knowledge management where data quality relies on being up-to-date.

Here is an example from our CMDB data quality pre-built application (video here!), showing which CIs are up-to-date by class:

Bonus technique: because Explore Analytics allows you to track long-term metrics in the same way as operational real-time report, here’s an aging KPI (among other CI metrics) to go along with that real-time dashboard:

Try it out today!

Excited to try out these use cases? If you don’t have a tool today to build these reports quickly, easily, and without writing code, try Explore Analytics free for thirty days here.