Metrics allow you to understand your customers and how the service is performing. Measuring benefits/value before and after transformation will help you to articulate the overall success of your service re-design.
Qualitative vs quantitative metrics
It is important to understand the difference between qualitative and quantitative metrics.
| Qualitative | Quantitative | |
|---|---|---|
| Conceptual | Concerned with understanding human behaviour from the customer’s perspective. | Concerned with discovering facts about social phenomena. |
| Methodological | Data is collected through interviews and participant observations. Data is analysed by themes from descriptions by customers. | Data is collected through measuring things. Data is analysed by numerical comparisons and statistical inferences. |
| Examples | Gathers information that is not in numerical form. i.e. diary accounts, open-ended questionnaires, unstructured interviews and unstructured observations. | Gathers information in numerical form, e.g. number of transactions, cost per transaction, number of customers affected, cycle time, number of pages, etc. |
When to gather and understand metrics
Metrics are applied in all phases - initially to understand your customers, then as a measurement to realise benefits.
| Phase | Why | Qualitative or quantitative |
|---|---|---|
| Pre-mobilisation | To qualify the problem statement and the level of value. | Quantitative |
| Mobilisation | To document additional benefits identified during workshops. | Quantitative |
| Discovery | Interviews and observations to understand customers. Quantify hypotheses to determine the best one to take forward. | Qualitative Quantitative |
| Alpha | To ensure benefits are still being realised. To develop a benefits road map. | Quantitative |
| Beta | To measure user satisfaction, digital take-up, completion rate and cost per transaction. | Quantitative |
How to collect metrics
Metrics can be collected in a number of ways:
- Websites - adding Google anaytics can provide behavioural statistics like page views, informing which pages people are viewing, or not.
- System - where forms or applications are entered into a system it can provide information on the number of applications received over a period, or cycle times, or anything else that can be collated from the information collected.
- Paper-based forms - these can still provide information simply by counting the number over a period or picking key items from the information collected. This is a very laborious process.
- Service centres - if your service is supported over the counter, you could record the number of visits, what services are being processed, how long it generally takes, what questions are being asked?
- Post office - if your service can be processed via the post office, you could record the number of transactions per service, types of users that use post offices and costs involved.
- Emails - if your service involves an email component, these can be counted and can record why the email was sent in the first place.
Example metrics
| Measurement | Description | Example |
|---|---|---|
| Transaction volumes | The number of transactions that are processed by a service over a period of time, e.g. quarterly, yearly. | 20,000 per year |
| Cross counter | The number of customers that physically present and have their service processed over the counter at a service centre over a period of time, e.g. quarterly, yearly. | 50,000 per quarter |
| Cycle time | How long does it take to complete a transaction? This is measured from the time an application is sent in for processing through to its completion. | 14 days |
| Call centre | How many calls are received, about what, and the average time per call. | 15 per day, 7 minutes per call |
| Web analytics | Can provide numerous statistics like page views, browsers, locations, exit pages. | Page views, bounce rates, browser used, location, exit pages, etc. |
| Cost per transaction | What is the current cost per transaction? This is measured by collating all costs involved in processing the transaction and dividing it by the number of transactions processed over that same duration. | $2 per transaction |
| Social benefit | Benefit to people of the environment, such as fuel consumed for a typical person driving to and from a service centre, or kilometres travelled. | 884,626 litres 7.2 million kms |
Team skills required to capture metrics and data
A good understanding of how to measure a service is required. It's important to ensure your measurements are correct as they could determine which problem you will or will not move into the Alpha phase. This task is usually left to someone who has a good understanding of the service, already does statistical measurements or is a business analyst.