Strategy

Research

Metrics are the signals that show whether your UX strategy is working. Using metrics is key to tracking changes over time….

Metrics are the signals that show whether your UX strategy is working. Using metrics is key to tracking changes over time, benchmarking against iterations of your own site or application or those of competitors, and setting targets.

Although most organizations are tracking metrics like conversion rate or engagement time, often they do not tie these metrics back to design decisions. The reason? Their metrics are too high level. A change in your conversion rate could relate to a design change, a promotion, or something that a competitor has done. Time on site could mean anything.

Continue Reading…

UX strategists need to take charge of the metrics for online experiences. First, we’ll look at the current state of metrics in most organizations and some of the problems in defining metrics for user experience. Then, we’ll focus on three key types of metrics for user experience, how to track them, and how to integrate them into an organization’s measurement framework.

The Signal Problem

The data that is available from off-the-shelf analytics, A/B tests, and even follow-up surveys does not always result in insights that inform the user experience.

There is so much data available on sites and applications that it seems amazing insights would be sure to surface, yet that does not happen without smart decisions. The data that is available from off-the-shelf analytics, A/B tests, and even follow-up surveys does not always result in insights that inform the user experience.

• The signals that are easiest to track don’t show what is really important. Pageviews, for example, are easy to collect, but do not tell you much about the experience people have actually had on your site or when using your application. Your organization’s goal is probably not to make sure that every user views a certain numbers of pages. So, while pageviews may be a great metric for ads, they are not a good way to track or encourage engagement.
• Signals can be ambiguous. Think about time on site, which people often equate with engagement. More time could be positive, but it could also be negative—time that the user spent feeling confused, distracted, or frustrated. Even if you track engagement by looking at time spent on your site and pages per visit together, it’s still not clear how that equates to engagement.
• The signals don't always map to design. Maybe after the launch of a new feature, traffic starts going up. The product team might think this is because of the new feature, sales may link the increase in traffic to a new promotion, and the UX team might assume that it’s connected with their new design. But the increased traffic might not relate to any of these potential causes. A/B tests do connect data with design, but the data is granular. A metric like the number of clicks for image A versus image B can help you to make tactical decisions regarding how to implement a user-interface element, but it does not work well for bigger design decisions.
• There may be too many signals to act on. A lot of metrics get reported simply because they are flowing in from analytics tools that can track hundreds of metrics and are endlessly customizable. It is tempting to measure everything and hope that insights will emerge on their own, but usually, they won’t.
• The right signals might not get captured at all. Companies usually track metrics post launch. Metrics that you could use to inform design need to be captured while you’re developing new ideas, but often those metrics are neither quantified nor tracked.

So, it’s difficult to find the right signals for user experience amidst all the noise in your data. Further complicating matters, companies often identify and track the metrics that could help you to understand how you’re doing against your UX objectives—the key performance indicators (KPIs)—elsewhere in the organization and without involving the UX team.

The Early, Modern History of UX Metrics

Most metrics are marketing oriented, not experience oriented. Unique visitors can tell you whether your marketing campaign worked and social mentions can tell you whether you’ve got a great headline, but these metrics do not reveal much about the experience people have had using a site or application.

Table 1 does not provide an exhaustive list of metrics, but it illustrates some of the differences between what marketing may be tracking and the types of metrics UX teams currently track.