The folks at Electric Pulp recently published some conversion numbers touting a successful responsive redesign for a client. While I’m sure it was a resounding success, the numbers they published don’t tell the whole story.
The only numbers we saw1 were the headline conversion improvements for mobile devices. And they were great numbers! But a lot of people are misreading what that means.
From their anaylsis, we don’t know what percentage of their traffic is mobile. We also don’t know how many of those mobile buyers were actually new, incremental customers to the business, as opposed to customers who would have otherwise completed their purchase later from another device.2
Without those two pieces of data, we can’t really say how much that redesign meant for the company’s bottom line. Measuring mobile share is simple, and I’m sure the team had that in mind when discussing the results with their client. But tracking visitors across devices is far from a solved problem.
There are a few things you can do to start painting a picture of how much device switching is impacting your measurements.
Watch your toplines
Keep an eye on your overall top-line revenue, orders or other transactions. Did they grow as expected when accounting for the increased conversion on mobile? If your growth rates are predictable, you should be able to model out the scenarios for a 100% marginal improvement for those mobile visitors and compare it to your actual growth rates.
Monitor direct load traffic
If you make a mobile-friendly change, analyze your direct load traffic from desktop devices. There’s a good chance you’ll see it drop, as device switchers will no longer need to pull up your site at a later time. (Natural branded search traffic could play out similarly.)
Send a survey
Surveying recent customers could give some insight into how many of them used your site on multiple devices. It wouldn’t be a precise measure, but combined with your topline growth, could help reinforce the overall story. (Of course, sometimes the cost of acquiring insight can be higher than it’s value - and this could likely fall into that bucket depending on the size and stage of your business.)
-
They only published the sexy headline numbers, but that doesn’t mean they don’t have the complete story back at HQ. This isn’t a knock on the published analysis, hopefully just shedding some light on faulty interpretations of the analysis. ↩
-
It’s still an experience improvement for customers if they can stay on the device they’re using, so we can’t entirely discount the improvements for users that would have bought from another device. But in terms of actual new revenue, we have to. ↩