In my last post, I wrote about why focusing too much on conversion rates alone is a bad idea because conversion rates can easily be manipulated by changing the quality of traffic to your site.

It got me thinking about whether there are metrics that can’t be manipulated. The key for such a metric is that improvements to it have to always be correlated with the health of the site or business. Put another way, if you can artifically inflate a metric or an improvement to that metric could actually be a bad thing, then it wouldn’t qualify; it wouldn’t be a so-called north star metric.

For example, a homepage to purchase conversion rate wouldn’t work. You can improve your conversion rate by dropping search traffic, reducing the price of your product, and more – all of which could actually be bad for your business.

It gets interesting when you start making the metric more and more specific to overcome ways it can be manipulated. What if instead of looking at your overall homepage to purchase conversion rate, you just looked at the conversion rate of direct traffic? That’s better, but it can still be manipulated by increasing your offline advertising which could change the quality of the direct traffic and therefore your conversion rates.

I go back and forth about whether it’s even possible. Like, will there always be a way to improve a metric but have it actually be bad for the business? Or is it possible to avoid manipulation by being really, really specific about the metric?

I’d love to hear from folks on this topic, especially if you have a metric in mind for your site/product/business that seems unhackable.