Making a Mess out of Measurement

Making a Mess out of Measurement


Measurement is such a tricky thing. Whether you’re trying to brew your morning coffee or analyze a full-scale marketing program, there are more ways to do it wrong than to do it right. But don’t let that scare you away from measurement. No matter how many terrible cups of coffee I make, I’m diligent. Mostly because I need the shot of caffeine and somewhat because every once in a while, I actually get it right.

Now, before I go any further let me point out that no one is ever going to mistake me for a measurement expert. I can’t geek out on statistical anomalies, p-values or standard deviations any more than I could lace up a green apron at Starbucks and make you a tasty Grande, Non-Fat, No-Whip Cinnamon Dolche Latte (My wife’s usual).

Which is fine because we have baristas and statisticians for complicated matters like that. My advice is more geared toward that solid cup of morning joe or those every day decisions we make based on “the data” that we so frequently torture until it tells us what we want to hear.

With that disclaimer out of the way, here are my favorite measurement flubs:

The N of 1 - ”We tried that, and it didn’t work.” Whenever someone says that to you, the first thing you should ask them is this: How many times did you try it? In most cases, that someone will respond with “once.” That’s a bad answer, by the way. This is a dangerous and unscientific way to make decisions.

The Pile On - This happens when you introduce too many variables into your experiment, and you can’t clearly evaluate what’s working and what’s not. Throwing one more log on the fire can quickly burn down the forest of insight you’re seeking. On a related point, just because you can measure something doesn’t mean you should.

The ROI Trap - Sometimes tactics can be wildly successful in “influencing” specific actions and buying decisions without getting credit for “closing the sale.”  If you focus myopically on ROI, you may conclude a tactic is not worthwhile, despite it being mission critical to your success. More often than not, it is the combination of marketing techniques that do the trick.

The Blink - Just because it works for Malcolm Gladwell doesn’t mean you should apply it to measurement. Sometimes people approach measurement like this: They plant vegetables in the garden, and then dig them up the next day to see if they are growing. In our instant gratification society, we are used to having our questions answered immediately. Don’t pull the plug at the first sign of trouble. Instead, allow time to capture enough data to make informed decisions.

The Gut Check - This is when someone blatantly disregards what the data shows, because it doesn’t validate his or her gut. Our field is both art and science. But sometimes we default to our gut, even when there’s ample proof that it’s wrong. We slice, dice, twist and turn until we get the numbers to tell the story we want.

The Missing Link -  Sometimes we fail to identify how we’re going to measure success in the first place. And then halfway through, we realize we don’t have all the data we need to truly evaluate the campaign. It’s obviously important to spend time up front considering how you’re going to measure. But campaigns have launched without this forethought.

In summary, to effectively measure:

  • Always plan ahead.
  • Avoid jumping to conclusions.
  • Don’t measure impact too narrowly.
  • Don’t try to measure EVERYTHING.
  • Definitely don’t try to measure everything at once.
  • And finally, it’s one heaping, but rounded scoop for every two cups of water. Turned to the strongest setting. Pour, drink and enjoy!

I would love to hear about other common missteps to measurement. Mostly so I don’t have to make them myself. Any tips on coffee brewing are welcome as well!


3 Ways Sustainable Brands Could Help Conscious Consumerism Make a Comeback

When shopper were asked why they aren’t buying from socially responsible brands anymore. The biggest problems: They can’t name any and think they’re too expensive.

Read More

The Pandemic Inspired a New Wave of Conscious Consumerism, But Only Temporarily

A rush of communal feeling that drove purposeful consumer decisions was replaced by a grinding pandemic, failing finances, and a contentious presidential campaign. Will an optimistic COVID recovery alter our path?

Read More

In the Wake of the Pandemic and Protests, Purpose Is Suddenly a Super Power.

Amidst the COVID-19 pandemic, companies that have purpose built into their bottom line are the most likely to remain standing. Suddenly, having a purpose is a superpower.

Read More

THE types of companies we help


You have a story. You are making a difference. We'll provide you with top-shelf marketing resources at affordable rates so everyone will know who you are, what you do and how they can help.

Learn More

Social Business

Warby Parker. TOMS Shoes. Patagonia. Starbucks. Ben & Jerry's. We'll make sure your social business is the next case study, the next media sensation, the next "best place to work".

Learn More

Health + Wellness

There is no greater challenge facing us than the growing healthcare crisis. We help wellness-minded orgs drive change, communicate effectively, and achieve healthy returns.

Learn More

Conscious Consumer Spending Index


Socially Responsible Spending Experiencing Massive Growth, According to 11th Annual Conscious Consumer Spending Index

Learn More