• Facebook
  • Twitter
  • LinkedIn
  • Pinterest
  • Email

When The Research Is Wrong

By Steve McKeeTuesday, May 15, 2012

"Maybe companies should pay less attention to what small 'focus groups' say..." That's how the brief article in the The Wall Street Journal began. It had me at hello.

Market research is notoriously misused and abused. The article, citing a forthcoming study (yes, that's ironic), went on to say:

To predict reactions of a broader public, 31 subjects interested in quitting smoking were recruited for a focus group to evaluate samples from three antismoking ad campaigns: A (a humorous effort that empathized about the difficulty of the task), B (more serious but also empathetic), and C (lighthearted). Ads in each campaign ended by posting a hotline number for help quitting smoking. Researchers measured the viewers' neural activity as they watched.


The viewers, rating the ads for their power, helpfulness, grabbiness and so on, ordered them B, A, C. But levels of brain activity in a region tied to behavioral change suggested the ranking C, B, A. That ranking better captured real-world performance. For example, Campaign C caused a 32-fold increase in calls to the hotline, in contrast to only a threefold rise for Campaign A.


Having recently conducted research on this very topic, this hit home. We were not using our focus groups to "test" ads (a fundamental abuse of the methodology); we were using them to explore smokers' thoughts and feelings about their habit and the role it plays in their lives. And we sensed at various points of the discussion that they were lying, perhaps even to themselves.

Not that the groups weren't valuable; on the contrary, we picked up insights that led to what we believe will be a breakthrough approach. But I'm convinced that the more we as marketers learn about consumers, the less we really  know.

One of the the critical mistakes that can come about as a result of a loss of nerve in a struggling company is over-reliance on market research. It's easy to hide behind what consumers say on the surface, not to mention the authoritative heft of a market research report. But it's dangerous.

No one believes in research more than we do. But it must be handled correctly and interpreted in the proper context. Bad research won't produce good strategy any more than a polluted well produces fresh water. It could, in fact, hasten one's demise.


By Steve McKeeTuesday, May 15, 2012

This smart looking guy is our agency’s president. When he’s not sitting in his not-so-oval office looking official, he’s busy...

*required