The trials of testing

How we learned to love consumer testing with the help of a little mind reading.

By Steve Barrett, Executive Creative Director

Creative professionals have a lot of names for focus groups and eye-tracking studies. Most of them are NSFW.

The perception is that testing kills great creative work. Focus groups lean towards the safe—the average person reacts more positively to familiar concepts. And eye-tracking studies can be frustratingly vague: So people spent 2.3 seconds staring at the violator? Does that mean they like it? Or is it just confusing? Maybe it’s so awful you can’t turn away, like watching Paris Hilton milk a cow?

There’s some truth in the griping, but when it comes to a brand reinventing its packaging—often the most visible face of the brand—there’s zero room for guess work. These kinds of tests and focus groups can prevent a consumer revolt.

Our recent packaging redesign for Krusteaz—an established baking and pancake mix brand on the West Coast that’s steadily carving out an audience nationwide—took us through testing that introduced us to some new methods and affirmed the value of traditional models.

Focus groups—those gladiator pits where many a designer’s dreams have been put to the sword—often bring to light nuances you’d never dig up in a tastefully appointed studio in a coastal metropolis. One example from our testing experience was how people in different regions of the country define “authentic.” Authentic is one of those unicorn marketing buzzwords that everyone’s chasing. Every consumer survey insists consumers—especially the coveted Millennials—demand authenticity.

But “authentic” in Seattle means something much different than it does in Atlanta, where it means brands that consumers had grown up with—namely, big corporate names like Betty Crocker and Duncan Hines. Essentially, the one definition of “authentic” that spans regional differences is not pretending to be something you’re not. Those Southerners were on to something. (Compare with Mast Brothers chocolate, a hipster brand that was recently outed for buying commercial chocolate, melting it down, and reselling for $10 a bar.)

The most exciting part—and yes, I used the word “exciting” to describe consumer research—were the rounds of neurological testing. Neuro testing is similar to eye-tracking surveys, but the subjects are hooked up to an apparatus that measures emotional reactions. We could not only see what they were paying attention to, but also how they felt about it.

This is where we really went down the rabbit hole. In between testing rounds, we’d rearranged the copy so it flowed logically from top to bottom. You could read the package out loud and it made sense. The tweaked design went in front of the neuro audience and the results came back markedly more negative. It turns out that the words you place next to images have a kind of synergy—get it right, and they both work better. Get it wrong, and that consumer’s reaching for the competition on the shelf. We went back to the original, non-linear copy and slept better at night knowing the testing had saved us from a serious unforced error.

It’s natural for agencies to feel a little trepidation at the thought of testing. After all, your work and reputation are on the table, in full view of clients. Luckily for us, in this situation we were working with a client team who had a sophisticated grasp of the strengths and weaknesses of each testing methodology.

Traditional testing can be extremely valuable for course correction and validation. You can’t let focus groups lead design, but testing can save you from making a huge mistake. We never walked into the room with anything we couldn’t stand behind, or didn’t support our client’s brand, but the experience allowed us to arrive at a final package design that measurably improved our client’s chances of winning new converts in the grocery aisle.