Making numbers count

Likewise, a flurry of GDPR-related emails is waking us to the sheer amount of companies that have an interest in us.

As we develop and refine our quantitative approaches at The Big Picture, we’ve been thinking about just that. How can we use the tension between an increasing thirst for data and the responsibilities of the industry, as an opportunity to take stock and change? How can we make data more transparent for participants and clients alike?

We recently ran an online poll with our clients. It had a simple question – ‘How much do you trust your quant data?’ The views you – our clients – shared, echoed our ambitions.

These comments were pretty typical. So what’s gone awry? Historically quantitative data was seen as the preserve of robustness, giving black and white illumination on marketing issues.

It seems to us that as researchers, we’ve lost the connection between the number and the person. We forget that data reflects behaviour, and that it’s not just about numbers in their own right. To quote The Prisoner, our participants would be right to shout “I am not a number!”

At The Big Picture we place great emphasis on getting to the heart of real behaviour. We try, as much as we can, not to ask people what they do, (or why they do things), but to observe what they do. And our approach focuses on eliciting and capturing immediate, intuitive responses to new ideas, packs or brands.

Within the industry there’s still a great reliance on direct questioning. There’s a tendency to ask people what they do, and to rank their emotional responses using rational, often numeric scales. Even the use of emoticons in surveys still masks the numbered scale behind the different smiley faces.

Why, when the market research world has been embracing the thinking of Daniel Kahnemann and his ilk for the last decade, are quantitative surveys still based around explicit questioning, when the drive from clients is all about uncovering (and measuring) the implicit?

To do so, is reducing the people we’re talking with to numbers. It doesn’t actually value their behaviour, or their emotional responses, but aims to reduce them into nice, neat numbers (preferably percentages to show to the board!) And in so doing, risks losing all the nuances and insights that can truly make the difference in launching successful brands or designs, because you’re failing to appreciate or capture the reality of the consumer’s world.

Jeff Bezos, founder and CEO of Amazon would seem to share our views:

We’re not sure that answering our clients’ problems using our intuition and instincts is quite enough, but we do share the view that quant on its own will tell you only part of the story. And quant which is based on explicit questioning is even less likely to get you to the real story.

In developing our own quantitative products, we’ve used the same philosophy and approach that we employ with our qualitative work – to use approaches and questioning techniques to uncover real behaviour and elicit a more System 1 response. In addition, we always include other data sources, be that a qualitative component, or eye-tracking for example.

We’ve already seen the benefit of this: in the redesign of Genius gluten-free bread, the mixture of qual insights and behavioural quant helped us reach a much deeper and strategic understanding of design performance. And eye-tracking interviews showed the difference that a change of colour made to shoppers’ engagement when we explored the new packaging for Jus-Rol’s Bake-it-Fresh range.

Don’t get me wrong. Numbers are great. We love numbers – as long as they’re based on behaviour. And as researchers, we need to be able to turn these numbers into a compelling story for our clients, for the success of their brands. But numbers on their own? They just don’t add up.