Big Data and Market Research

Big Data have made quantum leaps in the amount of consumer information gatherable from more sources than ever before. Organizations are looking at data sets as diverse as social media chatter, Point of Sale and other “digital footprint” data, from the Internet of Things (think smart appliances like the Nest thermostat). The challenge is trying to make sense of these data. Brands, social organizations and even political campaigns are scrambling to keep up with this fire hose of information about their audiences.

Unfortunately the emergence of Big Data has, in many instances, eclipsed the use of in-depth, qualitative research methods that are based on “Small Data” — meaning small sample sizes. According to Wharton Marketing Professor Peter Fader “Today [qualitative market research] is in the back seat, if companies are doing it at all. What happens more and more now, Fader notes, is “people say, ‘Let’s just try stuff and see what works.’ It’s a widely held belief that it has become much easier to test things in market [i.e., by saying,] ‘Let’s put out our concepts and see what gets clicked on the most.’ That can help you determine a winner, but it doesn’t help you design what would have been the best. By doing careful research and determining the underlying drivers that cause people to click, we can develop better products and services.”

However, some established companies like Procter and Gamble are finding innovative ways to integrate Big Data and qualitative research. “Even Wal-Mart is spending lots and lots of time doing analytics and data mining,” says Bradlow, “but they have not gotten rid of ethnographic studies, surveys and consumer psychology…. there is still nothing better than understanding the psychology of the consumer, and that’s harder to get.”

As PR strategist Sean Donahue writes, it’s “more important than ever to apply qualitative logic and human reasoning to online analytical models….subject matter expertise and deep knowledge will matter more than ever before given the rise of big data.” Context is key in making sense of Big Data. “This means committing more time, asking more questions, consuming more content and never losing sight of the fact that data without actionable insights is meaningless.”

Strategy firm Fresh Squeezed Ideas notes three ways to make use of Big Data as a complement to qualitative research:

• Use unstructured data as your jumping off point

  Social listening won’t give you all the answers about your users, but it can point you towards the insights you need to explore further. Quantitative data can help narrow the focus of your qualitative research.

• Pair Big Data with qualitative data points

  The impact of quantitative findings is stronger when it’s shown side-by-side with anecdotal comments and images, helping “bring the story to life”.

• Use a Big Data analysis approach to survey-based research

  Even a large-scale audience segmentation can reveal patterns of behavior that yield unexpected results.

Are you pairing Big Data with more traditional forms of qualitative research? What do you think the future holds for consumer research in light of this unprecedented access to information? Let us know what you think in the comments below.

Keep up with the latest insights from SayWhat Consumer Research. Follow us on Twitter and Like us on Facebook.