google financeHow can market researchers harness big data without falling into privacy traps.

Big data is one of the latest tech buzz phrases, and hardly a day passes when you don’t see big data mentioned on your social media stream. The phrase describes the vast amount of information that’s being generated on planet Earth every day; it’s enough to make your eyes water.

This volume of data presents an interesting and attractive prospect for the professional market researcher. In our ongoing quest for data, surely we can re-use big data in small market research projects to economise and get a bigger picture?

Well, yes, we can. But in order to leverage big data, we need to understand the limitations of knowing almost everything about everyone.

Big Data: the Challenge

As the name suggests, big data is all about massive amounts of information. Gigabytes, terabytes… and then some.  Last year, it’s estimated that we created 2.5 exabytes of data every day; that’s two to the power of 60 bytes. A full 90 per cent of the data ever produced was created in the last 24 months, and we’re churning out more and more ones and zeros by the second.

The researcher’s job is to filter that data and extract only the most meaningful facts and figures. Sounds tricky – and it is. What do you do with more information than you could ever possibly read?

But even more than that, the market researcher must create meaningful connections. That means interpreting big data and coming to conclusions about what the figures really mean. Researchers must somehow process data without coming to conclusions that aren’t there; they need to follow their instincts without forcing the numbers to fit a pre-defined hypothesis or a foregone conclusion.

Suddenly, access to all of this data seems more like a curse.

Finding Value in Big Data

In any market research job, the most important influencer is the person; the customer. Big data gives us information about millions of people on social media, on the internet, in stores and in our local area. We can slice and dice this data in many ways to look at demographic differences, shopping preferences and more  -all without identifying a single individual.

Using this big data effectively means knowing the risk involved:

  • Big data should never be used to identify people individually.
  • Market researchers must remain professional – resisting the urge to pry into others’ lives.
  • Data needs to be filtered and qualified; raw numbers mean nothing.
  • Researchers must take care not to imprint their own views on big data.
  • There may be conflicting signals between different chunks of big data, and it’s our job to work out why – not to use the conflicted data without questioning any of it.

The Future of Market Research

As the internet develops and we move more of our data into the cloud, big data will play a bigger role in marketing and market research. Brands will come to rely on it to inform advertising campaigns. And all of us will be responsible for respecting personal data, not misusing this information for dubious marketing gains.

Article WritingBy Sam Wright

Sam Wright is a writer working in association with Brand Republic.