Big Data, fundamentally, is about hypothesis testing. Sure, you can use the term to talk about large data sets of structured and unstructured data, multiple data sources, etc., but all of that is ultimately leading to better decision making through the analysis.
The shift that is at the heart of Big Data is the change from using data sampling to run analysis to using entire data sets. As prescribed by Moore’s Law, computing power is now affordable. So is storage.
With all of the available resources, there’s not a need to compromise by using a data sample… just crunch the whole thing! But then again, why are you crunching? Ultimately, it is because you want to see how one thing leads to another, which by definition is testing a hypothesis. As marketers, we are curious by nature and when we are given a set of information (or data), we want to uncover as many patterns and trends as possible to make sense of the information.
Using big data we can make determinations such as, people who buy brown shoes buy black shoes next. People who buy gas on a Friday almost always stick to Fridays. Big data gives us marketers the power to make predictions and then determine if these hypotheses hold water.
The difference between marketing of past and marketing of present is that marketing of the past was primarily based on intuition and gut. Today, marketers can use big data to conduct hypothesis testing to determine if their intuition was correct – and, then ultimately make predictions about customer behavior using the information we have.
Big data holds the key to providing many answers and it’s the responsibility of the marketer to use big data to make more informed marketing and business decisions.