Big data. Big deal.
In my last UpMarketing post, Dart and science, I described how combining quantitative and qualitative research with good old-fashioned gut instincts can drive results. This post examines a related topic: big data. Everyone’s talking about the mountains of data at their fingertips, just waiting for analysis and action. But it seems that no one has figured out an effective way to begin excavating to find the hidden treasure.
This amassing of data has ramped up in recent years due largely to new streams of information from shopper loyalty programs, Internet activity, and social media, among others. The hoard of data is growing so fast that it has been estimated the volume will now double every two years.
It’s like a major winter snowstorm that blows into the mid-section of the United States. Some municipalities are overwhelmed and literally shut down by the enormity of the snowfall. Some manage to move the snow out of the way, albeit too slowly to keep up with the rate of precipitation. And others meet the weather threat with over-exuberance, battening the hatches for a blizzard that never materializes.
Such is the case with Big Data. Many marketers suggest that they need more data to determine how to effectively mine the original data. Some walk the road of denial, affecting to believe that time spent filtering the data will not produce meaningful insights. A courageous few are beginning to isolate islands of data that, once strung together, can lead to meaningful conclusions. To me, the mystery hidden inside Big Data is what makes it compelling. Maybe it’s because I’ve always enjoyed discovering simple correlations between discrete data points that I throw in with those who eagerly forge ahead into the unknown. And because the nature of the discovery is undetermined at the outset, it’s like an archeological dig.
It might seem to some that mining of data should be as simple as collecting it. After all, what’s really involved? You have the information, you organize it, you read it, and you act on it. The problem is the volume. No one has a shovel big enough to figure out what’s in the data before new data comes pouring in. But waiting until appropriate excavating equipment is built is a waste of valuable time — and terabytes. Waiting until the analytical methodology is absolutely perfect is really the same as deciding not to analyze at all. Surely scratching the surface and finding some initial nuggets of fresh insight is preferable to going home empty-handed.
The key to any productive data mining is to determine ways to quickly and efficiently filter out the meaningful insights. Sometimes that means using traditional methods rather than “big” analytics. And just when you think you have more than enough data to mine, you still have to determine outages — a.k.a. missing data. Often a third-party source combined with existing data sources will actually help organizations more quickly align decision-making processes and lead to more immediate action.
All that’s left now is to get started. Prepare a list of questions needing answers. Prioritize the questions, and identify data sources, whether home-grown or third-party. Then let the archeological dig begin. It’s amazing what you can discover by removing even just the topsoil. If you never begin digging, however, you’ll never find any groundbreaking insight. Instead, all you’ll have is unbroken ground.
Hamacher Resource Group, Inc. (HRG) Vice President Dave Wendland, a 20+-year retail industry veteran, is a popular presenter and discussion facilitator available to speak at corporate and association events on a variety of retail-related topics. HRG is a research, marketing, and category management firm specializing in consumer healthcare at retail. Product manufacturers, healthcare distributors, retailers, technology partners, and others rely on HRG for strategic and creative solutions to help build their business. Learn more at www.hamacher.com.