Big Data: Love the Mission, Hate the Moniker

I enjoyed speaking on a big data panel at the E2 Conference on Wednesday in Boston, but the more I participate in these events, the more I agree that “big data” is a silly marketing term, a meaningless catch-all that provides answers to nothing. Perhaps it is needed to make a distinction from traditional data analytics – as people have formed their own perceptions of what that means – but there are a lot of misconceptions about what big data can provide, and how it can provide it. And in my experience, most companies that think they need a big data solution probably don’t, at least not yet.

Big data works for the companies with massive amounts of fast-moving data that comes from thousands of different sources. For most organizations, though, the answer lies in data analytics – examining data and drawing conclusions to make better business decisions.

This sounds simple, but the culture-change aspect of data analytics represents the biggest obstacle in my view, and many of the audience questions at E2Conf focused on cultural issues and company buy-in. Of course, there’s no silver bullet for this; a company simply must embrace a more data-driven philosophy for any data technology initiative to be effective.

It reminds me of the Moneyball craze during the early 2000’s, when Major League Baseball teams started to figure out that statistics could be used to build a winning ballclub, rather than relying on a scout’s stopwatch and gut. There was initial backlash against the “stat geeks,” but today every team has an advanced statistics department that helps general managers make better decisions. This was bringing data, and insights, to bear on decisions in a way that turned conventional wisdom on its head. It was not “big data”, but it led to big changes. It never would have started had one GM not been open-minded about statistics. His success forced others to follow.

To me, we’re just getting going in the “big data” space, and it’s going to take years for the potential of the technology to play out. But this is nothing new – the cloud and the Internet needed time to develop too.

But companies do need to pay attention to what’s happening with data analytics now. It might not make sense for companies to throw huge amounts of money at big data today, but getting started is a good idea. There is progress being made and there are companies getting to the next level of insights that can move the needle. Just like with the Internet, the companies that don’t buy in are going to wake up in 10 years and be heading toward irrelevance.

But as for the term “big data” itself, it means too many things to mean anything. Sometimes it’s about having the necessary tools to handle huge amounts of data, sometimes it’s about developing models and algorithms to apply to industry problems. I am a major believer of what smart data analytics and big data can do for an organization, but I’m done with the buzzword.