Why Creativity is Essential to Data

There’s a common misconception that engineers and data scientists are “number crunchers,” and are not creative. They use the left side of their brains, as the popular opinion goes, to present stodgy, black-and-white data, trends, ideas, etc. The “creatives” reside on the other side of the company, immersed in the innovative and artistic work – design, music, advertising, PR, etc.

The reality, however, is that data requires equal parts number-crunching and creativity. There is a lot of original thought needed to put together different data sets to find interesting outcomes. In fact, determining how to “tease the signal from the noise” of large integrated data sets is as much art as science. Complicated problems sometimes require innovative and sophisticated solutions where a by-the-book approach taught in programming or statistics classes won’t cut it. I argue that data scientists must use both sides of their brain to solve today’s business problems.

“Today’s creative data strategists are more than data crunchers,” a recent Fast Company article concurs. “Creative data strategists develop an intuitive feel with the data–where to collect it, which way to write the data schema and how to analyze it. We often work alone to crack the strategy, because the work itself requires shoot-to-kill methodological precision.”

There was similar sentiment at VentureBeat’s Data Science Summit last December, as panelists, including Jeremy Howard, the former president of data science competition site Kaggle, discussed creativity as a key trait for people working with data. “Some people who do well tend to ‘spend all their time being creative’ as they comb through and pull ideas out of the data they’re given,” Howard said.

We are witnessing this within our own portfolio companies as well. One prime example, ClickFox, is solving customer experience challenges and presenting new opportunities for major retail banks and communications companies. As COO Bill Hawley explained at our B2B IT Forum on data analytics, ClickFox is saving major brands more than $100 million annually by analyzing data across all customer interaction points and providing fresh insight on the “customer journey.”

Finally, the need for true creativity is recognized by the data scientists themselves. According to the 2012 Analytics Professional Study, analytics professionals – the folks who do data preparation, programming and analysis of business data- have a “clear psychometric fingerprint.” The study found they value out-of-the-box thinking and embrace a deeper understanding of their surroundings. Leading business and technology consulting firm Booz Allen Hamilton has written a book called The Field Guide of Data Science which suggests the top two desired qualities for a data science team are curiosity and creativity.

With all the hype and publicity around “big data” these days, data scientists are getting a lot of attention and are in high demand. I’d like to think that all this scrutiny will also bring a greater understanding of the skills actually needed to perform this often challenging work – with creativity at the top of the list. So-called “creative” types should not be intimidated by analytics, but in fact should embrace it as their strengths are highly valued. As we continue to shift from a business world that relies on intuition and gut feel to one driven more by data and insight, it is the collaboration of analytics and creativity that will bring new innovation and competitive advantage.