Good analytics is a combination of art and science, where the skillful analyst combines an entire matrix of prior knowledge with good judgment about authentically meaningful qualitative differences. Like software developers and computer scientists, there are a lot of self-styled big data analysts and “data scientists” out there, most minimally competent, some adequate, and a few good ones—the analytics wizards.
The analytics wizards do the work of ten, perhaps a hundred adequate analysts. If we are lucky we may work with one or two of the good ones in our careers. It is easy to recognize them from their work, but it is much harder to explain what they do that separates them from the rest.
Like the Pinball Wizard described in the same-titled song by The Who, good analysts seem to play by intuition. But looks are deceiving. What appears to be intuition is explained later in the song: “ain’t got no distractions/can’t hear those buzzers and bells/don’t see lights a-flashin’.” Indeed the hallmark of the analytics wizard is the ability to filter out the noise, and not get distracted by it. Keeping the ball alive and getting the replay is the object of the game, not lighting up the board or making more noise than everyone else.
What is the sound of noise not happening?
So how do the analytics wizards do it? A fundamental is that they know the difference between inputs and outputs, and don’t confuse the two. So at the very outset, good analysts realize that filtering out the noise is not the result of what they do, but the driver. Analytics wizards are masters of the robust and enduring theories that govern the phenomena they are studying, and can frame the goals of their analysis according to these theories. Note that these theories are not the same as “book knowledge”, and it doesn’t matter whether they are developed by academic study or acquired as know-how. What does matter is that good analysts compare everything they see to these theories, which allows them to classify phenomena quickly, and determine which surprising things that come up are unexplained, and which are the result of errors in the system, for example, poor observation.
Good analysts are likely quite skilled at mathematical modeling, but it is a necessary and not sufficient condition for being a good analyst. A math or code jock who is nevertheless ignorant of the enduring theories can still produce results, but cannot tell whether the results are good or even relevant. Rather, their criteria for good lie in some obscure statistical “goodness of fit” or parametric test. Analytics wizards have internalized some of the theories into their own mental models of how the systems of the world work (“stands like a statue/becomes part of the machine/feeling all the bumpers/always playing clean”), and are constantly comparing what they see to their own models. If they score a hit, it seems easy, but really they are testing everything they see against a script. The difference is that it’s a really, really good script.
What else do the analytics wizards differenly from the adequate ones? Find out by reading the full text version on Forbes.com.
