There are plenty of data science blogs online, but they rarely explore the day-to-day reality of working with customers; instead, they choose to concentrate on methods, tools and results. I want this blog to be different, so I’ll be writing a loose series of posts on what my work actually involves on a daily basis. The aim is to give you an idea of what it’s really like to be in the so-called “sexiest job of the 21st century” – without all the glamour and glitz.
When I began my university education almost 15 years ago, things were bleak for statisticians. Statistics had only survived as a degree program thanks to a large-scale promotional campaign. And for stats students, day-to-day university life wasn’t always easy. Pitied by the mathematicians and physicists, shunned by the med students, and simply ignored by the rest, we ranked among the obscurer categories of undergraduates. The mere fact that statistics was offered as a degree program in its own right was enough to provoke incredulity – so much so that a T-shirt was eventually printed, intended to answer, preemptively and with a statistician’s humor, the question so commonly posed at undergrad parties and during family visits:
“Yes, it is a real degree – with a probability approaching certainty.”
The world has changed in those 15 years, and statisticians are now more in demand than at any time in the past. Some in our profession who have attained rock-star status may soon even see their palm prints cast in concrete for posterity. We’re often perceived as founts of wisdom – like druids with long, flowing beards, stirring at cauldrons of a magic potion capable of making problems of any kind simply disappear. Given the esteem in which we’re now held, there are doubtless some among us who wish they could have been the coolest kid in school. (Parties would, presumably, have been a lot more fun …) And, doubtless, there are others who would prefer it if all the hype died down and a different job topped the “sexiest” list.
There are two sides to every coin and, on balance, I’m happy with the way things have changed – not just because my inboxes on business networking sites are full of inquiries from headhunters, but because a crucial realization has dawned generally in recent years: Data is valuable. Data therefore needs to be handled with care; it needs its own ecosystem, so to speak. Data has to be integrated in the right way if you want to tap into its value. Here, our work as statisticians (aka data scientists) is just one of many elements in a greater overall value chain. Nonetheless, this work is highly important in identifying potential value and exposing states of affairs that are less than ideal.
With artificial intelligence lining up to be the next big thing in our field, the everyday reality is still that many businesses have yet to get to grips with the basics. By this I mean that they need to understand the value of data, to change their systems and procedures accordingly, and to carry out initial analyses that can deliver a deeper understanding of their business processes. I’ve seen for myself how commonly held perceptions or assumptions that may have been the bedrock of decision-making at a business for perhaps an entire decade can suddenly be called into question through a basic analysis of data that has been consolidated for the first time.
Results like these have also shown that statistics (or data science) is not an end in itself. A complex problem might call for a sophisticated solution, particularly in situations where the results of statistical analyses need to be applied to a production environment – but this is not always the case. At the end of the day, what matters most is that the issues and areas addressed are brought together within a unified data culture that is embraced by the entire organization. At the core of such a culture is a cycle of analyses, findings and improvements that should be driven by a data scientist. For me personally, playing an active role in this way is a key objective in my daily work.
Contrary to my original plan, I’ve not actually discussed my daily work at all and the title of this post has been misleading. I’ll make up for it though, I promise …