Avoiding the ML hipster trap

Machine Learning hipster effect

Machine Learning is very in vogue at the moment. I feel that a pressure some junior data scientists and engineers feel is the need to do ML just to be a cool hipster, or as a friend of mine calls it ‘the ML hipster trap’.

What is the ML hipster trap?

This is simply where you use ML where you don’t need it. Or obsess over model performance without realising your job is to add value to a business or to a governmental organisation. A great discussion of when Machine Learning matters comes from Erik here

Tip: Ask yourself is there a simpler solution than ML? Does ML justify the investment for this project? Do you have a product without your ML?

Have you exhausted all the low hanging fruit of analytics?

I once in a project a few years ago, simplified what originally looked like an ML problem to a SQL query with some basic counting. It was 10x more reliable, I could hand it over easily and it integrated well with the reporting pipeline we had. It’s worth thinking about when you have counting problems first, and how to to invent and simplify.

Tip: Don’t be scared to look for a SQL query as a solution. If you add value you’re doing well.

Data Science is more than just supervised learning have you considered other approaches?

Currently at work I’m using some survival analysis, in my career so far I’ve built models with Ordinary Differential equations (some would say this isn’t Data Science, I think they’re being misled by ML hipsters), genetic algorithms  and many other kinds of statistical and algorithmic approaches. I don’t feel this makes me a worse Data Scientist just because I’m not using Deep Learning.

Tip: Use the right tool for the job, and don’t feel ashamed if you’re not using Deep Learning.

In short: Don’t let the current AI hype make you use the wrong tool for the job. An increasing part of growing as a Data Scientist or Engineer is to appreciate that it’s about adding value to those around you, not just using the fanciest toys.