What once was a shortage of coding and software engineering expertise has now translated into a shortage of skills in artificial intelligence and algorithmic engineering – machine learning talent.
According to a recent survey on enterprise AIOps adoption , 67% of enterprise IT organisations in the US have experimented with artificial intelligence (AI) and machine learning (ML) for data management and incident remediation.
What’s more, global research and advisory firm, Gartner, fully expects that artificial intelligence is expected to create more jobs than it replaces by 2020 . AI is moving fast and enterprises need new talent today, not tomorrow; and not just any old talent.
Here, I’m going to discuss some of the things I’ve learnt and some of the practical questions you can pose to uncover and secure the talent you need to help both your enterprise and potential employees succeed and excel.
No skills to pay the bills
Skills gaps are cited as among the biggest hurdles to AIOps adoption and implementation, and a recent EY survey of 200 senior leaders found that 56% see talent shortages as the single, largest barrier to implementing AI in business operations in 2018.
It’s clear that finding machine learning engineers is not an easy task. They’re a bit of a unicorn, combining engineering fundamentals with data modelling and statistical analysis. But, with the right framework, it’s entirely possible to build a team that has the right mix of data science, engineering know-how, and even a little robot emotional intelligence .
Math in the machine
To start, machine learning engineers need a deep expertise in predictive modelling and statistical analysis. I always look for engineers who can combine core engineering fundamentals with the ability to see patterns in data and translate that into action.
Not only do they need to be able to manipulate code and build software, but they also need to understand how mathematical models can create insights, and how those insights can drive action in order to establish a candidate’s potential and knowledge.
Some examples of the types of practical questions I would pose during initial interview rounds include:
- When should you use classification over regressions? Setting up use-case driven questions like this help us understand how a candidate builds mathematical models. Fundamentally, classification is about predicting a label and regression is about predicting a quantity
- Do you have experience with Apache Spark or other public machine-learning libraries such as TensorFlow etc.? Ultimately, we want to make software that’s scalable. This is the advantage of a machine learning library.
Today’s AI engineers must be proficient in modern tools to build enterprise-grade solutions. The next challenge for a true AI engineer is the ability to solve the big problem of clean data. Creating datasets that are rich, contextual and clean provide the best results for an artificial intelligence solution, as AI relies on data for decision-making.
Some further questions based on this might include:
- How would you handle an imbalanced dataset or how do you handle missing or corrupted data in a dataset? These questions get at the importance of building models using clean data. At its core, AI must use clean datasets for insights. Otherwise, the actions will be incorrect or useless.
Some other examples of questions I may ask in interviews include:
- Provide an example of why you would use quicksort versus binary? These are both algorithmic options for organising the data, and help illustrate a use case of two extremes. The AI engineer can employ either to derive actions from
- How do you clean and prepare data to ensure quality and relevance? This question is relatively tactical but helps us understand the critical processes the engineer employs at the most critical points in AI engineering.
In short, I look for engineers who can apply statistical analysis to data for enhancement, cleaning and processing – but who also understand the practical ramifications of data as the engine of software.
At its core, machine learning engineering sounds like a natural evolution in software engineering.
We have, heretofore, been looking for engineers that can translate basic human requests into some sort of computer-based action. We’re now simply trying to anticipate that human request with data. The basic mechanics of data ingestion, analysis, interpretation and action are the kind of actions that humans take every day.
Turning these steps into an action that a machine can take unsupervised is an entirely new challenge. This is where statistical analysis and predictive modelling come into focus.
The true machine learning engineer is both a geek and a craftsperson. She’s both a math nerd and a builder. Our talent search helps us identify candidates who live at this intersection. And who can help us continue to win the race for efficient, effective technology.
Bhanu Singh, vice president of engineering, OpsRamp