AI in Industry Panel - Data Science and Machine Learning in the Real World

Interest in artificial intelligence has exploded within the last few years - for many companies, it’s become clear that adopting AI or machine learning techniques will be key to competitiveness within their industry in the coming years. According to a Deloitte study, 81% of early AI adopters say it’s very or critically important to their business success today. Yet, incorporating AI or machine learning (ML) into a company’s existing business model can be anything but straightforward. 

A company wishing to adopt AI, ML or deep learning has several challenges to navigate: how to find and recruit the necessary expertise, how to avoid bias and errors, how to manage project deliverables / expectations, and incorporating AI into the company’s vision. At a recent “AI in Industry” tech talk, several industry leaders got together to share their insights into how companies can efficiently adopt AI. 

Setting a Vision for AI

The first challenge is setting a vision for the use of AI within the company. Most companies already have a vision or mission statement for what they hope to accomplish, but often it doesn’t yet incorporate the role AI will play. The industry speakers at the tech talk agreed it’s important to set expectations for what’s possible - avoiding unrealistic expectations or misconceptions of what AI can do. 

A good strategy is to tie the goals directly to the existing goals of the business. For example, if the company is in the medical devices space, they might ask how machine learning can improve client/customer satisfaction or how it could improve metrics related to accuracy or ease of use. The product should be aligned with the client’s needs. In many cases, machine learning isn’t about enabling them to do something that couldn’t be done before - often it’s about doing it faster, better or cheaper. 

Confronting a Skills Crisis

With the sudden and intense interest in data science and AI, there’s a shortage of skilled expertise available. Sixty-eight percent of the respondents in the Deloitte survey indicated they had moderate-to-extreme AI skills gaps - mostly in AI researchers, software developers, and data scientists. 

Serkan Piantino, of the AI infrastructure company Spell, mentioned he’s seen engineers with little prior AI experience ramp up quite quickly through fast-track online curriculums. He sees tools as critical aids towards adopting AI, saying, “Something that was very sophisticated research is now becoming pretty accessible, even to people without formal training in programming. I think that’s going to be a big story in the economy at large.”

Another speaker at the tech talk, Bonnie Ray of Talkspace, said she thinks it’s important to have “guardrails” around more automated techniques to prevent the project from losing track of the underlying assumptions that are important to the model. She says it’s important that the team have knowledge about why things work (or why they don’t) and under what conditions. Machine learning work is different from traditional engineering work, and it’s important to have a good understanding of the underlying data set. 

Oversight: Avoiding Bias and Errors

Another challenge with adopting AI or ML is it’s important to have informed oversight of the project - to be able to tell whether the algorithm actually does what you think it does, or whether it’s just producing an unintended result due to a poor input dataset. Alex Poon of AiCure noted, “Machine learning is not just a new technology, it’s somewhat a new way of thinking. If you don’t know the underlying dataset, the problem you’re trying to solve, it makes the effectiveness of your algorithms a lot harder.”

He says 95% is machine learning, and 5% is supervised learning. You need a large, well-labeled dataset - if you don’t have a good dataset, you can end up with what’s known as “garbage in, garbage out.” Feeding an inadequate dataset to a machine learning algorithm will just result in a model that does a good job of mimicking the wrong data. Therefore it’s important to have a way to inspect the decision making process and incorporate a feedback loop which can be used to iterate and improve on the technique.

The metrics for assessing the model or the project’s success need to be very application specific. Without the correct metrics, you can end up optimizing for the wrong objective. Bias in the dataset or errors in interpreting the results are a common problem in machine learning projects. “Explainability” should be part of the default toolkit - the tools should help the team understand how and why the experiment worked or didn’t work. 

This theme came up later in the talk as well, when asked how to deal with uncertainty in machine learning projects. The participants noted that a negative result is still a result. A “failed” experiment isn’t necessarily a problem if you understand why it didn’t work. Boris Daskalov of HyperScience recommended building prototypes and working on many things in parallel. If you have some short-term low risk projects going in tandem with long-term higher risk bets, there’s a better chance that some of them will pay off. 

Biggest Misconceptions about AI / ML

AI industry early adopters often have to deal with big misconceptions about what AI and ML are capable of. Due to news media that often hype and exaggerate the state of AI, people new to the space often think it can work miracles. People often interpret AI to mean general AI (AI seeking to mimic human cognition) - but that’s not what most AI specialists in the industry are working on. A machine learning algorithm that does a really good job of mimicking a dataset isn’t the same thing as general decision making. 

Another common misunderstanding is when managers think an AI or machine learning project can run on the same schedule as a traditional engineering project. This typically isn’t the case, because AI/ML projects have a lot more unknowns. Oftentimes there’s an exploratory phase where the team attempts to determine if it’s even possible to solve the problem with a machine learning solution. A good project management technique is to build in time for this exploratory phase in advance. 

Another misconception is thinking that AI is easy; in general it’s not. For example, big, complex problems like self driving cars take a long time - longer than some people predicted it would take. Scoping the problem to smaller, more manageable tasks can help a team achieve the objectives more quickly. AI also isn’t perfect - the errors need to be handled or managed in some way. 

Managing Expectations and Project Uncertainty

Companies adopting AI / ML within a traditional engineering organization face the challenge of learning how to manage a project that can’t be managed in the same way as their previous engineering projects. These types of projects have more uncertainty, and therefore it can be difficult for managers to know whether the project is making progress, how long it will take, or whether it will be successful. 

One way to monitor progress is to ensure your tools include a way to check-in commits to a repository - so that changes to the model or experiment can be tracked as well as correlated with improvements to project metrics. As mentioned earlier, it’s also important to reserve time in the project schedule for initial exploratory analysis and to run multiple experiments to improve the chance of success. Identifying an MVP - minimal viable product - can help the company avoid over-investing in a project or getting derailed into tangential aspects that aren’t critical to the core goal. 

Piantino advocates for the value of tools and infrastructure in helping accelerate machine learning projects. The typical software engineer doesn’t spend all of his or her time writing code. A lot of time is spent doing dev ops, infrastructure setup, or repetitive project overhead. Having tools to handle some of this can accelerate progress and also decrease the friction of trying an experiment. He says, “Being able to accelerate things not only lets people do things faster but also encourages them to explore more.”

The Future of AI in Industry

One thing that’s clear is that AI and ML will take on an increasingly important role in industry. While it can be difficult for companies to manage that transition, early adopters of AI in the industry are learning techniques for project planning, managing expectations, accelerating development, and tying a vision in to the core product goals. As better tooling and infrastructure are developed, incorporating AI and ML into traditional engineering organizations will only become easier and even more impactful. 

Ready to Get Started?

Create an account in minutes or connect with our team to learn how Spell can accelerate your business.