I'll talk a bit on how to avoid "over-promising and under-delivering". We're a tiny, boutique, consultancy that helps organizations solve problems, mainly using

I'll talk a bit on how to avoid "over-promising and under-delivering". We're a tiny, boutique, consultancy that helps organizations solve problems, mainly using machine learning, but we tell our would be clients they don't really need machine learning when we look at the problem and we think it is not a machine learning problem. [Note: we learned the hard way not to work on projects or build anything if the people we're building for are not involved. We were burned after a project where we only talked with executives instead of the people who were going to use our product, at the executives' demands, despite our pleas. Never again]. As of today, we worked on several problems in diverse sectors and industries. Banking, telecoms, energy, transportation, communication, storage, cosmetics, recruiting, industrial supplies. The problems we worked on: customer churn reduction, next best offer (and recommendation problems), predictive maintenance, forecasting, sound event detection, etc. One of the most important thing we did in our process is to thoroughly understand the problem before we even begin "AIying". Many times, organizations come to us because they don't want to be left behind and want to "use AI". Failing to truly understand what the job to be done is, what the problem is, what the end state is, is one of the major reasons of failed projects and underwhelming results and bitterness. Second: failing to include the domain experts and people who actually will be using a data product will lead to people suffering from the "AI project" instead of enjoying its fruits. This is horrible. What we started doing right off the bat when talking with executives is stating our position clearly: who will use this? Your marketing/sales/engineers? Good, get them to the table so they could talk about their workflow and process, and what they need help with, and what they want, and what would make them happy, with their own words. We share with them horror stories when we only talked with upper management and only met the domain experts when the project was "done", developed in a vacuum, and we had to have difficult conversations with disgruntled people who now had a solution they were not involved with on their lap. Third: it's important to have a cadence and regular meetings to monitor the progress of the project (and again, with the people involved around the table). They'll tell you what matters and what does not. They'll tremendously help with the metrics or what's an acceptable outcome, because by that time, you know it's obviously not the F1 score. Sometimes, it's useful to ask negated questions and to make sure something matters. For example, we were talking with someone in the energy sector to mitigate an event which, when it happens, leads to a huge loss (in the nine figures). The person said they wanted to predict the event 48 hours in advance. Now, you can take this at face value, but you can also ask "Why 48? Why not 49?". It seems arbitrary. One way we asked the question was "At what point alerting you is useless and you can do nothing about it?". The person said "Even if you alert me 2 minutes beforehand, I can do things to mitigate the damage and I'll take it". This dramatically changes the direction of the project. The same goes when talking with people who want a 80% accuracy or something. Why 80%? Why not 82%? What went into coming up with this figure? What's the actual objective? That was part of the effort of reshaping our consulting process upstream over the years. Coming up with better questions to scope the project, to include the actual stakeholders, to better understand the problem, and to reduce the chances of projects dying or disappointing. Downstream, we built our machine learning platform [collaborative notebooks, train, track, deploy, and monitor models, etc] so it does not take us an eternity to deliver, because the longer the "time to value", the higher the likelihood there would be any value at all, and the higher the chance the project will fail. It is a learning curve and there are bumps on the road, especially in the beginning. If you only had failures, there's an "imbalance" in your data and it's hard to make a model of how to go through it, yet it is paramount to examine these failures and systematically, ruthlessly, eliminate their root causes. One project is not really enough to extract all the learnings, and it took us many years and many projects to see a bigger picture of the tooling we wanted for ourselves.