I've tried using Google's AutoML to classify medical images as part of a Kaggle competition just to see what their process is like and how well it performs. Thi

I've tried using Google's AutoML to classify medical images as part of a Kaggle competition just to see what their process is like and how well it performs. This was Q4 2018, so things might have changed slightly. In brief, my experience was quite frustrating. First, getting my dataset to the cloud required quite a bit of manual labor. Uploading my 25GB of images on my 40mbit/s wasn't really ideal so I ended up spawning an virtual machine on GCP and downloading directly from Kaggle. Unzipping the files and writing it to Google Cloud storage with some terminal commands. Furthermore, to get the label data into AutoML I had to write some generation script that generated the exact format CSV that AutoML requires - which was hidden in their documentation somewhere with no mention in the AutoML environment itself. Nothing too cumbersome but generally not a very user friendly experience, or something I wish to repeat many times if I get a new/different dataset. Ultimately, when the data with labels was in the model started training. Then I found, that I didn't really have the tools and information to assess the model performance. They did a decent job of characterizing model performance through precision recall graphs and displaying incorrect predictions but that didn't really satisfy me. I was interested in getting more details about where it was misclassifying images, specifically how classification performance was distributed across the 28 classes the model was predicting (in a multi label context). This is the point where I think the downside of working with a platform such as AutoML starts showing. I tried reaching out to someone about gaining more insight into model performance by opening a ticket, since there was no phone number. After a couple of days I finally received an email from a product representative that told me that for any assistance I should contact one of their local cloud partners. These are third party vendors that typically assist companies in deploying cloud based applications in the GCP. However, after calling two of these companies that were highly recommended by Google's vendor page I was told that they don't have any experience with AutoML and that I was on my own. The other company didn't reply at all. In my view, choosing a product such as AutoML - for a company that is serious about adopting AI to improve their business - is currently not a good path (yet). And I see this space as being wide open for competition with current solutions not cutting it from my point of view.