4 ms·
"In evaluating Google’s claims, it’s valuable to keep in mind Google has a vested financial interest in convincing us that the key to effective use of deep lear
by halflings 8y ago
"In evaluating Google’s claims, it’s valuable to keep in mind Google has a vested financial interest in convincing us that the key to effective use of deep learning is more computational power, because this is an area where they clearly beat the rest of us. If true, we may all need to purchase Google products. On its own, this doesn’t mean that Google’s claims are false, but it’s good be aware of what financial motivations could underlie their statements."
Excuse the snark, but:
"In evaluating FastAI's claims, it’s valuable to keep in mind FastAI has a vested financial interest in convincing us that the key to effective use of deep learning is more machine learning experts, because ML education is their business model. If true, we may all need to purchase FastAI courses. On its own, this doesn’t mean that FastAI’s claims are false, but it’s good be aware of what financial motivations could underlie their statements."
I can think of thousands of ways that Google could increase the computational power required that would be much easier than the AutoML effort (for ex. simply recommending ridiculously deep models that take 100x to train without giving higher performance). They are putting so much effort into AutoML because it just works. A lot of the things included in later parts of this series are very useful (learning rate search, etc.) and decrease the computational power required, but most people just want to drop a dataset and pick the type of model (say, multiclass image classification) and leave the rest for machines to optimize.
- citrablue 8y agoYour FastAI criticism isn't really accurate, as FastAI is free online courseware. I suppose you could claim that jhoward makes his living from his personal brand, and FastAI helps that... but that's a bit of a stretch. It would be much more valid if they were a business, rather than an open source software and free course solution. Even as a competitor, it's probably useful criticism to get a negative perspective from a competitor.
- voidray 8y ago> most people just want to drop a dataset and pick the type of model (say, multiclass image classification) and leave the rest for machines to optimize. I think the disconnect here is that you can reuse existing architectures and get state-of-the-art performance without running something like AutoML. It's not clear that creating a bespoke architecture for your specific problem is always better, let alone always a good use of your resources.
- halflings 8y agoFrankly, I don't think this is obvious. Case in point: sequential models (granted, probably not yet included in the first batch of AutoML). There are so many ways to model the problem that tweaking each of these ways, in a way that makes sense for your dataset, takes a lot of work. I've built models that worked fairly OK, only to have a colleague build a separate model that had +10% prAUC by virtue of adding some additional mechanism (say, attention, a different RNN cell, more units, etc.). I'll also say that this is aimed towards people that are unfamiliar with ML and would have trouble finding and re-implementing the state-of-the-art in their specific field.
- tntn 8y agoWow, you sure got them good. Thanks for rooting out the financial motivations of fast.ai, a nonprofit research lab that publishes courses online for free. /s
- pwaai 8y agoSome people will be cynical for the sake of it.
- amrrs 8y agoIgnoring the fact that Auto ML can't have an impact is how Nokia and Blackberry laughed at iPhone that "huh! who'll use a full-touch phone". Data Scientists by profession are really insecure about this fact of Auto ML hence making a claim that can free up their mind. Data Scientists in many companies have become close to what SQL Job or any web script job runners used to do and then CRON jobs came into pic and that's what Auto ML will do here with a a set of sequential steps replacing mundance ML activities.
- halflings 8y agoSorry for the tone here (unusually snarky comment). Arguably, my point was that this argument, biasing your analysis by hidden motives that cannot be disproven, is invalid. But clearly, I was wrong because fast.ai's courses are indeed free so the comparison doesn't stand. I'll also add that fast.ai has great ML courses that go deep into details, and focus on practical, applicable, techniques. My comment was specifically about the argument that Google is doing this just to increase Cloud revenues.
- YeGoblynQueenne 8y ago>> They are putting so much effort into AutoML because it just works. It just works, provided your dataset can be solved well enough given AutoML's pretrained model. Which is not at all guaranteed.