3 ms·

Just a quick belief of myself (might change tomorrow): ML winter will not come until we hit the flattening of the specialized hardware s-curve. I know people be

Just a quick belief of myself (might change tomorrow): ML winter will not come until we hit the flattening of the specialized hardware s-curve. I know people believe that ML can still scale in funding (for even bigger models than GPT-3 with current hardware) but that shouldn’t be our hope. I also can not imagine that the exponential efficiency increase in architectures with optimized structure can continue for a couple more years (happy to be proven wrong here). Also the scaling in dataset sizes will ultimately halt (or might be already) for supervised learning which might give a bump for reinforcement learning where there is usually unlimited data (due to simulation)

mlthoughts2018 6y ago
Most commercial use of ML, including neural nets, is small data application-specific business logic. Think phishing / spam / fraud detection, anomaly detection, semantic search, image search, keyword labeling, classifying or segmenting customer data. These applications have well-understood business value propositions. Much, much less often the ML application focused on truly large data. I think we might see a winter in super big applications like self-driving cars or voice assistants, but ML in general is just a boring, non-controversial business tool with hundreds of valuable applications. You’ll still need statistical specialists to train and operate models and ensure systems avoid pitfalls like overfitting, poor convergence, multicollinearity, confounders, etc. So I doubt this will have much impact on ML job market. Companies that invest in ML will continue to run circles around companies that don’t. You’ll just see the unjustified over-focus on SOTA neural networks die down and become just another boring tool in the toolbox like everything else in ML.
- WanderPanda 6y ago
  I agree, or at least I hope you are right. I have some doubts about investment into ML without the hype (at least for legacy big corps) but probably their hype-driven ML efforts are/were misguided by poor incentives. But I agree again that companies that invest in ML from a grassroots/first principles way should run circles around the legacy ones. I think the „boring“ part you mention might be what I refer to as ML/AI winter. But then again do you think there is much space left in the ML toolbox apart from neural networks? I think at least for e.g. computer vision we can agree that neural nets ate all the cake, right?

mlthoughts2018 6y ago

Most commercial use of ML, including neural nets, is small data application-specific business logic. Think phishing / spam / fraud detection, anomaly detection, semantic search, image search, keyword labeling, classifying or segmenting customer data. These applications have well-understood business value propositions. Much, much less often the ML application focused on truly large data. I think we might see a winter in super big applications like self-driving cars or voice assistants, but ML in general is just a boring, non-controversial business tool with hundreds of valuable applications. You’ll still need statistical specialists to train and operate models and ensure systems avoid pitfalls like overfitting, poor convergence, multicollinearity, confounders, etc. So I doubt this will have much impact on ML job market. Companies that invest in ML will continue to run circles around companies that don’t. You’ll just see the unjustified over-focus on SOTA neural networks die down and become just another boring tool in the toolbox like everything else in ML.

WanderPanda 6y ago
I agree, or at least I hope you are right. I have some doubts about investment into ML without the hype (at least for legacy big corps) but probably their hype-driven ML efforts are/were misguided by poor incentives. But I agree again that companies that invest in ML from a grassroots/first principles way should run circles around the legacy ones. I think the „boring“ part you mention might be what I refer to as ML/AI winter. But then again do you think there is much space left in the ML toolbox apart from neural networks? I think at least for e.g. computer vision we can agree that neural nets ate all the cake, right?