6 ms·

At what point does the current AI hype machine crash? It feels like these startups all sell solutions to the top 20 or so companies, competing for a very limit

At what point does the current AI hype machine crash? It feels like these startups all sell solutions to the top 20 or so companies, competing for a very limited market. It's table stakes for everybody else, an "all-or-nothing" achievement for AI. By contrast, even a small amount of connectivity improved things dramatically for huge swaths of industry. That's how the internet started small and grew up. AI seems to rely on being everything or being nothing. There's no small victories.

SavantIdiot 5y ago
I can't speak to cloud deep learning, but edge NN's have two practical applications: anomaly detection and object defect detection. (Keyword spotting is completely stalled. I've been watching Arm-based companies work on keyword spotting for years and they are still 80% accurate in a real setting, which is like talking to an old person with hearing issues.) Anomaly detection (vibration analysis, audio anomalies) are working fairly well. Same goes for assembly lines where defective parts are booted off, and NN's don't improve that much over traditional methods. The hype machine is all the big sexy stuff. GPT, Protein folding and Alpha Go are truly amazing. But I haven't seen a big step-function in deployable NNets (github-ready) in years. Which given the number of people researching this I expected a linear progression. What have we seen since UNet, MobileNet, SSD, Yolo, BERT, ResNet150, Inception ... ? That's worrisome, but it also could be the pendulum swing while people figure out exactly why these NN's work the way they do. I think we need to wait another 20 years, which means lots of hype money will dry up, but I don't think it will vanish. The efficiency of neural nets as a general function approximation might not be super high accuracy, but it is amazing to see such a simple construct perform with such speed.
- bingohbangoh 5y ago
  > edge NN's have two practical applications: anomaly detection and object defect detection Do you have any links to companies working on this? I also haven't found any real innovations since adversarial neural networks in ~2016, which roughly fits in with your timeline. Not sure what happened to Geoff Hinton's Capsule networks, that looked like it was going to be a big thing at the time. This is admittedly a very hot take though.
  - SavantIdiot 5y ago
    At Embedded World 2020, every embedded semi and FPGA company had some kind of NNet running. One that was really impressive was Xilinx. They had multiple demos set up at Embedded World in 2020 showing NNet's for defects, person detection, and audio vibrational analysis with autoencoders. As did several other FPGA companies. I googled some links but they don't look like what I saw. So I'd recommend googling Xilinx if you are interested in industrial applications.
    - bingohbangoh 5y ago
      many thanks!
belval 5y ago
It won't because actual DL products solve problems that are not sexy but still works objectively better than traditional ML for most tasks. The best example of this is OCR/Document Intelligence. The old ML approaches of the 1990-2000 are really no match for what we have now. It's gotten to a point where extracting handwritten text is done reliably for very cheap. Same goes for tables and key-values extractions which require a higher level of understanding that simply wasn't available before. So no, it won't crash and you'll keep seeing these pieces every month that "AI can't do X" until we actually achieve X and then the goalpost is moved.
- tablespoon 5y ago
  > It's gotten to a point where extracting handwritten text is done reliably for very cheap. Is that freeform handwritten documents, or handwritten text filled into boxes on a form?
  - belval 5y ago
    Even free form handwritten text recognition is pretty good at this point and that's true for most languages. You can try with Google Lens if you have an Android phone.
- bingohbangoh 5y ago
  > ... "AI can't do X" until we actually achieve X and then the goalpost is moved. This gets said often but I don't think anybody whose credible in the field actually makes statements like this. Take Chess for instance. In the IBM "Big Blue" documentary from ~2004, they quote journalists as saying "AI can't play Chess as well as humans but if it could, then AI would be solved." Why did the techniques from Big Blue not seem to go anywhere? I know for a fact that scientists _were not saying this_. The Lighthill Debates on AI specifically talk about why playing games well doesn't really prove anything. [0] I do agree that OCR has improved greatly by AI but this feels very niche. Somebody mentioned defect anomaly detection in another comment and I was not aware of this. All useful for sure. Still, this doesn't amount to anywhere near the hype that was announced earlier last decade. Moreover, the economics in AI are mostly awful despite everybody's seemingly best efforts. [1] Even if its useful in some vague sense, it's not necessarily economically useful. Amazon has ~10,000 people working on Alexa. [2] Have they turned a profit on these endeavours? I understand they can absorb the costs but its not clear to me how the economics will work out here. ML models haven't even been useful in places where statistical methods have reined supreme such as Renaissance Technologies and other hedge funds. No large companies are using neural networks in a significant capacity to my knowledge. Another big tell for me is the lack of any consumer products in the space. Where did they go? why are they missing? This is what I mean by "everybody is competing for the top 20 or so customers." This is compounded by the unstructured nature of most data. Most databases are still terrible, especially at the few institutions large enough to have it and large enough for it to make a difference in their business. There should be more focus into this problem if anything. A well tuned and structured database will be many times more useful than a fancy model that needs constant retraining. But I guess its not as cool so nobody cares. [0]: https://www.youtube.com/watch?v=03p2CADwGF8 https://www.youtube.com/watch?v=03p2CADwGF8 -- highly recommended, with many of the arguments still resonating today. [1]: https://a16z.com/2020/02/16/the-new-business-of-ai-and-how-its-different-from-traditional-software/ https://a16z.com/2020/02/16/the-new-business-of-ai-and-how-i... [2]: https://qr.ae/pGJUKk https://qr.ae/pGJUKk -- couldn't find a better source offhand.
michannne 5y ago
I have largely been critical of machine learning for some time, I don't think it will crash though. I feel many people have been misled on what deep learning and machine learning actually do and therefore become confused on its limitations, but once you understand it as a general statistics-fitting paradigm, it is easy to ask yourself - can you solve this problem by fitting a complex pig pen around a dataset? Or, another way, is your solution based around classification in some way? If the answer is no, then modern AI techniques won't be able to get you an answer. As crude as it may seem, rules engines and long sheets of If this Then that heuristics tend to solve more complex systems that can't be statistically fit, but I feel machine learning is slowly approaching a point where it can deduce those If Then statements. The main problem is that as humans we cannot comprehend the wealth of connections that might lead to a certain heuristic, so it is very difficult to tell a modern AI that it is wrong in a more abstract or pattern-based sense, and these approaches tend to collapse back into statistical data fitting. I can easily communicate with you if your understanding is wrong, but it is hard to do that with an AI, mostly we can either fix the data, or adjust the training parameters and wait. That is why it seems to rely on all-or-nothing, as there isn't much control on the iterative learning process on a network apart from startup options and testing after the fact. To me, the real breakthrough will come by creating models of abstract thought. Some models exist as heuristic systems, which require an exhaustive set of rules to model various aspects of thinking, and some are probabilistic in nature and can be taught to a machine learning application, but neither of these model everything, nor do they do all of their topics equally well. Whichever paradigm can come up with a robust model first will likely come up top in the AI paradigm wars, but it could be tomorrow, could be next century.

SavantIdiot 5y ago

I can't speak to cloud deep learning, but edge NN's have two practical applications: anomaly detection and object defect detection. (Keyword spotting is completely stalled. I've been watching Arm-based companies work on keyword spotting for years and they are still 80% accurate in a real setting, which is like talking to an old person with hearing issues.) Anomaly detection (vibration analysis, audio anomalies) are working fairly well. Same goes for assembly lines where defective parts are booted off, and NN's don't improve that much over traditional methods. The hype machine is all the big sexy stuff. GPT, Protein folding and Alpha Go are truly amazing. But I haven't seen a big step-function in deployable NNets (github-ready) in years. Which given the number of people researching this I expected a linear progression. What have we seen since UNet, MobileNet, SSD, Yolo, BERT, ResNet150, Inception ... ? That's worrisome, but it also could be the pendulum swing while people figure out exactly why these NN's work the way they do. I think we need to wait another 20 years, which means lots of hype money will dry up, but I don't think it will vanish. The efficiency of neural nets as a general function approximation might not be super high accuracy, but it is amazing to see such a simple construct perform with such speed.

bingohbangoh 5y ago
> edge NN's have two practical applications: anomaly detection and object defect detection Do you have any links to companies working on this? I also haven't found any real innovations since adversarial neural networks in ~2016, which roughly fits in with your timeline. Not sure what happened to Geoff Hinton's Capsule networks, that looked like it was going to be a big thing at the time. This is admittedly a very hot take though.
- SavantIdiot 5y ago
  At Embedded World 2020, every embedded semi and FPGA company had some kind of NNet running. One that was really impressive was Xilinx. They had multiple demos set up at Embedded World in 2020 showing NNet's for defects, person detection, and audio vibrational analysis with autoencoders. As did several other FPGA companies. I googled some links but they don't look like what I saw. So I'd recommend googling Xilinx if you are interested in industrial applications.
  - bingohbangoh 5y ago
    many thanks!

belval 5y ago

It won't because actual DL products solve problems that are not sexy but still works objectively better than traditional ML for most tasks. The best example of this is OCR/Document Intelligence. The old ML approaches of the 1990-2000 are really no match for what we have now. It's gotten to a point where extracting handwritten text is done reliably for very cheap. Same goes for tables and key-values extractions which require a higher level of understanding that simply wasn't available before. So no, it won't crash and you'll keep seeing these pieces every month that "AI can't do X" until we actually achieve X and then the goalpost is moved.

tablespoon 5y ago
> It's gotten to a point where extracting handwritten text is done reliably for very cheap. Is that freeform handwritten documents, or handwritten text filled into boxes on a form?
- belval 5y ago
  Even free form handwritten text recognition is pretty good at this point and that's true for most languages. You can try with Google Lens if you have an Android phone.
bingohbangoh 5y ago
> ... "AI can't do X" until we actually achieve X and then the goalpost is moved. This gets said often but I don't think anybody whose credible in the field actually makes statements like this. Take Chess for instance. In the IBM "Big Blue" documentary from ~2004, they quote journalists as saying "AI can't play Chess as well as humans but if it could, then AI would be solved." Why did the techniques from Big Blue not seem to go anywhere? I know for a fact that scientists _were not saying this_. The Lighthill Debates on AI specifically talk about why playing games well doesn't really prove anything. [0] I do agree that OCR has improved greatly by AI but this feels very niche. Somebody mentioned defect anomaly detection in another comment and I was not aware of this. All useful for sure. Still, this doesn't amount to anywhere near the hype that was announced earlier last decade. Moreover, the economics in AI are mostly awful despite everybody's seemingly best efforts. [1] Even if its useful in some vague sense, it's not necessarily economically useful. Amazon has ~10,000 people working on Alexa. [2] Have they turned a profit on these endeavours? I understand they can absorb the costs but its not clear to me how the economics will work out here. ML models haven't even been useful in places where statistical methods have reined supreme such as Renaissance Technologies and other hedge funds. No large companies are using neural networks in a significant capacity to my knowledge. Another big tell for me is the lack of any consumer products in the space. Where did they go? why are they missing? This is what I mean by "everybody is competing for the top 20 or so customers." This is compounded by the unstructured nature of most data. Most databases are still terrible, especially at the few institutions large enough to have it and large enough for it to make a difference in their business. There should be more focus into this problem if anything. A well tuned and structured database will be many times more useful than a fancy model that needs constant retraining. But I guess its not as cool so nobody cares. [0]: https://www.youtube.com/watch?v=03p2CADwGF8 https://www.youtube.com/watch?v=03p2CADwGF8 -- highly recommended, with many of the arguments still resonating today. [1]: https://a16z.com/2020/02/16/the-new-business-of-ai-and-how-its-different-from-traditional-software/ https://a16z.com/2020/02/16/the-new-business-of-ai-and-how-i... [2]: https://qr.ae/pGJUKk https://qr.ae/pGJUKk -- couldn't find a better source offhand.

michannne 5y ago

I have largely been critical of machine learning for some time, I don't think it will crash though. I feel many people have been misled on what deep learning and machine learning actually do and therefore become confused on its limitations, but once you understand it as a general statistics-fitting paradigm, it is easy to ask yourself - can you solve this problem by fitting a complex pig pen around a dataset? Or, another way, is your solution based around classification in some way? If the answer is no, then modern AI techniques won't be able to get you an answer. As crude as it may seem, rules engines and long sheets of If this Then that heuristics tend to solve more complex systems that can't be statistically fit, but I feel machine learning is slowly approaching a point where it can deduce those If Then statements. The main problem is that as humans we cannot comprehend the wealth of connections that might lead to a certain heuristic, so it is very difficult to tell a modern AI that it is wrong in a more abstract or pattern-based sense, and these approaches tend to collapse back into statistical data fitting. I can easily communicate with you if your understanding is wrong, but it is hard to do that with an AI, mostly we can either fix the data, or adjust the training parameters and wait. That is why it seems to rely on all-or-nothing, as there isn't much control on the iterative learning process on a network apart from startup options and testing after the fact. To me, the real breakthrough will come by creating models of abstract thought. Some models exist as heuristic systems, which require an exhaustive set of rules to model various aspects of thinking, and some are probabilistic in nature and can be taught to a machine learning application, but neither of these model everything, nor do they do all of their topics equally well. Whichever paradigm can come up with a robust model first will likely come up top in the AI paradigm wars, but it could be tomorrow, could be next century.