4 ms·
Deep Learning Expert Says GPT Startups May Be in for a Rude Awakening
- mountainriver 4y agoAgree with this. The current evolution of GPTs are very impressive but still not very useful. This may change in 2023 but if it doesn't these startups are likely in the same vein as web3
- tluyben2 4y agoIt already has far more utility than web3 ever had. So many people are already using this to get far more productive. Don’t know about the startups, but GPT3+ is already a success for actually productivity.
- mountainriver 4y agoeh I don't know that I would say that yet, the accuracy problem with the current generation of GPTs makes them mostly toys. I have yet to meet a developer that has continued using them, most of the ones I know, including myself have tried. That being said we aren't far off from them being the most useful things ever
- zarzavat 4y agoThis is what boiling a frog looks like. We have had year after year of astounding results. From LSTM horse and cart to Transformer combustion engines in the blink of an eye. And yet people keep claiming that there’s nothing to worry about, AGI is very far away, it’s all going to fizzle out. I don’t buy it. I challenge anyone to come up with a quantitative/objective measurement that shows progress in natural language/reasoning is decelerating. What catches people out is that progress in AI is coming from a very oblique angle. People expect human-like progression of intelligence, from baby to toddler to child to adolescent. In humans basic reasoning develops first, then over our lifetime we learn knowledge to apply our reasoning to. But progress in AI is being made from the opposite direction: start with a model that knows vastly more than any human could ever know, then progressively improve its reasoning ability. So allow me to make the opposing case. We are seeing a fundamental change in our reality. AGI is coming and it’s coming much faster than anyone expects.
- deleted 4y ago[deleted]
- dougmwne 4y agoI will echo this. I don’t know if we will ever reach AGI, but several goalposts seem to have already flown past and we are still moving at top speed. I think it will take a decade or more to integrate the AI capabilities we have today into our economy. But until the tsunami of advancements abates, it’s impossible to guess how much our world will be changed by AI. Homework already seems to be a goner.
- fidgewidge 4y agoChollet's argument isn't about AGI or research progress. It's about business and real world impact of LLMs. He outlines two cases: 1. Bull case. LLMs become a kind of general speech/language interface to data. 2. Bear case. LLMs continue to be used primarily to automate customer support agents and help marketing people write blog posts, and within a few years become seen as a blockchain-like bubble: cool tech, not much real world impact. So you're not making an opposing case, because you're responding to a strawman argument that the article isn't about.
- xpe 4y agoI'm inclined to think the following hypothesis is true: both _under_-investment and _over_-investment have negative consequences for otherwise promising technologies. To be fair, the above statement may be too generalized to be usefully descriptive. There are at least two additional dimensions that probably should be included in any descriptive theory: 1. What is the nature of the technology in question? What do we know about similar technologies in terms of their evolution? 2. Over what time frame? This needs to mesh with the "nature" question (1, above). --- Here are some example cases or scenarios: ## Unproven directions / little institutional support? How does an idea go from a spark, maybe a paper, to having some institutional support? Perhaps someone in academia can speak to this. Overinvestment, it would seem, might create too many splinter groups, leading to a dispersion of necessary talent and connections. ## Promising opportunity for further research / good institutional support Long-term, sustained investment in pure science works well as a foundation, since it allows universities and students to build knowledge over many years, sustain careers, and foster multi-generational knowledge transfer. Overinvestment seems negative to the extent funding streams come from a fixed pie. ## Mature enough for technology transfer to small companies (often funded by more risk-tolerant investors, such as Angels, sometimes VC's) Funding of consortia that bridge academic and commercial interests. Such as the famous MCC of Austin, TX. Overinvestment would tax the resources of educational institutions, which struggle with the value proposition and tuition and fees increase _much_ more rapidly than inflation over the last ~20 years. ## Proven technology, but unclear market fit / sales to early adopters? This seems to be an area where investment works well. Small companies need to customize the technology and sell it. Overinvestment might lead to too many squirrels chasing too few acorns, though. Markets tend to do well when there are enough competitors to drive prices down and innovate. They serve humanity more broadly when market failure are mitigated and minimized. ## Proven market, though perhaps still niche / unscaled / viability for the mainstream? Similar to the case immediately above. Additionally: overinvestment runs the risk of creating inflated expectations and distorting labor costs. ## Proven business / opportunity to improve efficiency of known problems (e.g. practices -- perhaps 'boring' or 'messy' to some -- that nevertheless can dramatically improve quality and/or reduce expenses A sustained ecosystem of companies that build expertise among their employees, who share best practices indirectly (i.e. adoption of source control, automated testing, continuous integration) helps drive continued improvements in an otherwise 'mature' industry. --- Even if the above questions could be made more specific and testable, it would require _significant_ structural and behavioral changes to move technology investment towards a "happy middle".
- jt2190 4y agoThe Deep Learning expert, François Chollet, is actually saying something more nuanced, e.g: > Now, seeing [SEO/marketing engagement bait] tweets is compatible with both the bull case and the bear case. If the tech is revolutionary, it will be used in this way. What's interesting to me is that ~80% of ChatGPT tweets with >2000 likes fall into this category. https://twitter.com/fchollet/status/1612152222300901378?s=46&t=4uIEfD_CbxnSLeAx8Kfk-w https://twitter.com/fchollet/status/1612152222300901378?s=46...
- amf12 4y agoI don't really agree with the comparison to Web3. LLMs have real world use cases as an interface. It might not replace most current software/systems/processes but act as an interface to these, especially voice. This alone has excellent value proposition and could improve productivity. Use cases I envision: - Customer service automation (much better than the shit we have today). - Tutoring services (won't replace tutors but as an aid). - Conversational assistant. - Marketing/SEO. - Search enhancement. - Office productivity assistance (debugging, idea generation, search, etc). All of these are use cases that can generate money, unlike Web3.