8 ms·

Two years later, deep learning is still faced with the same basic challenges

61 points by spenvo 3y ago

christenchrist 3y ago
[dead]
sdenton4 3y ago
It's worth it to ask, when reading Gary Marcus pat himself on the back, how many of his predictions were actually falsifiable.... Few-to-none. Furthermore, there has been insane progress over the last few years, which presumably would not have happened if everyone had despaired on reading Marcus' 'hitting a wall' article and dropped everything to work on GOFAI. But let's widen the lens a little bit: `In 2015, shortly after Hinton joined Google, The Guardian reported that the company was on the verge of “developing algorithms with the capacity for logic, natural conversation and even flirtation.” In November 2020, Hinton told MIT Technology Review that “deep learning is going to be able to do everything.” I seriously doubt it.` Well, we've certainly got natural conversation and flirtation, along with some kind of logic - especially if you're OK with accepting python programs as a form of response. This is well beyond what was considered possible at the time. We're certainly not at 'everything' but *shrug*. Maybe in ten years. `nowhere near the ordinary day-to-day intelligence of Rosey the Robot, a science-fiction housekeeper that could not only interpret a wide variety of human requests but safely act on them in real time.` It exists, it just isn't anywhere near economical for household tasks... https://www.wired.com/story/google-robot-learned-to-take-orders-by-scraping-the-web/ https://www.wired.com/story/google-robot-learned-to-take-ord... LLM's look poised to upend robotics. Marcus then goes extensively into why everyone should be looking at symbol-manipulation systems. And that's fine... But would be much more convincing if he could actually produce something that works. At some point, one asks, if your idea is so good, why haven't you actually solved any problems?
- martindbp 3y ago
  Like Yann LeCun said, Gary Marcus has contributed exactly nothing to the field, he's an influencer that claims to be an expert. Just ignore him.
  - nh23423fefe 3y ago
    to wit: his appearance on Ezra Klein was pretty terrible >It’s a kind of glorified cut and paste. Pastiche is putting together things kind of imitating a style. And in some sense, that’s what it’s doing. It’s imitating particular styles, and it’s cutting and pasting a lot of stuff. It’s a little bit more complicated than that. But to a first approximation, that’s what it’s doing is cutting and pasting things. This argument is terrible because it proves too much. Either AIs are automata with no connection to the world, or if they are connected to the world its just copy-paste. And somehow infants aren't just copy paste because uuuuh they have "true understanding" whatever that is. [1] https://www.nytimes.com/2023/01/06/podcasts/transcript-ezra-klein-interviews-gary-marcus.html https://www.nytimes.com/2023/01/06/podcasts/transcript-ezra-...
- trimethylpurine 3y ago
  I can afford one. But if it tells me lies and picks up the wrong remote I want a refund, as would most customers spending more than $1000 on pretty much anything. And that's the real issue, I think. The marketing has far surpassed the reality of the actual product. People buy helicopters, and jets, and houses larger than an office building. It's not the cost that's at issue. There's room for bullshit in a chat bot. But there's absolutely zero tolerance for bullshit in a physical device roaming around children. I think the point of the article is that no one has even presented a workable theory for real AI with real comprehension. LLM isn't AI, regardless of the marketing. Until that theory exists, and is implemented, ideas about what AI will someday grow into are ignoring comprehension. That's a required component if you are using the word "intelligence" outside of fiction.
  - wruza 3y ago
    there's absolutely zero tolerance for bullshit in a physical device roaming around children Why not just put a muzzle on it? LLM isn't AI, regardless of the marketing That’s intelism. Different models have different capabilities, you can’t decide which is good enough for you to treat as intelligent. You use terms like intelligence and comprehension, which don’t even have a definition better than “I know it when I see it”. Not even going to discuss that you want to buy an actually intelligent being for picking up remotes.
    - trimethylpurine 3y ago
      Whoops. You've contradicted yourself. I didn't advertise a robot picking up a remote for sale as intelligent. I'm advocating that we stop calling it intelligent when it isn't. You're saying it's intelligent. So either you're ready to offend the previous commenter for their evil advocacy of selling an intelligent robot made to pick up remotes for an advertisement, or you're at odds with your point because you know damn well it's not intelligent. My guess is the latter. From the dictionary: the ability to learn or understand or to deal with new or trying situations : REASON also : the skilled use of reason (2) : the ability to apply knowledge to manipulate one's environment or to think abstractly as measured by objective criteria (such as tests) This robot or ChatGPT, or any LLM theorized to my knowledge does not meet that definition due to the underlying technology not approaching "abstract" or "skilled use of reason." It's pretty simply not addressing those items. There's no debate. It's a completely different approach to something that has nothing to do with intelligence. And marketing it as such is frankly a legal issue, in my opinion. I fully expect the lawsuits to continue until marketers are forced to stop using that term entirely. As an additional note, you've muddied up a lot of concepts here. I'm not sure if you're maybe not a native speaker of English, as that might explain a lot, or if you really accidentally made opposing points sequentially. The latter is a bit funny in a discussion about intelligence. "Intelligent robot" wouldn't refer to one that feels and needs civil rights. The definition is above for your review. It's very much clear, despite your protest, what intelligence means. It is, by definition, not subjective, but objective. And there are absolutely legal definitions with very real legal ramifications. Also, you should read back through your own comments. You're not a very likable person, and I'm telling you that as a personal favor. You should get some help for this. It's critical for your personal relationships, and it's not very expensive.
- dataflow 3y ago
  I have no idea whom this guy is, and have no thoughts here on his opinions or their falsifiability (happy to agree with you on that aspect if it matters), but just to reply to this particular point here: > This is well beyond what was considered possible at the time. We're certainly not at 'everything' but shrug. Maybe in ten years. This feels a lot like what people were saying about self-driving cars being imminent circa... 2015 or so? [1] The skeptical folks rolled their eyes at suggestions that we'd have self driving cars everywhere in a few years, but lo and behold, they were right. Just because we had a massive amount of progress in the years before, that doesn't mean we were on the cusp of achieving the goal. Turns out going from 99.9% accurate to 99.99% accurate (or whatever the numbers were) is harder than all the believers wanted to admit. This feels like the same thing all over again. Yes, there's been a lot of progress in LLMs. No, that doesn't mean we're anywhere close to deep learning being able to do 'everything' in 10 years, whatever that means. [1] https://www.theverge.com/24065447/self-driving-car-autonomous-tesla-gm-baidu https://www.theverge.com/24065447/self-driving-car-autonomou...
  - Animats 3y ago
    "Turns out going from 99.9% accurate to 99.99% accurate (or whatever the numbers were) is harder than all the believers wanted to admit." True. But Calfornia's DMV makes Waymo publish their stats, and they get maybe 50% better each year. That's a good rate of gain.
    - dataflow 3y ago
      Of course they've been making good gains. We may very well get there someday. It's just nowhere near the timeline many experts were so confident in.
    - JohnKemeny 3y ago
      Citation needed
      - Animats 3y ago
        https://www.dmv.ca.gov/portal/vehicle-industry-services/autonomous-vehicles/ https://www.dmv.ca.gov/portal/vehicle-industry-services/auto...
- aredox 3y ago
  What do mean, "not falsifiable"? Do we have self-driving cars? LLM attorneys? LLM radiologists? No, no and no. And the continuous failure of self-driving cars should worry you.
  - janalsncm 3y ago
    The one that jumped out to me was his claiming that computers don’t truly understand language. At this point I don’t know what that means. I’ll be the first to admit the limitations of statistical language models, but for people like Marcus who have made a name for themselves as skeptics, it’s not in their interests to have falsifiable criticisms.
  - kenjackson 3y ago
    We do have self driving cars.
    - aredox 3y ago
      Tesla says: >The currently enabled Autopilot, Enhanced Autopilot and Full Self-Driving features require active driver supervision and do not make the vehicle autonomous. Also, how long will you have self-driving? It is notorious for having dangerous regressions after an update. Those AI systems are black boxes with stochastic behavior.
      - sdenton4 3y ago
        And humans in cars are meat-boxes of stochastic behavior, prone to all kinds of dangerous regressions. I feel safer, as a cyclist, around Waymos than I do around many human-driven cars. That's even with the cyclist crash last week: it was one of the most common kinds of bicycle-car collisions (bike moving through an intersection hidden behind a truck, hit by a car turning left), and I feel confident that the vehicle responded faster to the situation than a human would have.
  - becquerel 3y ago
    My partner rides self-driving cars frequently, over a dozen times this year. The technology clearly and manifestly exists.
- Animats 3y ago
  "LLM's look poised to upend robotics." Maybe. Unstructured manipulation is getting better very slowly. But videos such as the Google one and this one [1] are mostly about making the system take instructions in natural language, not doing the actual picking. Bin-picking of uniform items is a solved problem, and was first working in the 1980s. Bin-picking of random items from a cluttered bin is still a struggle. Amazon has put a fair amount of effort into robotic bin-picking, but it's not yet good enough to deploy. There's been some good progress on bin-picking recently. Here's Brightpick's system.[2] It has a good vision system which involves LIDAR, and some nice engineering. It's not LLM-based, and it won't do everything, but it can handle most objects that can be picked up by a vacuum picker. [1] https://www.youtube.com/watch?v=31Xqw_59XZ8 https://www.youtube.com/watch?v=31Xqw_59XZ8 [2] https://www.youtube.com/watch?v=XWlQWgRfh6c https://www.youtube.com/watch?v=XWlQWgRfh6c
- garspin 3y ago
  The point Marcus is making is that LLMs are solely next word predictors - intelligence requires far more than that. LLMs will always have the same basic challenges - their 'kinda copy what a human has written' algorithm has hit its limit. He is an advocate of symbolic learning - getting that to work is a lot harder, however it's far more likely to result in genuine intelligence.
YetAnotherNick 3y ago
At this point either he is just trolling or writing it to get views from people who already believes this and looking for someone to say it with authority.
- deleted 3y ago
  [deleted]
supriyo-biswas 3y ago
Can we have a rate-limiter for submissions from the same domain, proportional to the log(TRANCO rank) of the domain? One or two posts are fine, but I’m a little tired of these hot takes that do little to acknowledge the advances the AI/ML field has made in recent years while berating it for the smallest deficiencies.
yinser 3y ago
You can’t convince me he isn’t responding to the meme of celebrating “hitting the wall day”. I can’t comment on every point he brings up here but I work for a company doing very well reading legal documents. Myth busted.
atleastoptimal 3y ago
> “We are still a long way from machines that can genuinely understand human language”. Still true It's crazy how you can just say things. If by his standards he himself can genuinely understand human language, even given how much of what he says is just smart-sounding hallucinations, then surely Claude 3 well surpasses that threshold. The secret to grifting is to only highlight your vacuous, unfalsifiable claims. He, like many AI skeptics, is a full-time goalpost mover.
- uh_uh 3y ago
  I only tried GPT4 but it has a way better command of language than many people I know and I include myself in that. If GPT4 doesn't understand language, then I don't either.
  - aredox 3y ago
    Does MS Word "understand language" because of autocorrect?
    - ALittleLight 3y ago
      Can autocorrect hold a conversation, write code or poetry, answer or pose riddles, summarize text, write novel stories, diagnose medical issues, comprehend diagrams, etc?
      - DrFalkyn 3y ago
        ELIZA could do that 50 years ago. I haven't been really impressed with ChatGPT. It basically seems like a more useful search engine.
        atleastoptimal 3y ago
        Are you using GPT 3.5 or GPT-4? And how are you using it? What queries are you giving it?
tdudhhu 3y ago
Deep learning is still not production ready. Maybe 90% of the results are great, but the other 10% are not only 'not great' but plain wrong and sometimes even dangerous. And I think gp posts is also showing that the last bump to make it production ready is still not taken and might even be impossible to take.
- jeroenvlek 3y ago
  Last time I checked ChatGPT is very much in production.
  - tdudhhu 3y ago
    I mean: ready to be used in production.
    - roenxi 3y ago
      Humans aren't ready to be used in production. That hasn't stopped us so far. LLMs can't be trusted to stick to the truth but they are very much ready for production use.
Animats 3y ago
From the article: “Deep learning, which is fundamentally a technique for recognizing patterns, is at its best when all we need are rough-ready results, where stakes are low and perfect results optional“ It is somewhat embarrassing how many tasks, from customer service to sportswriting to language translation to article summarizing to concept art for films, can be handled by deep learning at its current level of competence. That says more about the problem spaces and human intelligence that it does about deep learning.
- aredox 3y ago
  You mean, - customer service: chatbot invents refund policy that costs the company money and reputation https://www.wired.com/story/air-canada-chatbot-refund-policy/ https://www.wired.com/story/air-canada-chatbot-refund-policy... - sports writing: a private equity group is running Sports Illustrated - which was already in decline - to the ground - language translation: not much progress, and more risks of hallucinations - concept art for films: AI is basically doing plagiarism. And so much AI art is flooding the internet that the AI are basically poisoning themselves in the future. And it still can't get the right number of fingers on hands.
- becquerel 3y ago
  I wouldn't really say it's embarassing. If most human tasks required perfection (or at least high levels of correctness at all times), society would probably not have gotten to where we are. Imagine a world where every job was as hard as open-heart surgery! In a sense, it's an achievement of humanity that we're able to recognise when 'good enough' will let us achieve the things we want.
- tivert 3y ago
  >> From the article: “Deep learning, which is fundamentally a technique for recognizing patterns, is at its best when all we need are rough-ready results, where stakes are low and perfect results optional“ > It is somewhat embarrassing how many tasks, from customer service to sportswriting to language translation to article summarizing to concept art for films, can be handled by deep learning at its current level of competence. No, it's embarrassing how eagerly leaders will compromise quality to chase automation cost savings. The things you list, especially customer service and sports-writing, cannot "be handled by deep learning at its current level of competence." That doesn't stop people from forcing the results down our throats, though.
Nevermark 3y ago
No personal criticism of this person, but: "The glass isn't full yet. Sure its filling, but its filling and its not full yet, so obviously no matter how much you fill it, we know that doesn't actually fill it." ..is my breakdown on these non-expert non-specific (mathematically, algorithmically, information theoretically, any relevant expertise at all, ...) cynics. If he had predicted the progress that was made just in the last two years, it would give some crediblity to his opinions about what wasn't going to improve. But he didn't have that insight. He has no idea what's already cooking in the research kitchen. But if am going to critque predictions, shouldn't I put myself on the line? So here is my prediction: when general AI decisively advances past us, a lot of people will feel a strong need to write and read articles about how AI doesn't have a soul. And their creations have no heart. And no progress will ever be able to fix this. Despite a lack of concrete or coherent definitions, this will matter a great deal to them.
- aredox 3y ago
  "I haven't run into the wall yet. Sure the wall is there, but I am running full speed and I haven't it it yet"
caesil 3y ago
>“Few fields have been more filled with hype and bravado than artificial intelligence. It has flitted from fad to fad decade by decade, always promising the moon, and only occasionally delivering” Still true. >“We are still a long way from machines that can genuinely understand human language”. Still true, though some have argued there is some superficial understanding. >“Elon Musk recently said that the new humanoid robot he was hoping to build, Optimus, would someday be bigger than the vehicle industry”. I expressed skepticism. Still early days, but certainly domestic humanoid robots are not in the near term expected to be a big business for anyone. >The company’s charismatic CEO Sam Altman wrote a triumphant blog posttrumpeting “Moore’s Law for Everything,” claiming that we were just a few years away from “computers that can think,” “read legal documents,” and (echoing IBM Watson) “give medical advice.”Maybe, but maybe not.” Pending/still true. Two years later we don’t have reliable versions of any of that. Why does anyone listen to a thing this obvious fraud says?
guender 3y ago
[dead]
clauderoux 3y ago
Gary Marcus tries to exit his personal hole of irrelevance. I have been working in the fielf of computational linguistics for 30 years. Back in 2000, I worked with a team of linguists to implement a pretty refined syntactic parser (XIP) based on Shallow Parsing, a symbolic approach. We won with this system a SemEval competition as late as 2016 on Sentiment Analysis. But I never expected LLM to reach this level of competency in my lifetime. Critics are prone to describe the errors these models make, but they seem to forget that 1. It is not a search engine and 2. it is pretty knowledgeable in a huge range of domains, which no humans can equal. I use these models every day to create code or to explain concepts to me. It never ceases to amaze me.
janalsncm 3y ago
He wrote that deep learning is “hitting a wall” 6 months before ChatGPT was released. Definitely needs to save face somehow.

61 points by spenvo 3y ago

[dead]

It's worth it to ask, when reading Gary Marcus pat himself on the back, how many of his predictions were actually falsifiable.... Few-to-none. Furthermore, there has been insane progress over the last few years, which presumably would not have happened if everyone had despaired on reading Marcus' 'hitting a wall' article and dropped everything to work on GOFAI. But let's widen the lens a little bit: `In 2015, shortly after Hinton joined Google, The Guardian reported that the company was on the verge of “developing algorithms with the capacity for logic, natural conversation and even flirtation.” In November 2020, Hinton told MIT Technology Review that “deep learning is going to be able to do everything.” I seriously doubt it.` Well, we've certainly got natural conversation and flirtation, along with some kind of logic - especially if you're OK with accepting python programs as a form of response. This is well beyond what was considered possible at the time. We're certainly not at 'everything' but *shrug*. Maybe in ten years. `nowhere near the ordinary day-to-day intelligence of Rosey the Robot, a science-fiction housekeeper that could not only interpret a wide variety of human requests but safely act on them in real time.` It exists, it just isn't anywhere near economical for household tasks... https://www.wired.com/story/google-robot-learned-to-take-orders-by-scraping-the-web/ https://www.wired.com/story/google-robot-learned-to-take-ord... LLM's look poised to upend robotics. Marcus then goes extensively into why everyone should be looking at symbol-manipulation systems. And that's fine... But would be much more convincing if he could actually produce something that works. At some point, one asks, if your idea is so good, why haven't you actually solved any problems?

martindbp 3y ago
Like Yann LeCun said, Gary Marcus has contributed exactly nothing to the field, he's an influencer that claims to be an expert. Just ignore him.
- nh23423fefe 3y ago
  to wit: his appearance on Ezra Klein was pretty terrible >It’s a kind of glorified cut and paste. Pastiche is putting together things kind of imitating a style. And in some sense, that’s what it’s doing. It’s imitating particular styles, and it’s cutting and pasting a lot of stuff. It’s a little bit more complicated than that. But to a first approximation, that’s what it’s doing is cutting and pasting things. This argument is terrible because it proves too much. Either AIs are automata with no connection to the world, or if they are connected to the world its just copy-paste. And somehow infants aren't just copy paste because uuuuh they have "true understanding" whatever that is. [1] https://www.nytimes.com/2023/01/06/podcasts/transcript-ezra-klein-interviews-gary-marcus.html https://www.nytimes.com/2023/01/06/podcasts/transcript-ezra-...
trimethylpurine 3y ago
I can afford one. But if it tells me lies and picks up the wrong remote I want a refund, as would most customers spending more than $1000 on pretty much anything. And that's the real issue, I think. The marketing has far surpassed the reality of the actual product. People buy helicopters, and jets, and houses larger than an office building. It's not the cost that's at issue. There's room for bullshit in a chat bot. But there's absolutely zero tolerance for bullshit in a physical device roaming around children. I think the point of the article is that no one has even presented a workable theory for real AI with real comprehension. LLM isn't AI, regardless of the marketing. Until that theory exists, and is implemented, ideas about what AI will someday grow into are ignoring comprehension. That's a required component if you are using the word "intelligence" outside of fiction.
- wruza 3y ago
  there's absolutely zero tolerance for bullshit in a physical device roaming around children Why not just put a muzzle on it? LLM isn't AI, regardless of the marketing That’s intelism. Different models have different capabilities, you can’t decide which is good enough for you to treat as intelligent. You use terms like intelligence and comprehension, which don’t even have a definition better than “I know it when I see it”. Not even going to discuss that you want to buy an actually intelligent being for picking up remotes.
  - trimethylpurine 3y ago
    Whoops. You've contradicted yourself. I didn't advertise a robot picking up a remote for sale as intelligent. I'm advocating that we stop calling it intelligent when it isn't. You're saying it's intelligent. So either you're ready to offend the previous commenter for their evil advocacy of selling an intelligent robot made to pick up remotes for an advertisement, or you're at odds with your point because you know damn well it's not intelligent. My guess is the latter. From the dictionary: the ability to learn or understand or to deal with new or trying situations : REASON also : the skilled use of reason (2) : the ability to apply knowledge to manipulate one's environment or to think abstractly as measured by objective criteria (such as tests) This robot or ChatGPT, or any LLM theorized to my knowledge does not meet that definition due to the underlying technology not approaching "abstract" or "skilled use of reason." It's pretty simply not addressing those items. There's no debate. It's a completely different approach to something that has nothing to do with intelligence. And marketing it as such is frankly a legal issue, in my opinion. I fully expect the lawsuits to continue until marketers are forced to stop using that term entirely. As an additional note, you've muddied up a lot of concepts here. I'm not sure if you're maybe not a native speaker of English, as that might explain a lot, or if you really accidentally made opposing points sequentially. The latter is a bit funny in a discussion about intelligence. "Intelligent robot" wouldn't refer to one that feels and needs civil rights. The definition is above for your review. It's very much clear, despite your protest, what intelligence means. It is, by definition, not subjective, but objective. And there are absolutely legal definitions with very real legal ramifications. Also, you should read back through your own comments. You're not a very likable person, and I'm telling you that as a personal favor. You should get some help for this. It's critical for your personal relationships, and it's not very expensive.
dataflow 3y ago
I have no idea whom this guy is, and have no thoughts here on his opinions or their falsifiability (happy to agree with you on that aspect if it matters), but just to reply to this particular point here: > This is well beyond what was considered possible at the time. We're certainly not at 'everything' but shrug. Maybe in ten years. This feels a lot like what people were saying about self-driving cars being imminent circa... 2015 or so? [1] The skeptical folks rolled their eyes at suggestions that we'd have self driving cars everywhere in a few years, but lo and behold, they were right. Just because we had a massive amount of progress in the years before, that doesn't mean we were on the cusp of achieving the goal. Turns out going from 99.9% accurate to 99.99% accurate (or whatever the numbers were) is harder than all the believers wanted to admit. This feels like the same thing all over again. Yes, there's been a lot of progress in LLMs. No, that doesn't mean we're anywhere close to deep learning being able to do 'everything' in 10 years, whatever that means. [1] https://www.theverge.com/24065447/self-driving-car-autonomous-tesla-gm-baidu https://www.theverge.com/24065447/self-driving-car-autonomou...
- Animats 3y ago
  "Turns out going from 99.9% accurate to 99.99% accurate (or whatever the numbers were) is harder than all the believers wanted to admit." True. But Calfornia's DMV makes Waymo publish their stats, and they get maybe 50% better each year. That's a good rate of gain.
  - dataflow 3y ago
    Of course they've been making good gains. We may very well get there someday. It's just nowhere near the timeline many experts were so confident in.
  - JohnKemeny 3y ago
    Citation needed
    - Animats 3y ago
      https://www.dmv.ca.gov/portal/vehicle-industry-services/autonomous-vehicles/ https://www.dmv.ca.gov/portal/vehicle-industry-services/auto...
aredox 3y ago
What do mean, "not falsifiable"? Do we have self-driving cars? LLM attorneys? LLM radiologists? No, no and no. And the continuous failure of self-driving cars should worry you.
- janalsncm 3y ago
  The one that jumped out to me was his claiming that computers don’t truly understand language. At this point I don’t know what that means. I’ll be the first to admit the limitations of statistical language models, but for people like Marcus who have made a name for themselves as skeptics, it’s not in their interests to have falsifiable criticisms.
- kenjackson 3y ago
  We do have self driving cars.
  - aredox 3y ago
    Tesla says: >The currently enabled Autopilot, Enhanced Autopilot and Full Self-Driving features require active driver supervision and do not make the vehicle autonomous. Also, how long will you have self-driving? It is notorious for having dangerous regressions after an update. Those AI systems are black boxes with stochastic behavior.
    - sdenton4 3y ago
      And humans in cars are meat-boxes of stochastic behavior, prone to all kinds of dangerous regressions. I feel safer, as a cyclist, around Waymos than I do around many human-driven cars. That's even with the cyclist crash last week: it was one of the most common kinds of bicycle-car collisions (bike moving through an intersection hidden behind a truck, hit by a car turning left), and I feel confident that the vehicle responded faster to the situation than a human would have.
- becquerel 3y ago
  My partner rides self-driving cars frequently, over a dozen times this year. The technology clearly and manifestly exists.
Animats 3y ago
"LLM's look poised to upend robotics." Maybe. Unstructured manipulation is getting better very slowly. But videos such as the Google one and this one [1] are mostly about making the system take instructions in natural language, not doing the actual picking. Bin-picking of uniform items is a solved problem, and was first working in the 1980s. Bin-picking of random items from a cluttered bin is still a struggle. Amazon has put a fair amount of effort into robotic bin-picking, but it's not yet good enough to deploy. There's been some good progress on bin-picking recently. Here's Brightpick's system.[2] It has a good vision system which involves LIDAR, and some nice engineering. It's not LLM-based, and it won't do everything, but it can handle most objects that can be picked up by a vacuum picker. [1] https://www.youtube.com/watch?v=31Xqw_59XZ8 https://www.youtube.com/watch?v=31Xqw_59XZ8 [2] https://www.youtube.com/watch?v=XWlQWgRfh6c https://www.youtube.com/watch?v=XWlQWgRfh6c
garspin 3y ago
The point Marcus is making is that LLMs are solely next word predictors - intelligence requires far more than that. LLMs will always have the same basic challenges - their 'kinda copy what a human has written' algorithm has hit its limit. He is an advocate of symbolic learning - getting that to work is a lot harder, however it's far more likely to result in genuine intelligence.

YetAnotherNick 3y ago

At this point either he is just trolling or writing it to get views from people who already believes this and looking for someone to say it with authority.

deleted 3y ago
[deleted]

supriyo-biswas 3y ago

Can we have a rate-limiter for submissions from the same domain, proportional to the log(TRANCO rank) of the domain? One or two posts are fine, but I’m a little tired of these hot takes that do little to acknowledge the advances the AI/ML field has made in recent years while berating it for the smallest deficiencies.

yinser 3y ago

You can’t convince me he isn’t responding to the meme of celebrating “hitting the wall day”. I can’t comment on every point he brings up here but I work for a company doing very well reading legal documents. Myth busted.

atleastoptimal 3y ago

> “We are still a long way from machines that can genuinely understand human language”. Still true It's crazy how you can just say things. If by his standards he himself can genuinely understand human language, even given how much of what he says is just smart-sounding hallucinations, then surely Claude 3 well surpasses that threshold. The secret to grifting is to only highlight your vacuous, unfalsifiable claims. He, like many AI skeptics, is a full-time goalpost mover.

uh_uh 3y ago
I only tried GPT4 but it has a way better command of language than many people I know and I include myself in that. If GPT4 doesn't understand language, then I don't either.
- aredox 3y ago
  Does MS Word "understand language" because of autocorrect?
  - ALittleLight 3y ago
    Can autocorrect hold a conversation, write code or poetry, answer or pose riddles, summarize text, write novel stories, diagnose medical issues, comprehend diagrams, etc?
    - DrFalkyn 3y ago
      ELIZA could do that 50 years ago. I haven't been really impressed with ChatGPT. It basically seems like a more useful search engine.
      - atleastoptimal 3y ago
        Are you using GPT 3.5 or GPT-4? And how are you using it? What queries are you giving it?

tdudhhu 3y ago

Deep learning is still not production ready. Maybe 90% of the results are great, but the other 10% are not only 'not great' but plain wrong and sometimes even dangerous. And I think gp posts is also showing that the last bump to make it production ready is still not taken and might even be impossible to take.

jeroenvlek 3y ago
Last time I checked ChatGPT is very much in production.
- tdudhhu 3y ago
  I mean: ready to be used in production.
  - roenxi 3y ago
    Humans aren't ready to be used in production. That hasn't stopped us so far. LLMs can't be trusted to stick to the truth but they are very much ready for production use.

Animats 3y ago

From the article: “Deep learning, which is fundamentally a technique for recognizing patterns, is at its best when all we need are rough-ready results, where stakes are low and perfect results optional“ It is somewhat embarrassing how many tasks, from customer service to sportswriting to language translation to article summarizing to concept art for films, can be handled by deep learning at its current level of competence. That says more about the problem spaces and human intelligence that it does about deep learning.

aredox 3y ago
You mean, - customer service: chatbot invents refund policy that costs the company money and reputation https://www.wired.com/story/air-canada-chatbot-refund-policy/ https://www.wired.com/story/air-canada-chatbot-refund-policy... - sports writing: a private equity group is running Sports Illustrated - which was already in decline - to the ground - language translation: not much progress, and more risks of hallucinations - concept art for films: AI is basically doing plagiarism. And so much AI art is flooding the internet that the AI are basically poisoning themselves in the future. And it still can't get the right number of fingers on hands.
becquerel 3y ago
I wouldn't really say it's embarassing. If most human tasks required perfection (or at least high levels of correctness at all times), society would probably not have gotten to where we are. Imagine a world where every job was as hard as open-heart surgery! In a sense, it's an achievement of humanity that we're able to recognise when 'good enough' will let us achieve the things we want.
tivert 3y ago
>> From the article: “Deep learning, which is fundamentally a technique for recognizing patterns, is at its best when all we need are rough-ready results, where stakes are low and perfect results optional“ > It is somewhat embarrassing how many tasks, from customer service to sportswriting to language translation to article summarizing to concept art for films, can be handled by deep learning at its current level of competence. No, it's embarrassing how eagerly leaders will compromise quality to chase automation cost savings. The things you list, especially customer service and sports-writing, cannot "be handled by deep learning at its current level of competence." That doesn't stop people from forcing the results down our throats, though.

Nevermark 3y ago

No personal criticism of this person, but: "The glass isn't full yet. Sure its filling, but its filling and its not full yet, so obviously no matter how much you fill it, we know that doesn't actually fill it." ..is my breakdown on these non-expert non-specific (mathematically, algorithmically, information theoretically, any relevant expertise at all, ...) cynics. If he had predicted the progress that was made just in the last two years, it would give some crediblity to his opinions about what wasn't going to improve. But he didn't have that insight. He has no idea what's already cooking in the research kitchen. But if am going to critque predictions, shouldn't I put myself on the line? So here is my prediction: when general AI decisively advances past us, a lot of people will feel a strong need to write and read articles about how AI doesn't have a soul. And their creations have no heart. And no progress will ever be able to fix this. Despite a lack of concrete or coherent definitions, this will matter a great deal to them.

aredox 3y ago
"I haven't run into the wall yet. Sure the wall is there, but I am running full speed and I haven't it it yet"

caesil 3y ago

>“Few fields have been more filled with hype and bravado than artificial intelligence. It has flitted from fad to fad decade by decade, always promising the moon, and only occasionally delivering” Still true. >“We are still a long way from machines that can genuinely understand human language”. Still true, though some have argued there is some superficial understanding. >“Elon Musk recently said that the new humanoid robot he was hoping to build, Optimus, would someday be bigger than the vehicle industry”. I expressed skepticism. Still early days, but certainly domestic humanoid robots are not in the near term expected to be a big business for anyone. >The company’s charismatic CEO Sam Altman wrote a triumphant blog posttrumpeting “Moore’s Law for Everything,” claiming that we were just a few years away from “computers that can think,” “read legal documents,” and (echoing IBM Watson) “give medical advice.”Maybe, but maybe not.” Pending/still true. Two years later we don’t have reliable versions of any of that. Why does anyone listen to a thing this obvious fraud says?

guender 3y ago

[dead]

clauderoux 3y ago

Gary Marcus tries to exit his personal hole of irrelevance. I have been working in the fielf of computational linguistics for 30 years. Back in 2000, I worked with a team of linguists to implement a pretty refined syntactic parser (XIP) based on Shallow Parsing, a symbolic approach. We won with this system a SemEval competition as late as 2016 on Sentiment Analysis. But I never expected LLM to reach this level of competency in my lifetime. Critics are prone to describe the errors these models make, but they seem to forget that 1. It is not a search engine and 2. it is pretty knowledgeable in a huge range of domains, which no humans can equal. I use these models every day to create code or to explain concepts to me. It never ceases to amaze me.

janalsncm 3y ago

He wrote that deep learning is “hitting a wall” 6 months before ChatGPT was released. Definitely needs to save face somehow.