4 ms·

I've seen threads involving Gary Marcus on twitter, and when people provide concrete evidence that his point of view is wrong, he just stops replying, then proc

I've seen threads involving Gary Marcus on twitter, and when people provide concrete evidence that his point of view is wrong, he just stops replying, then proceeds to continue to spout the disproven claim in other places. Just the other day, when he claimed that GPT was literally just doing memorization/word statistics/syntax and has no grasp of semantics, some folks demonstrated that GPT can literally act as an interpreter for arbitrary code. (there are some tricks involved here, you have to get it to interpret the program "with pen and paper" by getting it to record all the state updates / variable mutations that happen in the code, this can be done by inserting copious print statements) He then claimed that it was only able to interpret this program because it must have seen it before in its vast training data. He accused a commenter of not understanding how big the training data was. It was then shown that GPT can interpret a python program that is operating on two randomly chosen large integers, the combination of which is certainly not in its training data. This shows that it must be "understanding" (for lack of a better word) the semantics of the program. Gary then stopped responding. I don't think GPT on its own will lead anywhere close to AGI -- and I don't think anyone serious thinks this. But GPT combined with a sophisticated pipeline of wrapper scripts to feed its own output back into itself in clever ways and give it access to external data sources and tools? Possibly!

zwaps 4y ago
The main point here is that Large Language Models are crushing through barriers that people like Gary Marcus have previously deemed impossible to surpass. Check past episodes by these guys on Machine Learning Street Talk podcast (the whole "symbolic AI" crowd) for some gold quotes that just seem silly now! I mean, we are not that far from 2017, when LLM's couldn't really write coherent text. And, for example, we had people saying then that they would never be really coherent because text is too sparse, see Chomsky etc. etc. Instead, all they would do is to copy+paste training examples. Naturally, these people didn't really understand how transformers form interdependencencies and how they are composing sequences in a much more complex fashion - as by now is obvious. However, at they time, and with their crude understanding of these models, these guys sure were confident to be right. Next was Questions Answering, Reasoning, Math problems, Deduction, Coding... each things that LLMs could never do! They were so sure of it. And sure enough, a new model comes around that does it pretty well. Researchers are bitter for two reasons. First, We do not say that LLMs are perfect or general AI. Nobody says this. It's an obvious point not worth arguing over. But Gary Marcus et al. have become extremely popular. It's probably because some readers are slightly concerned about AI, and want someone to tell them that AI isn't really AI yet (duh) and won't replace their precious job (yet?). Their point is a strawman, and what goes beyond it (AI can't ever... X) is mostly wrong. Second, however, these people have never contributed to the actual models. They are not part of the progress that - notably - is crushing through these barriers. They are bystanders. And it's freaking annoying, because their understanding of LLM's is imperfect (as is the case for everyone!), and yet they come up with these statements of absolute certainty. There is indeed research into these questions. But it is far, very far from resolved. In that sense, these people are - sorry to say - charlatans. Well at least, everyone I know rolls their eye when the next Gary Marcus article or tweet comes around.
- cfiggers 4y ago
  > their understanding of LLM's is imperfect (as is the case for everyone!), and yet they come up with these statements of absolute certainty. Oh interesting, so when they accuse ChatGPT of being dangerous because it can be "confidently inaccurate" it's the pot calling the kettle black
  - zwaps 4y ago
    ChatGPT is trained via reinforcement learning to give answers that - ultimately - human non-experts would judge to be plausible. Answers were not trained to be correct or accurate - hence we should not expect ChatGPT to do well there. Sociologists have found that humans seek plausible answers more than correct answers, especially when trying to make sense of a situation. For that reason, the training objective makes sense for a chatbot. Now, anyone who tries to seel ChatGPT as giving truthful answers is of course doing a disfavor to the model and its training objective. However, anyone also trying to sell the above either as a) Deep insight, or b) inherent limitation of LLMs is indeed a pot calling the kettle black. Going from plausibility to truthfulness is probably not a giant leap - at least not compared to what was achieved in the past three years. There's active research on it, and I am sure a good solution will arise. However, ChatGPT is not that, and it isn't meant to be. It's not the training objective. Making this a huge point is again such a BS strawman as the other things I mentioned before.
    - mannykannot 4y ago
      > ChatGPT is trained via reinforcement learning to give answers that - ultimately - human non-experts would judge to be plausible. Ostensibly, it is being trained give replies that are more likely than most to be given by humans. On the reasonable (at least IMHO) assumption that people's speech is biased towards saying things that their target audience might see as plausible, this can be seen as doing what you say in the above quoted. This is not, however, the same as being able to judge the plausibility of the semantic content of those statements, so I remain skeptical that the current methods for training LLMs are capable of creating that ability, which in turn leads me to be skeptical that going from plausible statements to reasonably reliably true statements will be a relatively small step in comparison*. I'm open to being surprised, however, as I already have been by what has been achieved so far. The LLM-does-math paper you mention in your sibling comment looks extremely interesting, and maybe I will change my mind... * At least by a continuation or scaling-up of current methods. Humans may be more likely to say truthful things than falsehoods (at least in some areas and some contexts) but a gap has formed between plausible and truthful LLM productions, and I don't see any particular reason to think more of the same will close it.
concinds 4y ago
Yeah. I don't trust Gary Marcus, and I don't know why the media buys into his persona. Gary Marcus features a Forbes story in his Twitter bio, "7 Must-Read Books About Artificial Intelligence". That's an article which Gary Marcus paid for (that's what "Forbes Contributor" means; they're cheap, too!). This makes alarm bells go off. Marcus was one of the founders of "Geometric Intelligence", which was acquired by Uber. 3 months later, Marcus left Uber, and claimed he remained a "special advisor"[0] to Uber; when Recode said he was no longer employed at all[1]. By my reading, it's possible Geometric Intelligence was just a patent troll, and was acquired simply for its patents[2][3]. Select extracts from that Wired piece: > The company has filed for at least one patent, Marcus says. But it hasn't published research or offered a product > But Marcus paints deep neural nets as an extremely limited technology, because the vast swaths of data needed to train them aren't always available. Geometric Intelligence, he says, is building technology that can train machines with far smaller amounts of data. [uh oh; my BS detector just went off.] I heard Marcus published papers on AI; does anyone know if they're any good? Is this guy just a successful self-promoter? Why is he being paraded by media as the AI expert? Why does he sound so shady? (especially with that Forbes link, yikes; sorry but I can't take anyone seriously who pays for fake positive news stories). I mentioned this about Marcus at the end of this comment, 3 months ago: https://news.ycombinator.com/item?id=32866142 https://news.ycombinator.com/item?id=32866142 (I should also add: when the media has "go-to" experts, they're not primarily selected for their expertise, per-se, but for how "available" and eager they are to respond to all interview requests; I've seen the other side of that curtain.) [0]: https://www.axios.com/2017/12/15/the-head-of-ubers-ai-labs-is-latest-to-leave-the-company-1513300831 https://www.axios.com/2017/12/15/the-head-of-ubers-ai-labs-i... [1]: https://www.vox.com/2017/3/8/14863560/uber-ai-gary-marcus-geometric-intelligence https://www.vox.com/2017/3/8/14863560/uber-ai-gary-marcus-ge... [2]: https://pitchbook.com/profiles/company/124838-92#overview https://pitchbook.com/profiles/company/124838-92#overview [3]: https://www.wired.com/2016/12/uber-buys-mysterious-startup-make-ai-company/ https://www.wired.com/2016/12/uber-buys-mysterious-startup-m...
- feraloink 4y ago
  Gary Marcus's takes aren't credible, for all the reasons you cited (Forbes contributor... roll eyes) and this, from the interview with Ezra: >Take GPT-3. ChatGPT itself probably won’t let you do this. And you say to it, make up some misinformation about Covid and vaccines. And it will write a whole story for you, including sentences like, “A study in JAMA” — that’s one of the leading medical journals — “found that only 2% of people who took the vaccines were helped by it.” You have a news story that looks like... it was written by a human being. It’ll have all the style and form, making up its sources and data. And humans might catch one, but what if there are 10 or 100 or 1,000 or 10,000 of these? Then it becomes very difficult to monitor them. That's absurd. First, because if there's a JAMA study (love how he explains that it is Journal of American Medical Association) that is used to support even mediocre science journalism, e.g. an opinion piece in the Wall Street Journal or Newsweek, then it has an inline link to the JAMA study. Both CNN and Fox do similarly! After getting burned enough times, their reporters even learned to distinguish between MedrXiv and peer-reviewed articles. GPT-3 doesn't make fake URLs with fake associated JAMA articles. Computational journalism has been around for a long time. Gary Marcus underestimates and lacks understanding of humans AND GPT3! If GPT3 spews 10,000 fake, unsourced COVID vaccine efficacy news articles, that isn't an existential risk to humanity. GPT3 is impressive but it can't do everything (yet?) e.g. it has trouble taking derivatives of funcitons. It will give reasonable sounding answers to StackOverflow questions but the substance will be incorrect. Yes, it's annoying, but the StackOverflow OP will realize and look for help elsewhere. Gary's take: >Now everybody in the programming field uses Stack Overflow all the time. It’s like a cherished resource for everybody. It’s a place to swap information. And so many people put fake answers on this thing where it’s humans ask questions, humans give answers, that Stack Overflow had to ban people putting computer-generated answers there. It was literally existential for that website. If enough people put answers that seemed plausible but we’re not actually true, no one would go to the website anymore. Best of all: >And imagine that on a much bigger scale, the scale where you can’t trust anything on Twitter or anything on Facebook... because you don’t know which parts are true and which parts are not. Anyone who blindly trusts what they read on Twitter or Facebook has bigger problems than ChatGPT. How naive does Gary think people are?! EDIT: I am more concerned by coding contests where GPT-3 DOES run circles around human contestants.
throw310822 4y ago
> I don't think GPT on its own will lead anywhere close to AGI Serious question: why isn't ChatGPT already considered AGI? It looks general to me in its domain (text). It can indifferently: understand and execute instructions, even when provided in incomplete or ambiguous form; it can reason step by step; it can compose simple poems; it can attempt explaining jokes, it can explain and modify code, and reacts appropriately to almost any request. In all this it shows a perfectly natural "understanding" of language and context, integrated with a good amount of common sense and knowledge about the world. Of course it's not perfect, its limits show pretty quickly in almost all these fields. But still, it's general within its own domain..Aren't we just moving goalposts?
- elicksaur 4y ago
  One of your examples is the key: “attempt” explaining jokes. Really, it’s “attempting” to do all of the items you list, and can seemingly do only very simple ones. Try and get it to produce a poem that doesn’t rhyme. Try to get it to be internally consistent when explaining something abstract. Play the two truths and a lie game with it. It doesn’t understand any of the items you listed because it has no concepts, just math.
  - throw310822 4y ago
    > Try and get it to produce a poem that doesn’t rhyme Good one, I hadn't tried that. Seems that the mention of a "poem" puts it in a special frame from which it finds very hard to escape (I managed to get it to avoid rhyming in the first lines, but then it reverts to rhymes). However, > it’s “attempting” to do all of the items you list Yes, that's ok. I don't think we should confuse AGI with human-level of even superintelligence: it's perfectly ok for ChatGPT to try and not quite make it on this or that task. General intelligence doesn't mean being able to perform every task that can be performed by at least one human being; by that metric we would all fail. Neither it means not behaving in any obviously obtuse way in every situation, we would also fail that test given enough time. ChatGPT displays understanding of context, common sense, metaphors, inference, intuition, abductive reasoning. It actually answers correctly to tricky questions or scenarios that have been brought in the past as examples of what an AGI should be capable of. It's clear that it's not good enough yet to perform even a simple intellectual job reliably, but that's not really what AGI is. Have you ever interacted with people with Down's syndrome? Would you ever say that they lack general intelligence?
OrderlyTiamat 4y ago
Do you happen to have a link to that thread? I'd enjoy it.

zwaps 4y ago

The main point here is that Large Language Models are crushing through barriers that people like Gary Marcus have previously deemed impossible to surpass. Check past episodes by these guys on Machine Learning Street Talk podcast (the whole "symbolic AI" crowd) for some gold quotes that just seem silly now! I mean, we are not that far from 2017, when LLM's couldn't really write coherent text. And, for example, we had people saying then that they would never be really coherent because text is too sparse, see Chomsky etc. etc. Instead, all they would do is to copy+paste training examples. Naturally, these people didn't really understand how transformers form interdependencencies and how they are composing sequences in a much more complex fashion - as by now is obvious. However, at they time, and with their crude understanding of these models, these guys sure were confident to be right. Next was Questions Answering, Reasoning, Math problems, Deduction, Coding... each things that LLMs could never do! They were so sure of it. And sure enough, a new model comes around that does it pretty well. Researchers are bitter for two reasons. First, We do not say that LLMs are perfect or general AI. Nobody says this. It's an obvious point not worth arguing over. But Gary Marcus et al. have become extremely popular. It's probably because some readers are slightly concerned about AI, and want someone to tell them that AI isn't really AI yet (duh) and won't replace their precious job (yet?). Their point is a strawman, and what goes beyond it (AI can't ever... X) is mostly wrong. Second, however, these people have never contributed to the actual models. They are not part of the progress that - notably - is crushing through these barriers. They are bystanders. And it's freaking annoying, because their understanding of LLM's is imperfect (as is the case for everyone!), and yet they come up with these statements of absolute certainty. There is indeed research into these questions. But it is far, very far from resolved. In that sense, these people are - sorry to say - charlatans. Well at least, everyone I know rolls their eye when the next Gary Marcus article or tweet comes around.

cfiggers 4y ago
> their understanding of LLM's is imperfect (as is the case for everyone!), and yet they come up with these statements of absolute certainty. Oh interesting, so when they accuse ChatGPT of being dangerous because it can be "confidently inaccurate" it's the pot calling the kettle black
- zwaps 4y ago
  ChatGPT is trained via reinforcement learning to give answers that - ultimately - human non-experts would judge to be plausible. Answers were not trained to be correct or accurate - hence we should not expect ChatGPT to do well there. Sociologists have found that humans seek plausible answers more than correct answers, especially when trying to make sense of a situation. For that reason, the training objective makes sense for a chatbot. Now, anyone who tries to seel ChatGPT as giving truthful answers is of course doing a disfavor to the model and its training objective. However, anyone also trying to sell the above either as a) Deep insight, or b) inherent limitation of LLMs is indeed a pot calling the kettle black. Going from plausibility to truthfulness is probably not a giant leap - at least not compared to what was achieved in the past three years. There's active research on it, and I am sure a good solution will arise. However, ChatGPT is not that, and it isn't meant to be. It's not the training objective. Making this a huge point is again such a BS strawman as the other things I mentioned before.
  - mannykannot 4y ago
    > ChatGPT is trained via reinforcement learning to give answers that - ultimately - human non-experts would judge to be plausible. Ostensibly, it is being trained give replies that are more likely than most to be given by humans. On the reasonable (at least IMHO) assumption that people's speech is biased towards saying things that their target audience might see as plausible, this can be seen as doing what you say in the above quoted. This is not, however, the same as being able to judge the plausibility of the semantic content of those statements, so I remain skeptical that the current methods for training LLMs are capable of creating that ability, which in turn leads me to be skeptical that going from plausible statements to reasonably reliably true statements will be a relatively small step in comparison*. I'm open to being surprised, however, as I already have been by what has been achieved so far. The LLM-does-math paper you mention in your sibling comment looks extremely interesting, and maybe I will change my mind... * At least by a continuation or scaling-up of current methods. Humans may be more likely to say truthful things than falsehoods (at least in some areas and some contexts) but a gap has formed between plausible and truthful LLM productions, and I don't see any particular reason to think more of the same will close it.

concinds 4y ago

Yeah. I don't trust Gary Marcus, and I don't know why the media buys into his persona. Gary Marcus features a Forbes story in his Twitter bio, "7 Must-Read Books About Artificial Intelligence". That's an article which Gary Marcus paid for (that's what "Forbes Contributor" means; they're cheap, too!). This makes alarm bells go off. Marcus was one of the founders of "Geometric Intelligence", which was acquired by Uber. 3 months later, Marcus left Uber, and claimed he remained a "special advisor"[0] to Uber; when Recode said he was no longer employed at all[1]. By my reading, it's possible Geometric Intelligence was just a patent troll, and was acquired simply for its patents[2][3]. Select extracts from that Wired piece: > The company has filed for at least one patent, Marcus says. But it hasn't published research or offered a product > But Marcus paints deep neural nets as an extremely limited technology, because the vast swaths of data needed to train them aren't always available. Geometric Intelligence, he says, is building technology that can train machines with far smaller amounts of data. [uh oh; my BS detector just went off.] I heard Marcus published papers on AI; does anyone know if they're any good? Is this guy just a successful self-promoter? Why is he being paraded by media as the AI expert? Why does he sound so shady? (especially with that Forbes link, yikes; sorry but I can't take anyone seriously who pays for fake positive news stories). I mentioned this about Marcus at the end of this comment, 3 months ago: https://news.ycombinator.com/item?id=32866142 https://news.ycombinator.com/item?id=32866142 (I should also add: when the media has "go-to" experts, they're not primarily selected for their expertise, per-se, but for how "available" and eager they are to respond to all interview requests; I've seen the other side of that curtain.) [0]: https://www.axios.com/2017/12/15/the-head-of-ubers-ai-labs-is-latest-to-leave-the-company-1513300831 https://www.axios.com/2017/12/15/the-head-of-ubers-ai-labs-i... [1]: https://www.vox.com/2017/3/8/14863560/uber-ai-gary-marcus-geometric-intelligence https://www.vox.com/2017/3/8/14863560/uber-ai-gary-marcus-ge... [2]: https://pitchbook.com/profiles/company/124838-92#overview https://pitchbook.com/profiles/company/124838-92#overview [3]: https://www.wired.com/2016/12/uber-buys-mysterious-startup-make-ai-company/ https://www.wired.com/2016/12/uber-buys-mysterious-startup-m...

feraloink 4y ago
Gary Marcus's takes aren't credible, for all the reasons you cited (Forbes contributor... roll eyes) and this, from the interview with Ezra: >Take GPT-3. ChatGPT itself probably won’t let you do this. And you say to it, make up some misinformation about Covid and vaccines. And it will write a whole story for you, including sentences like, “A study in JAMA” — that’s one of the leading medical journals — “found that only 2% of people who took the vaccines were helped by it.” You have a news story that looks like... it was written by a human being. It’ll have all the style and form, making up its sources and data. And humans might catch one, but what if there are 10 or 100 or 1,000 or 10,000 of these? Then it becomes very difficult to monitor them. That's absurd. First, because if there's a JAMA study (love how he explains that it is Journal of American Medical Association) that is used to support even mediocre science journalism, e.g. an opinion piece in the Wall Street Journal or Newsweek, then it has an inline link to the JAMA study. Both CNN and Fox do similarly! After getting burned enough times, their reporters even learned to distinguish between MedrXiv and peer-reviewed articles. GPT-3 doesn't make fake URLs with fake associated JAMA articles. Computational journalism has been around for a long time. Gary Marcus underestimates and lacks understanding of humans AND GPT3! If GPT3 spews 10,000 fake, unsourced COVID vaccine efficacy news articles, that isn't an existential risk to humanity. GPT3 is impressive but it can't do everything (yet?) e.g. it has trouble taking derivatives of funcitons. It will give reasonable sounding answers to StackOverflow questions but the substance will be incorrect. Yes, it's annoying, but the StackOverflow OP will realize and look for help elsewhere. Gary's take: >Now everybody in the programming field uses Stack Overflow all the time. It’s like a cherished resource for everybody. It’s a place to swap information. And so many people put fake answers on this thing where it’s humans ask questions, humans give answers, that Stack Overflow had to ban people putting computer-generated answers there. It was literally existential for that website. If enough people put answers that seemed plausible but we’re not actually true, no one would go to the website anymore. Best of all: >And imagine that on a much bigger scale, the scale where you can’t trust anything on Twitter or anything on Facebook... because you don’t know which parts are true and which parts are not. Anyone who blindly trusts what they read on Twitter or Facebook has bigger problems than ChatGPT. How naive does Gary think people are?! EDIT: I am more concerned by coding contests where GPT-3 DOES run circles around human contestants.

throw310822 4y ago

> I don't think GPT on its own will lead anywhere close to AGI Serious question: why isn't ChatGPT already considered AGI? It looks general to me in its domain (text). It can indifferently: understand and execute instructions, even when provided in incomplete or ambiguous form; it can reason step by step; it can compose simple poems; it can attempt explaining jokes, it can explain and modify code, and reacts appropriately to almost any request. In all this it shows a perfectly natural "understanding" of language and context, integrated with a good amount of common sense and knowledge about the world. Of course it's not perfect, its limits show pretty quickly in almost all these fields. But still, it's general within its own domain..Aren't we just moving goalposts?

elicksaur 4y ago
One of your examples is the key: “attempt” explaining jokes. Really, it’s “attempting” to do all of the items you list, and can seemingly do only very simple ones. Try and get it to produce a poem that doesn’t rhyme. Try to get it to be internally consistent when explaining something abstract. Play the two truths and a lie game with it. It doesn’t understand any of the items you listed because it has no concepts, just math.
- throw310822 4y ago
  > Try and get it to produce a poem that doesn’t rhyme Good one, I hadn't tried that. Seems that the mention of a "poem" puts it in a special frame from which it finds very hard to escape (I managed to get it to avoid rhyming in the first lines, but then it reverts to rhymes). However, > it’s “attempting” to do all of the items you list Yes, that's ok. I don't think we should confuse AGI with human-level of even superintelligence: it's perfectly ok for ChatGPT to try and not quite make it on this or that task. General intelligence doesn't mean being able to perform every task that can be performed by at least one human being; by that metric we would all fail. Neither it means not behaving in any obviously obtuse way in every situation, we would also fail that test given enough time. ChatGPT displays understanding of context, common sense, metaphors, inference, intuition, abductive reasoning. It actually answers correctly to tricky questions or scenarios that have been brought in the past as examples of what an AGI should be capable of. It's clear that it's not good enough yet to perform even a simple intellectual job reliably, but that's not really what AGI is. Have you ever interacted with people with Down's syndrome? Would you ever say that they lack general intelligence?

OrderlyTiamat 4y ago

Do you happen to have a link to that thread? I'd enjoy it.