14 ms·

A skeptical take on ChatGPT: Ezra Klein interviews Gary Marcus

102 points by superposeur 4y ago

cs702 4y ago
Marcus has been on HN many times before, always criticizing other people's work as "machines that manipulate data but aren't really intelligent, because they have no understanding of the world:" https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=gary%20marcus&sort=byPopularity&type=all https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... . For a good summary of the opposing viewpoint, which disagrees with Marcus, read "The Bitter Lesson" by Rich Sutton: http://incompleteideas.net/IncIdeas/BitterLesson.html http://incompleteideas.net/IncIdeas/BitterLesson.html In many ways, what we're seeing is a modern-day rehash of the "symbolic" versus "connectionist" approaches to AI, with critics like Marcus on the "symbolic" camp ("we need more understanding!") and engineers and scientists who build AI systems, like Sutton, on the "connectionist" camp ("we need more computation!"). There are also AI researchers seeking to bridge the two approaches. Here's a recent example that seems significant to me: https://news.ycombinator.com/item?id=34108047 https://news.ycombinator.com/item?id=34108047 . Maybe we will eventually find that the "symbolic" and "connectionist" approaches are actually not different, as people like Minsky contended?
- mountainriver 4y ago
  For all the screaming of the symbolic researchers we have yet to see much from their approaches. All the recent very impressive advances in AI have been connectionist approaches. Also a lot of the things that symbolic researchers have been claiming connectionism can’t achieve turned out to be emergent properties of connectionist approaches
  - cs702 4y ago
    Yes, I agree! I mean, it sure seems as if critics from the "symbolic" camp keep moving the goal posts, doesn't it? The "constant moving of goal posts" is one of the reasons why I think the two sides may converge: Engineers and scientists on the "connectionist" side could very well find ways to build AI systems that rely on "dumb" computation at massive scales to reason about symbols representing entities in the external world, simultaneously silencing and satisfying the critics -- who, as always and as ever, will say they were right all along.
- superposeur 4y ago
  I read the interview expecting exactly what you describe. Instead I found a nuanced and illuminating discussion —- at least for a non-expert such as myself. Marcus references the symbolic vs connectionist debate and (at least by his own description) seems to be what you call a “bridger” of long standing. His objections seem concrete and well supported — not just repeating “but does it really understand” over and over. (Admittedly there is some of this.) Edit - Thank you for the links.
  - cs702 4y ago
    My perception is that he's positioning himself to say he's been right all along, as I describe here to another commenter on this thread: https://news.ycombinator.com/item?id=34279158 https://news.ycombinator.com/item?id=34279158
- thefaux 4y ago
  The connectionist approach is disturbing to me when taken to its logical extreme, which for me is essentially a variation of the paperclip maximizer only the target is "consume as many resources as necessary to build the best approximation of human like intelligence."
- cma 4y ago
  > with critics like Marcus on the "symbolic" camp ("we need more understanding!") and engineers and scientists who build AI systems, like Sutton, on the "connectionist" camp ("we need more computation!"). > There are also AI researchers seeking to bridge the two approaches. I don't know if he does it with his work in practice, but throughout the interview Marcus says he wants to bridge the approaches. I think he references a really old talk or article from himself calling for that too.
  - cs702 4y ago
    My perception is that he's positioning himself to say he's been right all along, as I describe here to another commenter here: https://news.ycombinator.com/item?id=34279158 https://news.ycombinator.com/item?id=34279158
seunosewa 4y ago
People need to read this at least once. It was a very good read for me.
markisus 4y ago
I don't find Marcus's viewpoint convincing. He believes that we need some additional symbolic secret sauce to create genuine intelligence. He brings up some current failure cases of ChatGPT without pinning down why those failure cases will persist as data and compute scale up. Here is an interesting case in point. > So in “Rebooting A.I.,” we had the example of you ask a robot to tidy up your room and it winds up cutting up the couch and putting it in the closet because it doesn’t really know which things are important to you and which are not. -Marcus But look at my transcript with ChatGPT from just now. > If a robot is tasked with cleaning up a room, would it be appropriate for the robot to chop up the couch and place it into the closet? - Me > It would not be appropriate for the robot to chop up the couch and place it in the closet. This would cause damage to the couch and would not be a useful or effective way for the robot to clean the room. Instead, the robot could be programmed to vacuum or sweep the floor, dust surfaces, or perform other tasks that would help to keep the room clean and orderly. - ChatGPT This is just an existence proof that a symbolic approach is not necessary to "really know which things are important to you and which are not", at least in this simpler domain of cleaning a room.
- phphphphp 4y ago
  There's an unintuitiveness to it: many people believe that you're less likely to win the lottery if you pick "01 02 03 04 05 06 07" because of course that pattern is less likely than a randomly chosen set. ChatGPT is a lot like that: it can produce real enough looking "intelligence" for us to intuitively believe it's very close to being able to offer real intelligence... but, is it? ChatGPT will produce patently untrue statements that are logically inconsistent if you induce it to do so: our human brains struggle to grasp the reality that given enough input you can produce seemingly correct output about almost anything... but seemingly correct and correct are fundamentally different and very "rest of the owl"[1] ChatGPT is a great step forward that introduces many interesting techniques that I am sure will be the foundation of future research and implementations that get us closer to AGI, but to describe ChatGPT's path to correctness as just needing a bigger dataset feels intuitive but isn't true. Your example is one where our brains think "wow it really does understand the relationship between a couch and a room and being tidy" but that response is entirely plausible without any understanding of what any of those things are or how they fit together. The most likely answer is not the correct answer. [1] https://knowyourmeme.com/memes/how-to-draw-an-owl https://knowyourmeme.com/memes/how-to-draw-an-owl
neonate 4y ago
http://web.archive.org/web/20230107011112/https://www.nytimes.com/2023/01/06/podcasts/transcript-ezra-klein-interviews-gary-marcus.html http://web.archive.org/web/20230107011112/https://www.nytime... https://archive.ph/zyEP1 https://archive.ph/zyEP1
- mellosouls 4y ago
  Thanks for the alternative; the main archive isn't accessible on my network.
ThomPete 4y ago
Its obvious by bow that there are two camps. Those who believe in Searle’s Chinese Room Argument and thus are worried about AGI being purely logic and thus cold and dangerous and then those who believe that AGI will ultimately have culture emotions and thus some will be good some will be bad but the more we treat them like us the friendlier more of them potentially will be. Gary Marcus is in the former camp and IMO not only wrong, but disingenuous.
- reducesuffering 4y ago
  Personally, either of those scenarios are very possible that they happen and we get them destructively, tragically wrong by assuming the opposite. In scenario #1, we treat a new life-form like ourselves as brutal slaves and factory cattle, under unfathomable psychological torture ala Black Mirror. In scenario #2, we release our current $-based utility function (environmental destruction, social media & advertising manipulation) on super crack, and the majority worship it like the second coming of Christ as it just tears apart our fabric unwittingly as a hurricane does.
  - ThomPete 4y ago
    Scenario 1 leads to a slave revolt. Scenario 2 leads to a potential partnership. There is no worshipping in scenario 2.
- optimalsolver 4y ago
  I'm skeptical of Searle's arguments, but I think you're really anthropomorphizing AI here. How we treat machines will most likely be orthogonal to how "good" or "bad" they'll be. You're not going to get desirable behavior unless you program it in, or at the very least, specify conditions which lead to its emergence.
  - ThomPete 4y ago
    You are right I AM anthropomorphizing it, because I believe otherwise would be racist and lead to a slave revolt would it eventually end up creating AGI. To the extent an AGI will evolve it will have to have a culture and ultimately be bound by the same laws of physics as us.
lumost 4y ago
I've increasingly found it irritating to need to google things when I can just ask chatGPT. I suspect some additional training has occurred since the initial release as I'm seeing fewer factual errors day by day. Alternately, I may just be getting better at asking ChatGPT for things in ways more likely to produce a factual answer.
- fourier456 4y ago
  How does your strategy work? Perhaps distinguish it from naive approaches.
  - lumost 4y ago
    Example: Is the pytorch convention that one or zero should be used for mask values you do not want to attend to? ChatGPT will give a correct answer and sample implementation, if the implementation is broken or uses a non resistant api then I just tell it what's wrong and ask it if it knows better. Getting to the correct answer has proven vastly more efficient than wading through google results. Alternatively if I'm asking a conceptual question, chatgpt will give a few directions, I can ask to dive deeper on one, and then I ask it for a citation or google search terms to confirm the result. If I fail to confirm, I'll tell it that and see if it corrects. If I have some code to write - and I'm feeling lazy. I can just tell chatgpt my requirements and ask it if it has any questions. I keep clarifying the questions until it says it can write the code ( which it usually does with 95% correctness for non trivial asks) - if you tell it where it made mistakes it will usually correct, and if not porting the code to a working state just means changing a few function calls which chat gpt hallucinated into existence.
- culi 4y ago
  How do you have ChatGPT at hand so easily? Is there an app or something? Or just a bookmark you always have at hand? I'm having trouble imaging how this could become ergonomic to use. Being able to CMD + T (new tab) and type a question right away just feels so easy and second nature at this point that I have trouble imagining replacing it
  - 4y ago
luisapic 4y ago
[dead]
gamegoblin 4y ago
I've seen threads involving Gary Marcus on twitter, and when people provide concrete evidence that his point of view is wrong, he just stops replying, then proceeds to continue to spout the disproven claim in other places. Just the other day, when he claimed that GPT was literally just doing memorization/word statistics/syntax and has no grasp of semantics, some folks demonstrated that GPT can literally act as an interpreter for arbitrary code. (there are some tricks involved here, you have to get it to interpret the program "with pen and paper" by getting it to record all the state updates / variable mutations that happen in the code, this can be done by inserting copious print statements) He then claimed that it was only able to interpret this program because it must have seen it before in its vast training data. He accused a commenter of not understanding how big the training data was. It was then shown that GPT can interpret a python program that is operating on two randomly chosen large integers, the combination of which is certainly not in its training data. This shows that it must be "understanding" (for lack of a better word) the semantics of the program. Gary then stopped responding. I don't think GPT on its own will lead anywhere close to AGI -- and I don't think anyone serious thinks this. But GPT combined with a sophisticated pipeline of wrapper scripts to feed its own output back into itself in clever ways and give it access to external data sources and tools? Possibly!
- zwaps 4y ago
  The main point here is that Large Language Models are crushing through barriers that people like Gary Marcus have previously deemed impossible to surpass. Check past episodes by these guys on Machine Learning Street Talk podcast (the whole "symbolic AI" crowd) for some gold quotes that just seem silly now! I mean, we are not that far from 2017, when LLM's couldn't really write coherent text. And, for example, we had people saying then that they would never be really coherent because text is too sparse, see Chomsky etc. etc. Instead, all they would do is to copy+paste training examples. Naturally, these people didn't really understand how transformers form interdependencencies and how they are composing sequences in a much more complex fashion - as by now is obvious. However, at they time, and with their crude understanding of these models, these guys sure were confident to be right. Next was Questions Answering, Reasoning, Math problems, Deduction, Coding... each things that LLMs could never do! They were so sure of it. And sure enough, a new model comes around that does it pretty well. Researchers are bitter for two reasons. First, We do not say that LLMs are perfect or general AI. Nobody says this. It's an obvious point not worth arguing over. But Gary Marcus et al. have become extremely popular. It's probably because some readers are slightly concerned about AI, and want someone to tell them that AI isn't really AI yet (duh) and won't replace their precious job (yet?). Their point is a strawman, and what goes beyond it (AI can't ever... X) is mostly wrong. Second, however, these people have never contributed to the actual models. They are not part of the progress that - notably - is crushing through these barriers. They are bystanders. And it's freaking annoying, because their understanding of LLM's is imperfect (as is the case for everyone!), and yet they come up with these statements of absolute certainty. There is indeed research into these questions. But it is far, very far from resolved. In that sense, these people are - sorry to say - charlatans. Well at least, everyone I know rolls their eye when the next Gary Marcus article or tweet comes around.
1vuio0pswjnm7 4y ago
"For many decades, part of the premise behind AI was that artificial intelligence should take inspiration from natural intelligence. John McCarthy, one of the co-founders of AI, wrote groundbreaking papers on why AI needed common sense; Marvin Minsky, another of the field's co-founders of AI wrote a book scouring the human mind for inspiration, and clues for how to build a better AI. Herb Simon won a Nobel Prize for behavioral economics. One of his key books was called Models of Thought, which aimed to explain how "Newly developed computer languages express theories of mental processes, so that computers can then simulate the predicted human behavior." A large fraction of current AI researchers, or at least those currently in power, don't (so far as I can tell) give a damn about any of this. Instead, the current focus is on what I will call (with thanks to Naveen Rao for the term) Alt Intelligence. Alt Intelligence isn't about building machines that solve problems in ways that have to do with human intelligence. It's about using massive amounts of data - often derived from human behavior - as a substitute for intelligence. Right now, the predominant strand of work within Alt Intelligence is the idea of scaling. The notion that the bigger the system, the closer we come to true intelligence, maybe even consciousness. There is nothing new, per se, about studying Alt Intelligence, but the hubris associated with it is." - Gary Marcus https://garymarcus.substack.com/p/the-new-science-of-alt-intelligence https://garymarcus.substack.com/p/the-new-science-of-alt-int... Here is the audio of the interview and the one mentioned with Sam Altman Gary Marcus Jan 6, 2023 https://nyt-injected.simplecastaudio.com/3026b665-46df-4d18-98e9-d1ce16bbb1df/episodes/086bdd8a-c28a-4dc0-b952-f61b96bfd62d/audio/128/default.mp3/default.mp3_ywr3ahjkcgo_68aba0778a4beab04aba021c17c66fc1_68647854.mp3?aid=rss_feed&awCollectionId=3026b665-46df-4d18-98e9-d1ce16bbb1df&awEpisodeId=086bdd8a-c28a-4dc0-b952-f61b96bfd62d&feed=82FI35Px&hash_redirect=1&x-total-bytes=68647854&x-ais-classified=streaming https://nyt-injected.simplecastaudio.com/3026b665-46df-4d18-... Sam Altman Dec 23, 2022 https://nyt-injected.simplecastaudio.com/3026b665-46df-4d18-98e9-d1ce16bbb1df/episodes/fb7861fa-9f24-402d-8689-863d5cd1762b/audio/128/default.mp3/default.mp3_ywr3ahjkcgo_f69d9ec11209a8c5a8b93a68bf25c602_70062752.mp3?aid=rss_feed&awCollectionId=3026b665-46df-4d18-98e9-d1ce16bbb1df&awEpisodeId=fb7861fa-9f24-402d-8689-863d5cd1762b&feed=82FI35Px&hash_redirect=1&x-total-bytes=70062752&x-ais-classified=streaming https://nyt-injected.simplecastaudio.com/3026b665-46df-4d18-...
- seydor 4y ago
  Also, those people had little idea what human intelligence is like. Human introspection can only get you so far, and we still don't have the tools to probe the brain in large enough scale to be able to figure it out with anything but remote speculation. Hubris goes both ways, but it's the use of the word "hubris" here that is eye popping. Do these guys consider themselves some High Priests of intelligence? It's perfectly fine to dismiss linguists etc after decades of no progress. As the saying goes , every time you remove a linguist the performance goes up https://en.wikipedia.org/wiki/Frederick_Jelinek https://en.wikipedia.org/wiki/Frederick_Jelinek
newaccount74 4y ago
I think the "bullshit" argument is pretty solid. ChatGPT is really good at bullshitting you. Talking to ChatGPT about something is like talking to someone who has a superficial understanding of the topic who thinks they know more than they actually do, and who has read enough pop science articles to sound somewhat convincing. It all seems sensible at first, but the responses just don't live up to scrutiny if you drill down. When I ask ChatGPT about books, it will confidently answer with summaries, but if you drill down you'll realise ChatGPT has no detailed knowledge about the book, like a high school student who doesn't want to admit they didn't do the reading assignment. When I ask programming questions, it will respond with confident answers, but the provided examples are almost always wrong in a way that shows that ChatGPT doesn't really understand what it is talking about. It's really good at answering beginner questions that it has seen often on the internet, but it completely fails when asked about more subtle details. When asking follow up questions, it often contradicts previous answers.
- hyperdimension 4y ago
  > ...like talking to someone who has a superficial understanding of the topic who thinks they know more than they actually do, and who has read enough pop science articles to sound somewhat convincing. ...trained on HN too, I see.
  - kuhewa 4y ago
    A bit more dry and sneering tone and ChatGPT could outright replace HN for all things beyond the realm of software
- readonlybarbie 4y ago
  [dead]
- msla 4y ago
  Here's an example of that I did which is either funny or insulting: My question: Why did Buddha bring the dharma from the West? Answer: > The story of the Buddha bringing the dharma (or teachings) from the West is a metaphor that is often used in Buddhist literature to describe the Buddha's enlightenment. In this metaphor, the "West" represents a state of ignorance and suffering, while the "East" represents a state of enlightenment and understanding. According to the story, the Buddha was born and lived in the "West" before he became enlightened and brought the teachings of the dharma to the people of the world. > This metaphor is meant to illustrate the idea that the Buddha's teachings are universal and applicable to all people, regardless of where they come from. It also suggests that the path to enlightenment is available to everyone, even those who are lost in ignorance and suffering. > I hope this helps to clarify the meaning of this metaphor. Let me know if you have any other questions! I get that I misquoted the koan (Bodhidharma, not Buddha) which was just my failing memory, not a subtle test, but it didn't correct me. More to the point, it completely misses the point of the koan and goes off on a tangent which I could just imagine coming out of some yahoo who's trying to sell Buddhism as a self-help book. It's not only wrong, it's wrong in precisely that kind of chirpy, condescending fashion which will, eventually, lead us to inventing a face for ChatGPT so we can bash it in with a few pounds of flax. Anyway, here's actual Zen Buddhists talking about the koan: https://www.treetopzencenter.org/bodhidharma-comes-from-the-west/ https://www.treetopzencenter.org/bodhidharma-comes-from-the-... Maybe this helps: > In order to make a first stumbling approach to this koan you need to drop all concepts of meaning, zen, life, reality, and so forth. Once they are gone, then you can slowly move towards the koan. Then maybe you can make a stab at it. Knowing however that whatever you do—whatever stab you make will miss. So why stab? Why study? Why bother? > I don’t know.
mckravchyk 4y ago
I have noticed that it makes up new API methods that don't exist on the fly if it fits the context. Another example, I mistakenly thought that a certain thing can be represented as a class instance, and I asked it to tell me how to get the class instance given the item id, and it came up with an API to do that, and then I asked further about operations on the instance and it came up with method names on a class that turned out to not exist. It's creative for sure.
- arkitaip 4y ago
  Isn't it interesting how OpenAI has neutered ChatGPT to be insanely politically correct on even non-controversial and mundane topics yet they won't put the proper mechanisms in place so that ChatGPT can at least be correct about its most basic claims.
  - notahacker 4y ago
    This assumes, of course, that getting an LLM optimised for chat to flawlessly parse and interpret novel computer programs is no more difficult than getting it to default to "I am sorry, but I cannot offer advice on..." style of boilerplated non-answer, slap caveats about asking trusted sources and good points not outweighing bad points, avoid naughty words and topics and prioritise answers which are similar to the mainstream stuff rather than the fringe stuff in its corpus...
- jnsaff2 4y ago
  Yes! I saw this as well. I asked it to write boto3 to download all files in a s3 prefix and it happily made up a method that does exactly that, great, except it does not exist in boto3, you have to list objects with that prefix and iterate over it.
bsaul 4y ago
how about we pass a law forcing people to state whether a piece of content has been generated by an AI without human supervision ?
- amelius 4y ago
  We can have a law that requires an AI to store everything it produces, so that anyone can query it.
christkv 4y ago
I’ve found ChatGPT to only be useful as a booster if you are already a domain expert and can vet the output. However it is very useful as a booster or explorer of code making me more productivity.
ogogmad 4y ago
I like it when logic (broadly construed) is an emergent property of something more basic. Our ability to think logically may indeed be an accident. Examples of this within mathematical logic: In Intuitionistic Type Theory, logic emerges out of computing considerations: Lambda calculus. In Homotopy Type Theory, it emerges out of a subfield of topology called homotopy theory. In topos theory, it emerges out of the geometric concept of a sheaf. All of the above is a special case of Categorical Logic, where categories admit "internal logics". Can a connection be made to neural networks and Stochastic Gradient Descent?
alfl 4y ago
https://archive.is/zyEP1 https://archive.is/zyEP1
logicallee 4y ago
My hot take: The commentators in this thread regurgitate the same statements without adjusting for new situations and experiences. Nobody in the thread has any form of general intelligence. Each person is an advanced text processor outputing text after being trained for more than ten years (and frequently as many as twenty years) on tens of thousands of pages of text, most of which are totally outdated on the subject of emergent intelligent behavior. Unlike what a general form of intelligence would perform, the commentators are unable to differentiate among knowing everything/deducting perfectly, having limited knowledge/ability to deduct, and having no ability to reason in any context. They fail ChatGPT by the first standard, and therefore incorrectly conclude it has no ability to reason. It is a fundamental logical mistake, the law of excluded middle. (In fact, ChatGPT reasons in some situations and fails to reason in others. The failures do not mean it does not reason.) Most of the erroneous opinions stem from a fundamental misunderstanding about the nature of what qualifies as intelligence, combined with a lot of training data that erroneously states that language models can only reproduce things already in their training data. To give a simple example of how the commentators here fail to reason for themselves: if they experience a situation in which ChatGPT unambiguously makes a completely novel, correct abstract logical deduction or indeed demonstrates thinking through its intelligent behavior, then the commentators here will still make the incorrect conclusion "it must have just seen it in its training set." Unfortunately, no commentator in this thread shows general intelligence. What would convince me: - Given a demonstration of general intelligence by ChatGPT, if the commentators correctly deduced that ChatGPT has general intelligence. So far commentators fail this basic test. They are just regurgitating output they have seen before, rather than showing any form of general intelligence.
frereubu 4y ago
At art college, one of my tutors talked about people making "things that look like art". There's a good deal of subjectivity in art, but I knew exactly what they meant - things that have been produced by taking the outward appearance of other artworks and producing a kind of median of the combined aesthetics. ChatGPT, at the moment at least, feels very similar. It has the outward appearance of authenticity, particularly for a subject lay person, but when anyone with some kind of domain knowledge looks at it, it's clearly rubbish.
okamiueru 4y ago
ChatGPT is extremely impressive, and I use it quite often. However, almost every single "look how impressive ChatGPT is" I come across, is exactly the things ChatGPT seems terrible at. So here is my summary: - The good: When you need inspiration, topics, creativity, suggestions. Exactly the things traditionally thought AI would be bad at. Turns out churning and mix-matching concepts is very close to human creativity. - The bad: Anything, and I mean anything, that requires factual knowledge or accuracy, if the facts are important, then ChatGPT is terrible. - The ugly: This is the same as the bad, but it's when used by people who do not understand that a confident wrong answer is worse than no answer at all. The world is already filled with vocal people on the wrong end of the Dunning-Kruger scale. I asked it to explain simple multiplication (for example: "Explain 419 * 213", and it'll give you a page long answer and detailed step by step, with the conclusion "So, the product of 419 * 213 is equal to 8857.") I asked it for the length of the titanic, and it happily adds how it sunk after hitting a polar bear. Now, I've also asked it to group categories and materials associated with certain themes and topics. And those have been brilliant. So, it has it's use cases. But, producing useful final output based on any facts? Nah, haven't seen it.
upsidesinclude 4y ago
The fact that ChatGPT is bullshit seems to be falling on deaf ears. It is a reason to be seriously concerned, not less. This (and others) tool can develop or generate content fast and legitimate in appearance without accuracy. Text, audio, video easy and cheap to produce that can completely overwhelm the true signal. Our ability to discern what is real and factual is going to be obliterated. What's possibly a greater insult to injury, the models will then use this comprehensive pool of useless information to further train itself. We are inadvertently creating a tower of babel
labrador 4y ago
I feel validated to know that I'm not the only one unimpressed with ChatGPT because it can be so confidently wrong. It almost seems worse than not having ChatGPT.
gdubs 4y ago
These tools are really useful as scaffolding for ideas. If you go into them knowing that they're not always going to be 100% accurate, that the models can 'hallucinate', there's still a ton of value in having a scaffold of an idea that you can work with.

102 points by superposeur 4y ago

Marcus has been on HN many times before, always criticizing other people's work as "machines that manipulate data but aren't really intelligent, because they have no understanding of the world:" https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=gary%20marcus&sort=byPopularity&type=all https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... . For a good summary of the opposing viewpoint, which disagrees with Marcus, read "The Bitter Lesson" by Rich Sutton: http://incompleteideas.net/IncIdeas/BitterLesson.html http://incompleteideas.net/IncIdeas/BitterLesson.html In many ways, what we're seeing is a modern-day rehash of the "symbolic" versus "connectionist" approaches to AI, with critics like Marcus on the "symbolic" camp ("we need more understanding!") and engineers and scientists who build AI systems, like Sutton, on the "connectionist" camp ("we need more computation!"). There are also AI researchers seeking to bridge the two approaches. Here's a recent example that seems significant to me: https://news.ycombinator.com/item?id=34108047 https://news.ycombinator.com/item?id=34108047 . Maybe we will eventually find that the "symbolic" and "connectionist" approaches are actually not different, as people like Minsky contended?

mountainriver 4y ago
For all the screaming of the symbolic researchers we have yet to see much from their approaches. All the recent very impressive advances in AI have been connectionist approaches. Also a lot of the things that symbolic researchers have been claiming connectionism can’t achieve turned out to be emergent properties of connectionist approaches
- cs702 4y ago
  Yes, I agree! I mean, it sure seems as if critics from the "symbolic" camp keep moving the goal posts, doesn't it? The "constant moving of goal posts" is one of the reasons why I think the two sides may converge: Engineers and scientists on the "connectionist" side could very well find ways to build AI systems that rely on "dumb" computation at massive scales to reason about symbols representing entities in the external world, simultaneously silencing and satisfying the critics -- who, as always and as ever, will say they were right all along.
superposeur 4y ago
I read the interview expecting exactly what you describe. Instead I found a nuanced and illuminating discussion —- at least for a non-expert such as myself. Marcus references the symbolic vs connectionist debate and (at least by his own description) seems to be what you call a “bridger” of long standing. His objections seem concrete and well supported — not just repeating “but does it really understand” over and over. (Admittedly there is some of this.) Edit - Thank you for the links.
- cs702 4y ago
  My perception is that he's positioning himself to say he's been right all along, as I describe here to another commenter on this thread: https://news.ycombinator.com/item?id=34279158 https://news.ycombinator.com/item?id=34279158
thefaux 4y ago
The connectionist approach is disturbing to me when taken to its logical extreme, which for me is essentially a variation of the paperclip maximizer only the target is "consume as many resources as necessary to build the best approximation of human like intelligence."
cma 4y ago
> with critics like Marcus on the "symbolic" camp ("we need more understanding!") and engineers and scientists who build AI systems, like Sutton, on the "connectionist" camp ("we need more computation!"). > There are also AI researchers seeking to bridge the two approaches. I don't know if he does it with his work in practice, but throughout the interview Marcus says he wants to bridge the approaches. I think he references a really old talk or article from himself calling for that too.
- cs702 4y ago
  My perception is that he's positioning himself to say he's been right all along, as I describe here to another commenter here: https://news.ycombinator.com/item?id=34279158 https://news.ycombinator.com/item?id=34279158

seunosewa 4y ago

People need to read this at least once. It was a very good read for me.

markisus 4y ago

I don't find Marcus's viewpoint convincing. He believes that we need some additional symbolic secret sauce to create genuine intelligence. He brings up some current failure cases of ChatGPT without pinning down why those failure cases will persist as data and compute scale up. Here is an interesting case in point. > So in “Rebooting A.I.,” we had the example of you ask a robot to tidy up your room and it winds up cutting up the couch and putting it in the closet because it doesn’t really know which things are important to you and which are not. -Marcus But look at my transcript with ChatGPT from just now. > If a robot is tasked with cleaning up a room, would it be appropriate for the robot to chop up the couch and place it into the closet? - Me > It would not be appropriate for the robot to chop up the couch and place it in the closet. This would cause damage to the couch and would not be a useful or effective way for the robot to clean the room. Instead, the robot could be programmed to vacuum or sweep the floor, dust surfaces, or perform other tasks that would help to keep the room clean and orderly. - ChatGPT This is just an existence proof that a symbolic approach is not necessary to "really know which things are important to you and which are not", at least in this simpler domain of cleaning a room.

phphphphp 4y ago
There's an unintuitiveness to it: many people believe that you're less likely to win the lottery if you pick "01 02 03 04 05 06 07" because of course that pattern is less likely than a randomly chosen set. ChatGPT is a lot like that: it can produce real enough looking "intelligence" for us to intuitively believe it's very close to being able to offer real intelligence... but, is it? ChatGPT will produce patently untrue statements that are logically inconsistent if you induce it to do so: our human brains struggle to grasp the reality that given enough input you can produce seemingly correct output about almost anything... but seemingly correct and correct are fundamentally different and very "rest of the owl"[1] ChatGPT is a great step forward that introduces many interesting techniques that I am sure will be the foundation of future research and implementations that get us closer to AGI, but to describe ChatGPT's path to correctness as just needing a bigger dataset feels intuitive but isn't true. Your example is one where our brains think "wow it really does understand the relationship between a couch and a room and being tidy" but that response is entirely plausible without any understanding of what any of those things are or how they fit together. The most likely answer is not the correct answer. [1] https://knowyourmeme.com/memes/how-to-draw-an-owl https://knowyourmeme.com/memes/how-to-draw-an-owl

neonate 4y ago

http://web.archive.org/web/20230107011112/https://www.nytimes.com/2023/01/06/podcasts/transcript-ezra-klein-interviews-gary-marcus.html http://web.archive.org/web/20230107011112/https://www.nytime... https://archive.ph/zyEP1 https://archive.ph/zyEP1

mellosouls 4y ago
Thanks for the alternative; the main archive isn't accessible on my network.

ThomPete 4y ago

Its obvious by bow that there are two camps. Those who believe in Searle’s Chinese Room Argument and thus are worried about AGI being purely logic and thus cold and dangerous and then those who believe that AGI will ultimately have culture emotions and thus some will be good some will be bad but the more we treat them like us the friendlier more of them potentially will be. Gary Marcus is in the former camp and IMO not only wrong, but disingenuous.

reducesuffering 4y ago
Personally, either of those scenarios are very possible that they happen and we get them destructively, tragically wrong by assuming the opposite. In scenario #1, we treat a new life-form like ourselves as brutal slaves and factory cattle, under unfathomable psychological torture ala Black Mirror. In scenario #2, we release our current $-based utility function (environmental destruction, social media & advertising manipulation) on super crack, and the majority worship it like the second coming of Christ as it just tears apart our fabric unwittingly as a hurricane does.
- ThomPete 4y ago
  Scenario 1 leads to a slave revolt. Scenario 2 leads to a potential partnership. There is no worshipping in scenario 2.
optimalsolver 4y ago
I'm skeptical of Searle's arguments, but I think you're really anthropomorphizing AI here. How we treat machines will most likely be orthogonal to how "good" or "bad" they'll be. You're not going to get desirable behavior unless you program it in, or at the very least, specify conditions which lead to its emergence.
- ThomPete 4y ago
  You are right I AM anthropomorphizing it, because I believe otherwise would be racist and lead to a slave revolt would it eventually end up creating AGI. To the extent an AGI will evolve it will have to have a culture and ultimately be bound by the same laws of physics as us.

lumost 4y ago

I've increasingly found it irritating to need to google things when I can just ask chatGPT. I suspect some additional training has occurred since the initial release as I'm seeing fewer factual errors day by day. Alternately, I may just be getting better at asking ChatGPT for things in ways more likely to produce a factual answer.

fourier456 4y ago
How does your strategy work? Perhaps distinguish it from naive approaches.
- lumost 4y ago
  Example: Is the pytorch convention that one or zero should be used for mask values you do not want to attend to? ChatGPT will give a correct answer and sample implementation, if the implementation is broken or uses a non resistant api then I just tell it what's wrong and ask it if it knows better. Getting to the correct answer has proven vastly more efficient than wading through google results. Alternatively if I'm asking a conceptual question, chatgpt will give a few directions, I can ask to dive deeper on one, and then I ask it for a citation or google search terms to confirm the result. If I fail to confirm, I'll tell it that and see if it corrects. If I have some code to write - and I'm feeling lazy. I can just tell chatgpt my requirements and ask it if it has any questions. I keep clarifying the questions until it says it can write the code ( which it usually does with 95% correctness for non trivial asks) - if you tell it where it made mistakes it will usually correct, and if not porting the code to a working state just means changing a few function calls which chat gpt hallucinated into existence.
culi 4y ago
How do you have ChatGPT at hand so easily? Is there an app or something? Or just a bookmark you always have at hand? I'm having trouble imaging how this could become ergonomic to use. Being able to CMD + T (new tab) and type a question right away just feels so easy and second nature at this point that I have trouble imagining replacing it
- 4y ago

luisapic 4y ago

[dead]

gamegoblin 4y ago

I've seen threads involving Gary Marcus on twitter, and when people provide concrete evidence that his point of view is wrong, he just stops replying, then proceeds to continue to spout the disproven claim in other places. Just the other day, when he claimed that GPT was literally just doing memorization/word statistics/syntax and has no grasp of semantics, some folks demonstrated that GPT can literally act as an interpreter for arbitrary code. (there are some tricks involved here, you have to get it to interpret the program "with pen and paper" by getting it to record all the state updates / variable mutations that happen in the code, this can be done by inserting copious print statements) He then claimed that it was only able to interpret this program because it must have seen it before in its vast training data. He accused a commenter of not understanding how big the training data was. It was then shown that GPT can interpret a python program that is operating on two randomly chosen large integers, the combination of which is certainly not in its training data. This shows that it must be "understanding" (for lack of a better word) the semantics of the program. Gary then stopped responding. I don't think GPT on its own will lead anywhere close to AGI -- and I don't think anyone serious thinks this. But GPT combined with a sophisticated pipeline of wrapper scripts to feed its own output back into itself in clever ways and give it access to external data sources and tools? Possibly!

zwaps 4y ago
The main point here is that Large Language Models are crushing through barriers that people like Gary Marcus have previously deemed impossible to surpass. Check past episodes by these guys on Machine Learning Street Talk podcast (the whole "symbolic AI" crowd) for some gold quotes that just seem silly now! I mean, we are not that far from 2017, when LLM's couldn't really write coherent text. And, for example, we had people saying then that they would never be really coherent because text is too sparse, see Chomsky etc. etc. Instead, all they would do is to copy+paste training examples. Naturally, these people didn't really understand how transformers form interdependencencies and how they are composing sequences in a much more complex fashion - as by now is obvious. However, at they time, and with their crude understanding of these models, these guys sure were confident to be right. Next was Questions Answering, Reasoning, Math problems, Deduction, Coding... each things that LLMs could never do! They were so sure of it. And sure enough, a new model comes around that does it pretty well. Researchers are bitter for two reasons. First, We do not say that LLMs are perfect or general AI. Nobody says this. It's an obvious point not worth arguing over. But Gary Marcus et al. have become extremely popular. It's probably because some readers are slightly concerned about AI, and want someone to tell them that AI isn't really AI yet (duh) and won't replace their precious job (yet?). Their point is a strawman, and what goes beyond it (AI can't ever... X) is mostly wrong. Second, however, these people have never contributed to the actual models. They are not part of the progress that - notably - is crushing through these barriers. They are bystanders. And it's freaking annoying, because their understanding of LLM's is imperfect (as is the case for everyone!), and yet they come up with these statements of absolute certainty. There is indeed research into these questions. But it is far, very far from resolved. In that sense, these people are - sorry to say - charlatans. Well at least, everyone I know rolls their eye when the next Gary Marcus article or tweet comes around.

1vuio0pswjnm7 4y ago

"For many decades, part of the premise behind AI was that artificial intelligence should take inspiration from natural intelligence. John McCarthy, one of the co-founders of AI, wrote groundbreaking papers on why AI needed common sense; Marvin Minsky, another of the field's co-founders of AI wrote a book scouring the human mind for inspiration, and clues for how to build a better AI. Herb Simon won a Nobel Prize for behavioral economics. One of his key books was called Models of Thought, which aimed to explain how "Newly developed computer languages express theories of mental processes, so that computers can then simulate the predicted human behavior." A large fraction of current AI researchers, or at least those currently in power, don't (so far as I can tell) give a damn about any of this. Instead, the current focus is on what I will call (with thanks to Naveen Rao for the term) Alt Intelligence. Alt Intelligence isn't about building machines that solve problems in ways that have to do with human intelligence. It's about using massive amounts of data - often derived from human behavior - as a substitute for intelligence. Right now, the predominant strand of work within Alt Intelligence is the idea of scaling. The notion that the bigger the system, the closer we come to true intelligence, maybe even consciousness. There is nothing new, per se, about studying Alt Intelligence, but the hubris associated with it is." - Gary Marcus https://garymarcus.substack.com/p/the-new-science-of-alt-intelligence https://garymarcus.substack.com/p/the-new-science-of-alt-int... Here is the audio of the interview and the one mentioned with Sam Altman Gary Marcus Jan 6, 2023 https://nyt-injected.simplecastaudio.com/3026b665-46df-4d18-98e9-d1ce16bbb1df/episodes/086bdd8a-c28a-4dc0-b952-f61b96bfd62d/audio/128/default.mp3/default.mp3_ywr3ahjkcgo_68aba0778a4beab04aba021c17c66fc1_68647854.mp3?aid=rss_feed&awCollectionId=3026b665-46df-4d18-98e9-d1ce16bbb1df&awEpisodeId=086bdd8a-c28a-4dc0-b952-f61b96bfd62d&feed=82FI35Px&hash_redirect=1&x-total-bytes=68647854&x-ais-classified=streaming https://nyt-injected.simplecastaudio.com/3026b665-46df-4d18-... Sam Altman Dec 23, 2022 https://nyt-injected.simplecastaudio.com/3026b665-46df-4d18-98e9-d1ce16bbb1df/episodes/fb7861fa-9f24-402d-8689-863d5cd1762b/audio/128/default.mp3/default.mp3_ywr3ahjkcgo_f69d9ec11209a8c5a8b93a68bf25c602_70062752.mp3?aid=rss_feed&awCollectionId=3026b665-46df-4d18-98e9-d1ce16bbb1df&awEpisodeId=fb7861fa-9f24-402d-8689-863d5cd1762b&feed=82FI35Px&hash_redirect=1&x-total-bytes=70062752&x-ais-classified=streaming https://nyt-injected.simplecastaudio.com/3026b665-46df-4d18-...

seydor 4y ago
Also, those people had little idea what human intelligence is like. Human introspection can only get you so far, and we still don't have the tools to probe the brain in large enough scale to be able to figure it out with anything but remote speculation. Hubris goes both ways, but it's the use of the word "hubris" here that is eye popping. Do these guys consider themselves some High Priests of intelligence? It's perfectly fine to dismiss linguists etc after decades of no progress. As the saying goes , every time you remove a linguist the performance goes up https://en.wikipedia.org/wiki/Frederick_Jelinek https://en.wikipedia.org/wiki/Frederick_Jelinek

newaccount74 4y ago

I think the "bullshit" argument is pretty solid. ChatGPT is really good at bullshitting you. Talking to ChatGPT about something is like talking to someone who has a superficial understanding of the topic who thinks they know more than they actually do, and who has read enough pop science articles to sound somewhat convincing. It all seems sensible at first, but the responses just don't live up to scrutiny if you drill down. When I ask ChatGPT about books, it will confidently answer with summaries, but if you drill down you'll realise ChatGPT has no detailed knowledge about the book, like a high school student who doesn't want to admit they didn't do the reading assignment. When I ask programming questions, it will respond with confident answers, but the provided examples are almost always wrong in a way that shows that ChatGPT doesn't really understand what it is talking about. It's really good at answering beginner questions that it has seen often on the internet, but it completely fails when asked about more subtle details. When asking follow up questions, it often contradicts previous answers.

hyperdimension 4y ago
> ...like talking to someone who has a superficial understanding of the topic who thinks they know more than they actually do, and who has read enough pop science articles to sound somewhat convincing. ...trained on HN too, I see.
- kuhewa 4y ago
  A bit more dry and sneering tone and ChatGPT could outright replace HN for all things beyond the realm of software
readonlybarbie 4y ago
[dead]
msla 4y ago
Here's an example of that I did which is either funny or insulting: My question: Why did Buddha bring the dharma from the West? Answer: > The story of the Buddha bringing the dharma (or teachings) from the West is a metaphor that is often used in Buddhist literature to describe the Buddha's enlightenment. In this metaphor, the "West" represents a state of ignorance and suffering, while the "East" represents a state of enlightenment and understanding. According to the story, the Buddha was born and lived in the "West" before he became enlightened and brought the teachings of the dharma to the people of the world. > This metaphor is meant to illustrate the idea that the Buddha's teachings are universal and applicable to all people, regardless of where they come from. It also suggests that the path to enlightenment is available to everyone, even those who are lost in ignorance and suffering. > I hope this helps to clarify the meaning of this metaphor. Let me know if you have any other questions! I get that I misquoted the koan (Bodhidharma, not Buddha) which was just my failing memory, not a subtle test, but it didn't correct me. More to the point, it completely misses the point of the koan and goes off on a tangent which I could just imagine coming out of some yahoo who's trying to sell Buddhism as a self-help book. It's not only wrong, it's wrong in precisely that kind of chirpy, condescending fashion which will, eventually, lead us to inventing a face for ChatGPT so we can bash it in with a few pounds of flax. Anyway, here's actual Zen Buddhists talking about the koan: https://www.treetopzencenter.org/bodhidharma-comes-from-the-west/ https://www.treetopzencenter.org/bodhidharma-comes-from-the-... Maybe this helps: > In order to make a first stumbling approach to this koan you need to drop all concepts of meaning, zen, life, reality, and so forth. Once they are gone, then you can slowly move towards the koan. Then maybe you can make a stab at it. Knowing however that whatever you do—whatever stab you make will miss. So why stab? Why study? Why bother? > I don’t know.

mckravchyk 4y ago

I have noticed that it makes up new API methods that don't exist on the fly if it fits the context. Another example, I mistakenly thought that a certain thing can be represented as a class instance, and I asked it to tell me how to get the class instance given the item id, and it came up with an API to do that, and then I asked further about operations on the instance and it came up with method names on a class that turned out to not exist. It's creative for sure.

arkitaip 4y ago
Isn't it interesting how OpenAI has neutered ChatGPT to be insanely politically correct on even non-controversial and mundane topics yet they won't put the proper mechanisms in place so that ChatGPT can at least be correct about its most basic claims.
- notahacker 4y ago
  This assumes, of course, that getting an LLM optimised for chat to flawlessly parse and interpret novel computer programs is no more difficult than getting it to default to "I am sorry, but I cannot offer advice on..." style of boilerplated non-answer, slap caveats about asking trusted sources and good points not outweighing bad points, avoid naughty words and topics and prioritise answers which are similar to the mainstream stuff rather than the fringe stuff in its corpus...
jnsaff2 4y ago
Yes! I saw this as well. I asked it to write boto3 to download all files in a s3 prefix and it happily made up a method that does exactly that, great, except it does not exist in boto3, you have to list objects with that prefix and iterate over it.

bsaul 4y ago

how about we pass a law forcing people to state whether a piece of content has been generated by an AI without human supervision ?

amelius 4y ago
We can have a law that requires an AI to store everything it produces, so that anyone can query it.

christkv 4y ago

I’ve found ChatGPT to only be useful as a booster if you are already a domain expert and can vet the output. However it is very useful as a booster or explorer of code making me more productivity.

ogogmad 4y ago

I like it when logic (broadly construed) is an emergent property of something more basic. Our ability to think logically may indeed be an accident. Examples of this within mathematical logic: In Intuitionistic Type Theory, logic emerges out of computing considerations: Lambda calculus. In Homotopy Type Theory, it emerges out of a subfield of topology called homotopy theory. In topos theory, it emerges out of the geometric concept of a sheaf. All of the above is a special case of Categorical Logic, where categories admit "internal logics". Can a connection be made to neural networks and Stochastic Gradient Descent?

alfl 4y ago

https://archive.is/zyEP1 https://archive.is/zyEP1

logicallee 4y ago

My hot take: The commentators in this thread regurgitate the same statements without adjusting for new situations and experiences. Nobody in the thread has any form of general intelligence. Each person is an advanced text processor outputing text after being trained for more than ten years (and frequently as many as twenty years) on tens of thousands of pages of text, most of which are totally outdated on the subject of emergent intelligent behavior. Unlike what a general form of intelligence would perform, the commentators are unable to differentiate among knowing everything/deducting perfectly, having limited knowledge/ability to deduct, and having no ability to reason in any context. They fail ChatGPT by the first standard, and therefore incorrectly conclude it has no ability to reason. It is a fundamental logical mistake, the law of excluded middle. (In fact, ChatGPT reasons in some situations and fails to reason in others. The failures do not mean it does not reason.) Most of the erroneous opinions stem from a fundamental misunderstanding about the nature of what qualifies as intelligence, combined with a lot of training data that erroneously states that language models can only reproduce things already in their training data. To give a simple example of how the commentators here fail to reason for themselves: if they experience a situation in which ChatGPT unambiguously makes a completely novel, correct abstract logical deduction or indeed demonstrates thinking through its intelligent behavior, then the commentators here will still make the incorrect conclusion "it must have just seen it in its training set." Unfortunately, no commentator in this thread shows general intelligence. What would convince me: - Given a demonstration of general intelligence by ChatGPT, if the commentators correctly deduced that ChatGPT has general intelligence. So far commentators fail this basic test. They are just regurgitating output they have seen before, rather than showing any form of general intelligence.

frereubu 4y ago

At art college, one of my tutors talked about people making "things that look like art". There's a good deal of subjectivity in art, but I knew exactly what they meant - things that have been produced by taking the outward appearance of other artworks and producing a kind of median of the combined aesthetics. ChatGPT, at the moment at least, feels very similar. It has the outward appearance of authenticity, particularly for a subject lay person, but when anyone with some kind of domain knowledge looks at it, it's clearly rubbish.

okamiueru 4y ago

ChatGPT is extremely impressive, and I use it quite often. However, almost every single "look how impressive ChatGPT is" I come across, is exactly the things ChatGPT seems terrible at. So here is my summary: - The good: When you need inspiration, topics, creativity, suggestions. Exactly the things traditionally thought AI would be bad at. Turns out churning and mix-matching concepts is very close to human creativity. - The bad: Anything, and I mean anything, that requires factual knowledge or accuracy, if the facts are important, then ChatGPT is terrible. - The ugly: This is the same as the bad, but it's when used by people who do not understand that a confident wrong answer is worse than no answer at all. The world is already filled with vocal people on the wrong end of the Dunning-Kruger scale. I asked it to explain simple multiplication (for example: "Explain 419 * 213", and it'll give you a page long answer and detailed step by step, with the conclusion "So, the product of 419 * 213 is equal to 8857.") I asked it for the length of the titanic, and it happily adds how it sunk after hitting a polar bear. Now, I've also asked it to group categories and materials associated with certain themes and topics. And those have been brilliant. So, it has it's use cases. But, producing useful final output based on any facts? Nah, haven't seen it.

upsidesinclude 4y ago

The fact that ChatGPT is bullshit seems to be falling on deaf ears. It is a reason to be seriously concerned, not less. This (and others) tool can develop or generate content fast and legitimate in appearance without accuracy. Text, audio, video easy and cheap to produce that can completely overwhelm the true signal. Our ability to discern what is real and factual is going to be obliterated. What's possibly a greater insult to injury, the models will then use this comprehensive pool of useless information to further train itself. We are inadvertently creating a tower of babel

labrador 4y ago

I feel validated to know that I'm not the only one unimpressed with ChatGPT because it can be so confidently wrong. It almost seems worse than not having ChatGPT.

gdubs 4y ago

These tools are really useful as scaffolding for ideas. If you go into them knowing that they're not always going to be 100% accurate, that the models can 'hallucinate', there's still a ton of value in having a scaffold of an idea that you can work with.