5 ms·

You picked one trivial failure and built an entire worldview around it while ignoring the tidal wave of success stories that define what these models can alread

You picked one trivial failure and built an entire worldview around it while ignoring the tidal wave of success stories that define what these models can already do. ChatGPT can draft legal documents, debug code in multiple languages, generate functional architectures, summarize thousand page reports, compose music, write poetry, design marketing campaigns, and tutor students in real time. It can hold domain specific conversations with doctors, engineers, and lawyers and produce coherent, context aware reasoning that would have been considered impossible five years ago. And you’re pointing to a single pizza dough error as if that somehow invalidates all of it. If that’s your bar, then every human who ever made a mistake in a kitchen is disqualified from being intelligent too. You’re cherry picking the single dumbest moment and pretending it defines the whole picture. It doesn’t. The real story is that these models already demonstrate reasoning and generalization across virtually every intellectual domain. They write, argue, and problem solve with flexibility and intent. They’re not perfect, but perfection was never the standard. The Turing test was passed the moment you could no longer draw a clear line between where imitation ends and understanding begins. You can sneer about yeast all you want, but the irony is that while you mock, the machines are already doing useful work coding, researching, analyzing, and creating, quietly exceeding every benchmark that once defined general intelligence.

dns_snek 1y ago
> You’re cherry picking the single dumbest moment and pretending it defines the whole picture. It doesn’t. Yes, it's just one example that betrays their lack of conceptual understanding but it's not a cherry-picked example like you keep repeating. It's an entire failure mode that I discovered by myself because while the average ChatGPT user enjoys sycophancy and quick, simple answer, I value correctness and "I don't know". > The Turing test was passed the moment you could no longer draw a clear line between where imitation ends and understanding begins. Where? By whom? How was the test designed? Citation needed. > You can sneer about yeast all you want, but the irony is that while you mock, the machines are already doing useful work coding, researching, analyzing, and creating, quietly exceeding every benchmark that once defined general intelligence. Did you have an AI write this entire sales pitch? I've seen all the "useful" work LLMs are doing, especially with coding. It's not going to be pretty when all this puffery is forced to confront reality in a few months - maybe a few years. It's hard to tell how long they can keep up the act.
- ninetyninenine 1y ago
  You’re not uncovering some grand flaw, you’re just mistaking noise for revelation. The yeast example doesn’t prove a lack of understanding, it proves that when the framing changes, the response changes. That’s not confusion, that’s how reasoning works. Humans do it constantly. Ask a question one way, you get one answer. Ask it another, you trigger a different heuristic. You’d call that “intuition” if it came from a person. You talk about “valuing correctness” as if that’s a unique virtue, but what you’re really demanding is godlike infallibility. A human chef could have told you the exact same wrong thing with the same misplaced confidence, and you wouldn’t declare the entire field of culinary knowledge invalid. One bad answer isn’t a philosophical collapse. It’s just one bad answer. You ask “where” and “by whom” the Turing test was passed, as if it’s some formal event with a ribbon cutting. It was passed the moment you needed to start asking that question. The line between imitation and understanding disappeared while people were still arguing over definitions. You don’t see it because you’re too busy guarding a finish line that’s already behind you. And the “sales pitch” insult is just a refuge when the facts don’t bend your way. You can call it hype, but the work exists. The tools are shipping code, drafting legal arguments, summarizing research, and running quietly in the background of every major industry. You don’t have to like it, but pretending it’s all theater just makes you look like the guy on the shore yelling that the tide isn’t real.
  - dns_snek 1y ago
    > Ask a question one way, you get one answer. Ask it another, you trigger a different heuristic. You’d call that “intuition” if it came from a person. No, This is the person who aces certain types of exams but doesn't know how to apply any of that "knowledge" in the real world because it was built upon rote memorization and not a true understanding. They're not looked upon favorably and I'd quietly question their intelligence. > A human chef could have told you the exact same wrong thing with the same misplaced confidence You're missing the point. The problem is that the answer changes depending on how the question is presented. Chefs are indeed often mistaken in their understanding of how salt and yeast interact and how vulnerable yeast is to being killed by salt. The difference is that their answer doesn't flip like this. Their answer is fundamentally based upon an underlying understanding (however wrong), it's not simply a statistical continuation sequence of the words in my question. This has serious implications because when a chef's understanding is corrected, they will update their underlying understanding and automatically apply that new knowledge in every scenario that relies on this understanding, where as an LLM can't do that, it can never be corrected without rewriting every instance of the training data where this falsehood comes from. > You ask “where” and “by whom” the Turing test was passed, as if it’s some formal event with a ribbon cutting No, I expect a research paper where the test design and its results are documented. > The tools are shipping code Yes I've seen the code they "ship", that's why your grandiose claims ring hollow.
Forgeties79 1y ago
> You picked one trivial failure and built an entire worldview around it They gave one small example of a larger problem we have all seen, including yourself I’m sure. This is hardly a fair response. It’s also incredibly easy to verify it was wrong when given a very simple question. That’s the really alarming thing. This isn’t niche information. Their example really effectively highlights one of the major problems with LLM’s: you can’t use them unless you already have at least a foundational knowledge of what you are asking because you have to be able to identify the inevitable mistakes it regularly makes. So someone like me who is not very confident cooking cannot use an LLM to help in the kitchen. It is going to get me in trouble on a regular basis. And if I have to search online to double check everything it says, why am I using it in the first place? Now apply this to car maintenance, coding, gardening, etc. all of which are fair examples because everyone seems so intent on making generalized LLM “assistants.” I am not saying LLM’s are worthless, just to be clear. But there are severe limitations that seem to get glossed over constantly, especially when it’s brought up that you have to be able to regularly verify the information which is very regularly at least partially wrong.
- ninetyninenine 1y ago
  I disagree. Not with the part that it gets things severely wrong at times. I disagree that it gets glossed over. It’s fucking obvious how wrong LLMs get and I think this part is over exaggerated while the parts it gets right like basically slowly replacing parts of your average HNers day job as a programmer is deliberately ignored and scoffed at. The sentiment against AI by the overwhelming majority is hatred and negativity especially on HN. It is a small minority (especially among entrepreneurs and founders) that are overly positive about AI. But make no mistake the overwhelming sentiment is negative to the point of delusion like the LLMs themselves. Like it’s unmistakable to me how LLMs can basically up developer productivity to a much much higher degree than ever before. Yet we have plenty and plenty of people who can’t even take the middle ground and say it kind of helps. All kinds of developers everywhere saying LLMs are fucking completely useless. Which is mind bogglingly irrational. Most Artists for example are decrying it because it produces soulless work. I agree the work is soulless but indistinguishable and often materially better than what a human can do. In fact the concept of soul becomes utter bullshit in a double blind test. They aren’t decrying it because it’s soulless that’s bullshit. They decry it because it’s on the trajectory of becoming better than them. That’s the same fucking reason you see HNers siding with the best possible scaffolding of logic and reasoning that will support their day job. That’s why you see people claiming random shit like LLMs don’t actually understand anything when we in fact have no clue or ability to even properly answer that question.

dns_snek 1y ago

> You’re cherry picking the single dumbest moment and pretending it defines the whole picture. It doesn’t. Yes, it's just one example that betrays their lack of conceptual understanding but it's not a cherry-picked example like you keep repeating. It's an entire failure mode that I discovered by myself because while the average ChatGPT user enjoys sycophancy and quick, simple answer, I value correctness and "I don't know". > The Turing test was passed the moment you could no longer draw a clear line between where imitation ends and understanding begins. Where? By whom? How was the test designed? Citation needed. > You can sneer about yeast all you want, but the irony is that while you mock, the machines are already doing useful work coding, researching, analyzing, and creating, quietly exceeding every benchmark that once defined general intelligence. Did you have an AI write this entire sales pitch? I've seen all the "useful" work LLMs are doing, especially with coding. It's not going to be pretty when all this puffery is forced to confront reality in a few months - maybe a few years. It's hard to tell how long they can keep up the act.

ninetyninenine 1y ago
You’re not uncovering some grand flaw, you’re just mistaking noise for revelation. The yeast example doesn’t prove a lack of understanding, it proves that when the framing changes, the response changes. That’s not confusion, that’s how reasoning works. Humans do it constantly. Ask a question one way, you get one answer. Ask it another, you trigger a different heuristic. You’d call that “intuition” if it came from a person. You talk about “valuing correctness” as if that’s a unique virtue, but what you’re really demanding is godlike infallibility. A human chef could have told you the exact same wrong thing with the same misplaced confidence, and you wouldn’t declare the entire field of culinary knowledge invalid. One bad answer isn’t a philosophical collapse. It’s just one bad answer. You ask “where” and “by whom” the Turing test was passed, as if it’s some formal event with a ribbon cutting. It was passed the moment you needed to start asking that question. The line between imitation and understanding disappeared while people were still arguing over definitions. You don’t see it because you’re too busy guarding a finish line that’s already behind you. And the “sales pitch” insult is just a refuge when the facts don’t bend your way. You can call it hype, but the work exists. The tools are shipping code, drafting legal arguments, summarizing research, and running quietly in the background of every major industry. You don’t have to like it, but pretending it’s all theater just makes you look like the guy on the shore yelling that the tide isn’t real.
- dns_snek 1y ago
  > Ask a question one way, you get one answer. Ask it another, you trigger a different heuristic. You’d call that “intuition” if it came from a person. No, This is the person who aces certain types of exams but doesn't know how to apply any of that "knowledge" in the real world because it was built upon rote memorization and not a true understanding. They're not looked upon favorably and I'd quietly question their intelligence. > A human chef could have told you the exact same wrong thing with the same misplaced confidence You're missing the point. The problem is that the answer changes depending on how the question is presented. Chefs are indeed often mistaken in their understanding of how salt and yeast interact and how vulnerable yeast is to being killed by salt. The difference is that their answer doesn't flip like this. Their answer is fundamentally based upon an underlying understanding (however wrong), it's not simply a statistical continuation sequence of the words in my question. This has serious implications because when a chef's understanding is corrected, they will update their underlying understanding and automatically apply that new knowledge in every scenario that relies on this understanding, where as an LLM can't do that, it can never be corrected without rewriting every instance of the training data where this falsehood comes from. > You ask “where” and “by whom” the Turing test was passed, as if it’s some formal event with a ribbon cutting No, I expect a research paper where the test design and its results are documented. > The tools are shipping code Yes I've seen the code they "ship", that's why your grandiose claims ring hollow.

Forgeties79 1y ago

> You picked one trivial failure and built an entire worldview around it They gave one small example of a larger problem we have all seen, including yourself I’m sure. This is hardly a fair response. It’s also incredibly easy to verify it was wrong when given a very simple question. That’s the really alarming thing. This isn’t niche information. Their example really effectively highlights one of the major problems with LLM’s: you can’t use them unless you already have at least a foundational knowledge of what you are asking because you have to be able to identify the inevitable mistakes it regularly makes. So someone like me who is not very confident cooking cannot use an LLM to help in the kitchen. It is going to get me in trouble on a regular basis. And if I have to search online to double check everything it says, why am I using it in the first place? Now apply this to car maintenance, coding, gardening, etc. all of which are fair examples because everyone seems so intent on making generalized LLM “assistants.” I am not saying LLM’s are worthless, just to be clear. But there are severe limitations that seem to get glossed over constantly, especially when it’s brought up that you have to be able to regularly verify the information which is very regularly at least partially wrong.

ninetyninenine 1y ago
I disagree. Not with the part that it gets things severely wrong at times. I disagree that it gets glossed over. It’s fucking obvious how wrong LLMs get and I think this part is over exaggerated while the parts it gets right like basically slowly replacing parts of your average HNers day job as a programmer is deliberately ignored and scoffed at. The sentiment against AI by the overwhelming majority is hatred and negativity especially on HN. It is a small minority (especially among entrepreneurs and founders) that are overly positive about AI. But make no mistake the overwhelming sentiment is negative to the point of delusion like the LLMs themselves. Like it’s unmistakable to me how LLMs can basically up developer productivity to a much much higher degree than ever before. Yet we have plenty and plenty of people who can’t even take the middle ground and say it kind of helps. All kinds of developers everywhere saying LLMs are fucking completely useless. Which is mind bogglingly irrational. Most Artists for example are decrying it because it produces soulless work. I agree the work is soulless but indistinguishable and often materially better than what a human can do. In fact the concept of soul becomes utter bullshit in a double blind test. They aren’t decrying it because it’s soulless that’s bullshit. They decry it because it’s on the trajectory of becoming better than them. That’s the same fucking reason you see HNers siding with the best possible scaffolding of logic and reasoning that will support their day job. That’s why you see people claiming random shit like LLMs don’t actually understand anything when we in fact have no clue or ability to even properly answer that question.