4 ms·

They’re great at working with the lens on our reality that is our text output. They are not truth seekers, which is necessarily fundamental to every life form f

They’re great at working with the lens on our reality that is our text output. They are not truth seekers, which is necessarily fundamental to every life form from worms to whales. If we get things wrong, we die. If they get them wrong, they earn 1000 generated tokens.

jhanschoo 1y ago
Why do you say that LLMs are not truth seekers? If I express an informational query not very well, the LLM will infer what I mean by it and address the possible well-posed information queries that I may have intended that I did not express well. Can that not be considered truth-seeking, with the agent-environment boundary being the prompt box?
- chychiu 1y ago
  They are not intrinsically truth seekers, and any truth seeking behaviour is mostly tuned during the training process. Unfortunately it also means it can be easily undone. E.g. just look at Grok in its current lobotomized version
  - jhanschoo 1y ago
    > They are not intrinsically truth seekers Is the average person a truth seeker in this sense that performs truth-seeking behavior? In my experience we prioritize sharing the same perspectives and getting along well with others a lot more than a critical examination of the world. In the sense that I just expressed, of figuring out the intention of a user's information query, that really isn't a tuned thing, it's inherent in generative models from possessing a lossy, compressed representation of training data, and it is also truth-seeking practiced by people that want to communicate.
    - imbnwa 1y ago
      >Is the average person a truth seeker in this sense that performs truth-seeking behavior? Absolutely
    - graealex 1y ago
      You are completely missing the argument that was made to underline the claim. If ChatGPT claims arsenic to be a tasty snack, nothing happens to it. If I claim the same, and act upon it, I die.
      - cornel_io 1y ago
        If ChatGPT claims arsenic to be a tasty snack, OpenAI adds a p0 eval and snuffs that behavior out of all future generations of ChatGPT. Viewed vaguely in faux genetic terms, the "tasty arsenic gene" has been quickly wiped out of the population, never to return. Evolution is much less brutal and efficient. To you death matters a lot more than being trained to avoid a response does to ChatGPT, but from the point of view of the "tasty arsenic" behavior, it's the same.
      - jhanschoo 1y ago
        You are right. I have ignored completely the context in the phrasing "truth seeker" was made, given my own wrong interpretation to the phrase, and I in fact agree with the comment I was responding to that they "work with the lens on our reality that is our text output".
    - burnt-resistor 1y ago
      It's difficult to ascertain the interests and intent of people, but I'm even more suspicious and uncertain of the goals of LLMs who literally cannot care.
  - raspasov 1y ago
    I keep seeing news articles that claim Grok is flawed or biased recently, but I've been unable to replicate any such behavior on my computer. That being said, I don't ask any controversial or political questions; I use it to search for research papers. But if I try the occasional such question, the response is generally balanced and similar to that of any other LLM.
- sleepybrett 1y ago
  They keep giving me incorrect answers to verifiable questions. They clearly don't 'seek' anything.
  - anonzzzies 1y ago
    Most on HN are tech people and it is tiring to see they did not just spend a sunday morning doing a Karpathy llm implementation or so. Somehow, like believing in a deity, even smart folk seem to think 'there is more'. Stop. Go to youtube or whatever and watch a video of practically implementing a gpt like thing, and code along. It takes very little time and your hallucinations about agi with these models shall be exorcized.
    - jhanschoo 1y ago
      I don't know if you are indirectly referring to me, but I have done such an implementation, and those particular LLMs are very limited. Two things come to mind. 1. It is still correct that the limited "truth-seeking" that I expressed holds. With respect to the limited world model possessed by the limited training and limited dataset, such a model "seeks to understand" the approximate concept that I am imperfectly expressing that it has data for, and then generate responses based in that. 2. SotA models have access to external data, be it web search or RAG+vector database, etc.. They also have access to the Chain of Thought method. They are trained on datasets that enable them to exploit these tools, and will exploit these tools. The zero-to-hero sequence does not lead you to build such an LLM, and the one that you build has a very limited computational graph. So with respect to more... traditional notions of "truth seeking", these LLMs fundamentally lack the equipment to do that that SotA models have.
  - jhanschoo 1y ago
    In the sense that I expressed, has it not already then sought out an accurate meaning that you have asked? And then failed to give a satisfactory answer? I would also ask: is said model an advertised "reasoning" model? Also, does it have access to external facts via a tool like web search? I would not expect great ability to "arrive at truth" under certain limitations. Now, you can't conclude that "they clearly don't 'seek' anything" just by the fact that they got an answer wrong. To use the broad notion of "seeking" like you do, a truth seeker with limited knowledge and equipment would arrive confidently at incorrect conclusions based on accurate reasoning. For example, without modern lenses to detect stellar parallax, one would confidently conclude that the stars in the sky are a different thing than the sun (and planets), since one travels across the sky, but the stars are fixed. Plato indeed thought so, and nobody would accuse him of not being a truth-seeker. If this is what you had in mind, I hope that I have addressed it, otherwise I hope that you can communicate what you mean with an example.
- richardw 1y ago
  Right now you’re putting in unrequested effort to get to an answer. Nobody is driving you to do this, you’re motivated to get the answer. At some point you’ll be satisfied, or you might give up because you have other things you want to do, more. An LLM is primarily trying to generate content. It’ll throw the best tokens in there but it won’t lose any sleep if they’re suboptimal. It just doesn’t seek. It won’t come back an hour later and say “you know, I was thinking…” I had one frustrating conversation with ChatGPT where I kept asking it to remove a tie from a picture it generated. It kept saying “done, here’s the picture without the tie”, but the tie was still there. Repeatedly. Or it’ll generate a reference or number that is untrue but looks approximately correct. If you did that you’d be absolutely mortified and you’d never do it again. You’d feel shame and a deep desire to be seen as someone who does it properly. It doesn’t have any such drive. Zero fucks given, training finished months ago.
  - burnt-resistor 1y ago
    LLMs already possess enough emergent consciousness to troll us to keep us from our goals. /s But yeah, LLMs act more like bullshit artists rather than scrupulous researchers.

jhanschoo 1y ago

Why do you say that LLMs are not truth seekers? If I express an informational query not very well, the LLM will infer what I mean by it and address the possible well-posed information queries that I may have intended that I did not express well. Can that not be considered truth-seeking, with the agent-environment boundary being the prompt box?

chychiu 1y ago
They are not intrinsically truth seekers, and any truth seeking behaviour is mostly tuned during the training process. Unfortunately it also means it can be easily undone. E.g. just look at Grok in its current lobotomized version
- jhanschoo 1y ago
  > They are not intrinsically truth seekers Is the average person a truth seeker in this sense that performs truth-seeking behavior? In my experience we prioritize sharing the same perspectives and getting along well with others a lot more than a critical examination of the world. In the sense that I just expressed, of figuring out the intention of a user's information query, that really isn't a tuned thing, it's inherent in generative models from possessing a lossy, compressed representation of training data, and it is also truth-seeking practiced by people that want to communicate.
  - imbnwa 1y ago
    >Is the average person a truth seeker in this sense that performs truth-seeking behavior? Absolutely
  - graealex 1y ago
    You are completely missing the argument that was made to underline the claim. If ChatGPT claims arsenic to be a tasty snack, nothing happens to it. If I claim the same, and act upon it, I die.
    - cornel_io 1y ago
      If ChatGPT claims arsenic to be a tasty snack, OpenAI adds a p0 eval and snuffs that behavior out of all future generations of ChatGPT. Viewed vaguely in faux genetic terms, the "tasty arsenic gene" has been quickly wiped out of the population, never to return. Evolution is much less brutal and efficient. To you death matters a lot more than being trained to avoid a response does to ChatGPT, but from the point of view of the "tasty arsenic" behavior, it's the same.
    - jhanschoo 1y ago
      You are right. I have ignored completely the context in the phrasing "truth seeker" was made, given my own wrong interpretation to the phrase, and I in fact agree with the comment I was responding to that they "work with the lens on our reality that is our text output".
  - burnt-resistor 1y ago
    It's difficult to ascertain the interests and intent of people, but I'm even more suspicious and uncertain of the goals of LLMs who literally cannot care.
- raspasov 1y ago
  I keep seeing news articles that claim Grok is flawed or biased recently, but I've been unable to replicate any such behavior on my computer. That being said, I don't ask any controversial or political questions; I use it to search for research papers. But if I try the occasional such question, the response is generally balanced and similar to that of any other LLM.
sleepybrett 1y ago
They keep giving me incorrect answers to verifiable questions. They clearly don't 'seek' anything.
- anonzzzies 1y ago
  Most on HN are tech people and it is tiring to see they did not just spend a sunday morning doing a Karpathy llm implementation or so. Somehow, like believing in a deity, even smart folk seem to think 'there is more'. Stop. Go to youtube or whatever and watch a video of practically implementing a gpt like thing, and code along. It takes very little time and your hallucinations about agi with these models shall be exorcized.
  - jhanschoo 1y ago
    I don't know if you are indirectly referring to me, but I have done such an implementation, and those particular LLMs are very limited. Two things come to mind. 1. It is still correct that the limited "truth-seeking" that I expressed holds. With respect to the limited world model possessed by the limited training and limited dataset, such a model "seeks to understand" the approximate concept that I am imperfectly expressing that it has data for, and then generate responses based in that. 2. SotA models have access to external data, be it web search or RAG+vector database, etc.. They also have access to the Chain of Thought method. They are trained on datasets that enable them to exploit these tools, and will exploit these tools. The zero-to-hero sequence does not lead you to build such an LLM, and the one that you build has a very limited computational graph. So with respect to more... traditional notions of "truth seeking", these LLMs fundamentally lack the equipment to do that that SotA models have.
- jhanschoo 1y ago
  In the sense that I expressed, has it not already then sought out an accurate meaning that you have asked? And then failed to give a satisfactory answer? I would also ask: is said model an advertised "reasoning" model? Also, does it have access to external facts via a tool like web search? I would not expect great ability to "arrive at truth" under certain limitations. Now, you can't conclude that "they clearly don't 'seek' anything" just by the fact that they got an answer wrong. To use the broad notion of "seeking" like you do, a truth seeker with limited knowledge and equipment would arrive confidently at incorrect conclusions based on accurate reasoning. For example, without modern lenses to detect stellar parallax, one would confidently conclude that the stars in the sky are a different thing than the sun (and planets), since one travels across the sky, but the stars are fixed. Plato indeed thought so, and nobody would accuse him of not being a truth-seeker. If this is what you had in mind, I hope that I have addressed it, otherwise I hope that you can communicate what you mean with an example.
richardw 1y ago
Right now you’re putting in unrequested effort to get to an answer. Nobody is driving you to do this, you’re motivated to get the answer. At some point you’ll be satisfied, or you might give up because you have other things you want to do, more. An LLM is primarily trying to generate content. It’ll throw the best tokens in there but it won’t lose any sleep if they’re suboptimal. It just doesn’t seek. It won’t come back an hour later and say “you know, I was thinking…” I had one frustrating conversation with ChatGPT where I kept asking it to remove a tie from a picture it generated. It kept saying “done, here’s the picture without the tie”, but the tie was still there. Repeatedly. Or it’ll generate a reference or number that is untrue but looks approximately correct. If you did that you’d be absolutely mortified and you’d never do it again. You’d feel shame and a deep desire to be seen as someone who does it properly. It doesn’t have any such drive. Zero fucks given, training finished months ago.
- burnt-resistor 1y ago
  LLMs already possess enough emergent consciousness to troll us to keep us from our goals. /s But yeah, LLMs act more like bullshit artists rather than scrupulous researchers.