4 ms·

> much of the thinking happens in a much higher dimensional space that just happens to be decoded as text. What do you mean by that? It’s literally text predic

> much of the thinking happens in a much higher dimensional space that just happens to be decoded as text. What do you mean by that? It’s literally text prediction, isn’t it?

cyanydeez 7mo ago
There was a paper recently that demonstrated that you can input different human languages and the middle layers of the model end up operating on the same probabilistic vectors. It's just the encoding/decoding layers that appear to do the language management. So the conclusion was that these middle layers have their own language and it's converting the text into this language and this decoding it. It explains why sometime the models switch to chinese when they have a lot of chinese language inputs, etc.
- DrewADesign 7mo ago
  Ok — that sounds more like a theory rather than an open-and-shut causal explanation, but I’ll read the paper.
  - trenchgun 7mo ago
    You’re a literature cycle behind. ‘Middle-layer shared representations exist’ is the observed phenomenon; ‘why exactly they form’ is the theory. You are also confusing ‘mechanistic explanation still incomplete’ with ‘empirical phenomenon unestablished.’ Those are not the same thing. PS. Em dash? So you are some LLM bot trying to bait mine HN for reasoning traces? :D
    - DrewADesign 7mo ago
      Oh, Jesus Christ. I learned to write at a college with a strict style guide that taught us how to use different types of punctuation to juxtapose two ideas in one sentence. In fact, they did/do a bunch of LLM work so if anyone ever used student data to train models, I’m probably part of the reason they do that. You sound like you’re trying to sound impressive. Like I said, I’ll read the paper.
      - cyanydeez 7mo ago
        Congrats on reading.
        DrewADesign 7mo ago
        Sick burn
- skydhash 7mo ago
  Pretty obvious when you think that neural networks operate with numbers and very complex formulas (by combining several simple formulas with various weights). You can map a lot of things to number (words, colors, music notes,…) but that does not means the NN is going to provide useful results.
  - DrewADesign 7mo ago
    Everything is obvious if you ignore enough of the details/problem space. I’ll read the paper rather than rely on my own thought experiments and assumptions.
pennaMan 7mo ago
>It’s literally text prediction, isn’t it? you are discovering that the favorite luddite argument is bullshit
- DrewADesign 7mo ago
  Feel free to elucidate if you want to add anything to this thread other than vibes.
  - electroglyph 7mo ago
    after you go from from millions of params to billions+ models start to get weird (depending on training) just look at any number of interpretability research papers. Anthropic has some good ones.
    - DrewADesign 7mo ago
      Getting weird doesn’t mean calling it text prediction is actually ‘bullshit’? Text prediction isn’t pejorative…
    - HumanOstrich 7mo ago
      > things start to get weird > just look at research papers You didn't add anything other than vibes either.
    - Barbing 7mo ago
      Interesting, what kind of weird?
- ericjmorey 7mo ago
  I don't consider these researchers luddites. https://machinelearning.apple.com/research/illusion-of-thinking https://machinelearning.apple.com/research/illusion-of-think... https://arxiv.org/abs/2508.01191 https://arxiv.org/abs/2508.01191
K0balt 7mo ago
It is text prediction. But to predict text, other things follow that need to be calculated. If you can step back just a minute, i can provide a very simple but adjacent idea that might help to intuit the complexity of “ text prediction “ . I have a list of numbers, 0 to9, and the + , = operators. I will train my model on this dataset, except the model won’t get the list, they will get a bunch of addition problems. A lot. But every addition problem possible inside that space will not be represented, not by a long shot, and neither will every number. but still, the model will be able to solve any math problem you can form with those symbols. It’s just predicting symbols, but to do so it had to internalize the concepts.
- qsera 7mo ago
  >internalize the concepts. This gives the impression that it is doing something more than pattern matching. I think this kind of communication where some human attribute is used to name some concept in the LLM domain is causing a lot of damage, and ends up inadvertently blowing up the hype for the AI marketing...
  - K0balt 7mo ago
    Except I actually mean to infer the concept of adding things from examples. LLMs are amply capable of applying concepts to data that matches patterns not ever expressed in the training data. It’s called inference for a reason. Anthropomorphic descriptions are the most expressive because of the fact that LLMs based on human cultural output mimic human behaviours, intrinsically. Other terminology is not nearly as expressive when describing LLM output. Pattern matching is the same as saying text prediction. While being technically truthy, it fails to convey the external effect. Anthropomorphic terms, while being less truthy overall, do manage to effectively convey the external effect. It does unfortunately imply an internal cause that does not follow, but the externalities are what matter in most non-philosophical contexts.
    - qsera 7mo ago
      >do manage to effectively convey the external effect But the problem is that this does not inform about the failure mode. So if I am understanding correctly, you are saying that the behavior of LLM, when it works, is like it has internalized the concepts. But then it does not inform that it can also say stuff that completely contradicts what it said before, there by also contradicting the notion of having "internalized" the concept. So that will turn out to be a lie.

> much of the thinking happens in a much higher dimensional space that just happens to be decoded as text. What do you mean by that? It’s literally text prediction, isn’t it?

cyanydeez 7mo ago

There was a paper recently that demonstrated that you can input different human languages and the middle layers of the model end up operating on the same probabilistic vectors. It's just the encoding/decoding layers that appear to do the language management. So the conclusion was that these middle layers have their own language and it's converting the text into this language and this decoding it. It explains why sometime the models switch to chinese when they have a lot of chinese language inputs, etc.

DrewADesign 7mo ago
Ok — that sounds more like a theory rather than an open-and-shut causal explanation, but I’ll read the paper.
- trenchgun 7mo ago
  You’re a literature cycle behind. ‘Middle-layer shared representations exist’ is the observed phenomenon; ‘why exactly they form’ is the theory. You are also confusing ‘mechanistic explanation still incomplete’ with ‘empirical phenomenon unestablished.’ Those are not the same thing. PS. Em dash? So you are some LLM bot trying to bait mine HN for reasoning traces? :D
  - DrewADesign 7mo ago
    Oh, Jesus Christ. I learned to write at a college with a strict style guide that taught us how to use different types of punctuation to juxtapose two ideas in one sentence. In fact, they did/do a bunch of LLM work so if anyone ever used student data to train models, I’m probably part of the reason they do that. You sound like you’re trying to sound impressive. Like I said, I’ll read the paper.
    - cyanydeez 7mo ago
      Congrats on reading.
      - DrewADesign 7mo ago
        Sick burn
skydhash 7mo ago
Pretty obvious when you think that neural networks operate with numbers and very complex formulas (by combining several simple formulas with various weights). You can map a lot of things to number (words, colors, music notes,…) but that does not means the NN is going to provide useful results.
- DrewADesign 7mo ago
  Everything is obvious if you ignore enough of the details/problem space. I’ll read the paper rather than rely on my own thought experiments and assumptions.

pennaMan 7mo ago

>It’s literally text prediction, isn’t it? you are discovering that the favorite luddite argument is bullshit

DrewADesign 7mo ago
Feel free to elucidate if you want to add anything to this thread other than vibes.
- electroglyph 7mo ago
  after you go from from millions of params to billions+ models start to get weird (depending on training) just look at any number of interpretability research papers. Anthropic has some good ones.
  - DrewADesign 7mo ago
    Getting weird doesn’t mean calling it text prediction is actually ‘bullshit’? Text prediction isn’t pejorative…
  - HumanOstrich 7mo ago
    > things start to get weird > just look at research papers You didn't add anything other than vibes either.
  - Barbing 7mo ago
    Interesting, what kind of weird?
ericjmorey 7mo ago
I don't consider these researchers luddites. https://machinelearning.apple.com/research/illusion-of-thinking https://machinelearning.apple.com/research/illusion-of-think... https://arxiv.org/abs/2508.01191 https://arxiv.org/abs/2508.01191

K0balt 7mo ago

It is text prediction. But to predict text, other things follow that need to be calculated. If you can step back just a minute, i can provide a very simple but adjacent idea that might help to intuit the complexity of “ text prediction “ . I have a list of numbers, 0 to9, and the + , = operators. I will train my model on this dataset, except the model won’t get the list, they will get a bunch of addition problems. A lot. But every addition problem possible inside that space will not be represented, not by a long shot, and neither will every number. but still, the model will be able to solve any math problem you can form with those symbols. It’s just predicting symbols, but to do so it had to internalize the concepts.

qsera 7mo ago
>internalize the concepts. This gives the impression that it is doing something more than pattern matching. I think this kind of communication where some human attribute is used to name some concept in the LLM domain is causing a lot of damage, and ends up inadvertently blowing up the hype for the AI marketing...
- K0balt 7mo ago
  Except I actually mean to infer the concept of adding things from examples. LLMs are amply capable of applying concepts to data that matches patterns not ever expressed in the training data. It’s called inference for a reason. Anthropomorphic descriptions are the most expressive because of the fact that LLMs based on human cultural output mimic human behaviours, intrinsically. Other terminology is not nearly as expressive when describing LLM output. Pattern matching is the same as saying text prediction. While being technically truthy, it fails to convey the external effect. Anthropomorphic terms, while being less truthy overall, do manage to effectively convey the external effect. It does unfortunately imply an internal cause that does not follow, but the externalities are what matter in most non-philosophical contexts.
  - qsera 7mo ago
    >do manage to effectively convey the external effect But the problem is that this does not inform about the failure mode. So if I am understanding correctly, you are saying that the behavior of LLM, when it works, is like it has internalized the concepts. But then it does not inform that it can also say stuff that completely contradicts what it said before, there by also contradicting the notion of having "internalized" the concept. So that will turn out to be a lie.