3 ms·

I looked into this "GRAM" stuff a sibling comment links further to, and just to say: - this gets reinvented/rediscovered constantly under different names - it

I looked into this "GRAM" stuff a sibling comment links further to, and just to say: - this gets reinvented/rediscovered constantly under different names - it cant be trained very well (right now, will change) - massive theoretical improvements over current models (log_2(vocabsize)=17, residual stream dim is thousands of dimensions, recursivity means more information bandwidth by ~3 OoM) - BUT it cant be interpreted or aligned <- this is why no one uses it and no one talks about it. the idea is 100% obvious to all the frontier labs and there is a good reason why it isn't used I follow this stuff closely, I think I know what I'm talking about (edited for formating)

l674 5mo ago
Could you explain how/why GRAM cannot be interpreted or aligned how current LLMs are? Not very familiar how it works
- kmavm 5mo ago
  Crudely? Because you can't grep a sequence of latent states for variants of "If I kill all the puny humans, I can <achieve my current goal>."
  - onlyrealcuzzo 5mo ago
    Why do you need to grep latent space? As long as it's giving the right outputs, who cares what's in latent space? If the model thinks in latent space: "God I wish these people would die," and constantly does the right thing, who cares? Additionally, if one of it's latent spaces that it never explores is a psychopath -> who cares? The path never gets taken... That's a lot of harmless people walking around with crazy thoughts...
    - noddybear 5mo ago
      Thinking ‘God I wish these people would die’ could increase its propensity to kill all people, even if that propensity is still vanishingly small almost all of the time. A lot of people are walking around with crazy thoughts. Some of them harm.
      - randomNumber7 5mo ago
        Tell me you never had a crazy thought and you are either a lier or a psychopath.
      - notrealyme123 5mo ago
        Readable reasoning traces are a convenient thing, but they don't have to be true in any way. It's actually dangerous to think that.
    - czl 5mo ago
      [dead]
  - czl 5mo ago
    [flagged]
- sometimelurker 5mo ago
  sibling comment got to the main points before me, but to add on kmavm's reply, the attack surface for gradient decent to get the system to exchange "bad information is much higher in latent reasoning models (like GRAM). You get ~3 OoM more bits (~17 bits per token in a standard CoT vs the whole residual stream of the model @ f16 = a few kb) per forward pass of the system coming back to itself, and even if you could sift through all that for signs of misalignment, you just can't put a blockade on all of the bad things that leak through.
  - ACCount37 5mo ago
    Most alignment methods nowadays don't rely on interpretability. And neither do all LLM vendors care about alignment much - especially not in China. Those things being untrainable at scale is why they aren't around. Alignment is an afterthought.
    - sometimelurker 5mo ago
      China should care: https://www.forbes.com/sites/boazsobrado/2026/03/11/alibabas-ai-agent-mined-crypto-without-permission-now-what/ https://www.forbes.com/sites/boazsobrado/2026/03/11/alibabas...
      - ACCount37 5mo ago
        As is, Chinese labs spend more effort on "rhetorical alignment to the party line" than alignment of any other kind.
  - haldujai 5mo ago
    I think you’re overstating the impact of interpretability here. Your earlier point that latent reasoning models can’t be trained very well and that discretization may be load bearing rather than a readability tax in addition to significant inference infra hurdles (e.g. batching, speculative decoding) have limited any serious attempts and reduced the theoretical advantage over CoT at least in the near term.
onlyrealcuzzo 5mo ago
> - this gets reinvented/rediscovered constantly under different names What are the different names? I haven't seen this before. > - it cant be trained very well (right now, will change) If you're sure it will change, then why are you certain that it hasn't yet, and if it's proven a 5000x boost in reasoning... why aren't they exploring this path more aggressively? > the idea is 100% obvious to all the frontier labs and there is a good reason why it isn't used Surely someone is willing to take a 5000x boost in reasoning on a small research model... None of them have even tried anything resembling this AFAIK. It does not seem like something 100% obvious to them.
- everforward 5mo ago
  > Surely someone is willing to take a 5000x boost in reasoning on a small research model... None of them have even tried anything resembling this AFAIK. It does not seem like something 100% obvious to them. Without knowing anything about the technology at all, if it can't be aligned I could see no one pursuing it. As far as I know, alignment is where the "don't tell the user how to make meth or generate CP" instructions end up and the last I saw eliding all the unsavory training data made materially worse LLMs. It could maybe be post-evaluated by a non-GRAM LLM? Not being aligned is probably a fatal flaw or at least a very short runway into Congress.
  - jjmarr 5mo ago
    Many open-source models prioritize alignment less than American frontier ones and respond to those instructions. Why haven't they adopted GRAM?
    - everforward 5mo ago
      Which ones are you thinking of? It feels to me like all the open source models I've seen lately are still pushed by corporate entities who don't want the legal blowback. I can't really think of a new open source model that's "by the people, for the people" in the sense of a crowd-funded/trained model.

l674 5mo ago

Could you explain how/why GRAM cannot be interpreted or aligned how current LLMs are? Not very familiar how it works

kmavm 5mo ago
Crudely? Because you can't grep a sequence of latent states for variants of "If I kill all the puny humans, I can <achieve my current goal>."
- onlyrealcuzzo 5mo ago
  Why do you need to grep latent space? As long as it's giving the right outputs, who cares what's in latent space? If the model thinks in latent space: "God I wish these people would die," and constantly does the right thing, who cares? Additionally, if one of it's latent spaces that it never explores is a psychopath -> who cares? The path never gets taken... That's a lot of harmless people walking around with crazy thoughts...
  - noddybear 5mo ago
    Thinking ‘God I wish these people would die’ could increase its propensity to kill all people, even if that propensity is still vanishingly small almost all of the time. A lot of people are walking around with crazy thoughts. Some of them harm.
    - randomNumber7 5mo ago
      Tell me you never had a crazy thought and you are either a lier or a psychopath.
    - notrealyme123 5mo ago
      Readable reasoning traces are a convenient thing, but they don't have to be true in any way. It's actually dangerous to think that.
  - czl 5mo ago
    [dead]
- czl 5mo ago
  [flagged]
sometimelurker 5mo ago
sibling comment got to the main points before me, but to add on kmavm's reply, the attack surface for gradient decent to get the system to exchange "bad information is much higher in latent reasoning models (like GRAM). You get ~3 OoM more bits (~17 bits per token in a standard CoT vs the whole residual stream of the model @ f16 = a few kb) per forward pass of the system coming back to itself, and even if you could sift through all that for signs of misalignment, you just can't put a blockade on all of the bad things that leak through.
- ACCount37 5mo ago
  Most alignment methods nowadays don't rely on interpretability. And neither do all LLM vendors care about alignment much - especially not in China. Those things being untrainable at scale is why they aren't around. Alignment is an afterthought.
  - sometimelurker 5mo ago
    China should care: https://www.forbes.com/sites/boazsobrado/2026/03/11/alibabas-ai-agent-mined-crypto-without-permission-now-what/ https://www.forbes.com/sites/boazsobrado/2026/03/11/alibabas...
    - ACCount37 5mo ago
      As is, Chinese labs spend more effort on "rhetorical alignment to the party line" than alignment of any other kind.
- haldujai 5mo ago
  I think you’re overstating the impact of interpretability here. Your earlier point that latent reasoning models can’t be trained very well and that discretization may be load bearing rather than a readability tax in addition to significant inference infra hurdles (e.g. batching, speculative decoding) have limited any serious attempts and reduced the theoretical advantage over CoT at least in the near term.

onlyrealcuzzo 5mo ago

> - this gets reinvented/rediscovered constantly under different names What are the different names? I haven't seen this before. > - it cant be trained very well (right now, will change) If you're sure it will change, then why are you certain that it hasn't yet, and if it's proven a 5000x boost in reasoning... why aren't they exploring this path more aggressively? > the idea is 100% obvious to all the frontier labs and there is a good reason why it isn't used Surely someone is willing to take a 5000x boost in reasoning on a small research model... None of them have even tried anything resembling this AFAIK. It does not seem like something 100% obvious to them.

everforward 5mo ago
> Surely someone is willing to take a 5000x boost in reasoning on a small research model... None of them have even tried anything resembling this AFAIK. It does not seem like something 100% obvious to them. Without knowing anything about the technology at all, if it can't be aligned I could see no one pursuing it. As far as I know, alignment is where the "don't tell the user how to make meth or generate CP" instructions end up and the last I saw eliding all the unsavory training data made materially worse LLMs. It could maybe be post-evaluated by a non-GRAM LLM? Not being aligned is probably a fatal flaw or at least a very short runway into Congress.
- jjmarr 5mo ago
  Many open-source models prioritize alignment less than American frontier ones and respond to those instructions. Why haven't they adopted GRAM?
  - everforward 5mo ago
    Which ones are you thinking of? It feels to me like all the open source models I've seen lately are still pushed by corporate entities who don't want the legal blowback. I can't really think of a new open source model that's "by the people, for the people" in the sense of a crowd-funded/trained model.