4 ms·

Well it's cool that they released a paper, but at this point it's been 11 months and you can't download a Titans-architecture model code or weights anywhere. Th

by mapmeld 10mo ago

Well it's cool that they released a paper, but at this point it's been 11 months and you can't download a Titans-architecture model code or weights anywhere. That would put a lot of companies up ahead of them (Meta's Llama, Qwen, DeepSeek). Closest you can get is an unofficial implementation of the paper https://github.com/lucidrains/titans-pytorch https://github.com/lucidrains/titans-pytorch

informal007 10mo ago
I don't think model code is a big deal compared to the idea. If public can recognize the value of idea 11 months ago, they could implement the code quickly because there are so much smart engineers in AI field.
- jstummbillig 10mo ago
  If that is true, does it follow this idea does not actually have a lot of value?
  - deleted 10mo ago
    [deleted]
  - fancy_pantser 10mo ago
    Student: Look, there’s hundred dollar bill on the ground! Economist: No there isn’t. If there were, someone would have picked it up already. To wit, it's dangerous to assume the value of this idea based on the lack of public implementations.
    - lukas099 10mo ago
      If the hundred dollar bill was in an accessible place and the fact of its existence had been transmitted to interested parties worldwide, then yeah, the economist would probably be right.
    - NavinF 10mo ago
      That day the student was the 100th person to pick it up, realize it's fake, and drop it
    - dotancohen 10mo ago
      In my opinion, a refined analogy would be: Student: Look, a well known financial expert placed what could potentially be a hundred dollar bill on the ground, other well-known financial experts just leave it there!
- mapmeld 10mo ago
  Well we have the idea and the next best thing to official code, but if this was a big revelation where are all of the Titan models? If this were public, I think we'd have a few attempts at variants (all of the Mamba SSMs, etc.) and get a better sense if this is valuable or not.
alyxya 10mo ago
The hardest part about making a new architecture is that even if it is just better than transformers in every way, it’s very difficult to both prove a significant improvement at scale and gain traction. Until google puts in a lot of resources into training a scaled up version of this architecture, I believe there’s plenty of low hanging fruit with improving existing architectures such that it’ll always take the back seat.
- UltraSane 10mo ago
  Yes. The path dependence for current attention based LLMs is enormous.
  - patapong 10mo ago
    At the same time, there is now a ton of data for training models to act as useful assistants, and benchmarks to compare different assistant models. The wide availability and ease of obtaining new RLHF training data will make it more feasible to build models on new architectures I think.
- m101 10mo ago
  Prove it beats models of different architectures trained under identical limited resources?
- p1esk 10mo ago
  Until google puts in a lot of resources into training a scaled up version of this architecture If Google is not willing to scale it up, then why would anyone else?
  - 8note 10mo ago
    chatgpt is an example on why.
    - falcor84 10mo ago
      You think that this might be another ChatGPT/Docker/Hadoop case, where Google comes up with the technology but doesn't care to productize it?
root_axis 10mo ago
I don't think the comparison is valid. Releasing code and weights for an architecture that is widely known is a lot different than releasing research about an architecture that could mitigate fundamental problems that are common to all LLM products.
innagadadavida 10mo ago
Just keep in mind it is performance review time for all the tech companies. Their promotion of these seems to be directly correlated with that event.
SilverSlash 10mo ago
The newer one is from late May: https://arxiv.org/abs/2505.23735 https://arxiv.org/abs/2505.23735
AugSun 10mo ago
Gemini 3 _is_ that architecture.
- FpUser 10mo ago
  I've read many very positive reviews about Gemini 3. I tried using it including Pro and to me it looks very inferior to ChatGPT. What was very interesting though was when I caught it bullshitting me I called its BS and Gemini expressed very human like behavior. It did try to weasel its way out, degenerated down to "true Scotsman" level but finally admitted that it was full of it. this is kind of impressive / scary.
mupuff1234 10mo ago
> it's been 11 months Is that supposed to be a long time? Seems fair that companies don't rush to open up their models.

by mapmeld 10mo ago

informal007 10mo ago

I don't think model code is a big deal compared to the idea. If public can recognize the value of idea 11 months ago, they could implement the code quickly because there are so much smart engineers in AI field.

jstummbillig 10mo ago
If that is true, does it follow this idea does not actually have a lot of value?
- deleted 10mo ago
  [deleted]
- fancy_pantser 10mo ago
  Student: Look, there’s hundred dollar bill on the ground! Economist: No there isn’t. If there were, someone would have picked it up already. To wit, it's dangerous to assume the value of this idea based on the lack of public implementations.
  - lukas099 10mo ago
    If the hundred dollar bill was in an accessible place and the fact of its existence had been transmitted to interested parties worldwide, then yeah, the economist would probably be right.
  - NavinF 10mo ago
    That day the student was the 100th person to pick it up, realize it's fake, and drop it
  - dotancohen 10mo ago
    In my opinion, a refined analogy would be: Student: Look, a well known financial expert placed what could potentially be a hundred dollar bill on the ground, other well-known financial experts just leave it there!
mapmeld 10mo ago
Well we have the idea and the next best thing to official code, but if this was a big revelation where are all of the Titan models? If this were public, I think we'd have a few attempts at variants (all of the Mamba SSMs, etc.) and get a better sense if this is valuable or not.

alyxya 10mo ago

The hardest part about making a new architecture is that even if it is just better than transformers in every way, it’s very difficult to both prove a significant improvement at scale and gain traction. Until google puts in a lot of resources into training a scaled up version of this architecture, I believe there’s plenty of low hanging fruit with improving existing architectures such that it’ll always take the back seat.

UltraSane 10mo ago
Yes. The path dependence for current attention based LLMs is enormous.
- patapong 10mo ago
  At the same time, there is now a ton of data for training models to act as useful assistants, and benchmarks to compare different assistant models. The wide availability and ease of obtaining new RLHF training data will make it more feasible to build models on new architectures I think.
m101 10mo ago
Prove it beats models of different architectures trained under identical limited resources?
p1esk 10mo ago
Until google puts in a lot of resources into training a scaled up version of this architecture If Google is not willing to scale it up, then why would anyone else?
- 8note 10mo ago
  chatgpt is an example on why.
  - falcor84 10mo ago
    You think that this might be another ChatGPT/Docker/Hadoop case, where Google comes up with the technology but doesn't care to productize it?

root_axis 10mo ago

I don't think the comparison is valid. Releasing code and weights for an architecture that is widely known is a lot different than releasing research about an architecture that could mitigate fundamental problems that are common to all LLM products.

innagadadavida 10mo ago

Just keep in mind it is performance review time for all the tech companies. Their promotion of these seems to be directly correlated with that event.

SilverSlash 10mo ago

The newer one is from late May: https://arxiv.org/abs/2505.23735 https://arxiv.org/abs/2505.23735

AugSun 10mo ago

Gemini 3 _is_ that architecture.

FpUser 10mo ago
I've read many very positive reviews about Gemini 3. I tried using it including Pro and to me it looks very inferior to ChatGPT. What was very interesting though was when I caught it bullshitting me I called its BS and Gemini expressed very human like behavior. It did try to weasel its way out, degenerated down to "true Scotsman" level but finally admitted that it was full of it. this is kind of impressive / scary.

mupuff1234 10mo ago

> it's been 11 months Is that supposed to be a long time? Seems fair that companies don't rush to open up their models.