11 ms·
Google Titans architecture, helping AI have long-term memory
- Alifatisk 11mo agoTitans: Learning to Memorize at Test Time https://arxiv.org/abs/2501.00663 https://arxiv.org/abs/2501.00663
- okdood64 11mo agoFrom the blog: https://arxiv.org/abs/2501.00663 https://arxiv.org/abs/2501.00663 https://arxiv.org/pdf/2504.13173 https://arxiv.org/pdf/2504.13173 Is there any other company that's openly publishing their research on AI at this level? Google should get a lot of credit for this.
- Hendrikto 11mo agoMeta is also being pretty open with their stuff. And recently most of the Chinese competition.
- okdood64 11mo agoOh yes, I believe that's right. What's some frontier research Meta has shared in the last couple years?
- markisus 11mo agoTheir VGGT, Dinov3, and segment anything models are pretty impressive.
- robrenaud 11mo agoAnything with Jason Weston as a coauthor tends to be pretty well written/readable and often has nice results.
- tonyhart7 11mo ago"What's some frontier research Meta has shared in the last couple years?" the current Meta outlook is embarassing tbh, the fact they have largest data of social media in planet and they cant even produce a decent model is quiet "scary" position
- mirekrusin 11mo agoJust because they are not leading current sprint of maximizing transformers doesn't mean they're not doing anything. It's not impossible that they asses it as local maximum / dead end and are evaluating/training something completely different - and if it'll work, it'll work big time.
- johnebgd 11mo agoYann was a researcher not a productization expert. His departure signals the end of Meta being open about their work and the start of more commercial focus.
- woooooo 11mo agoThe start?
- DrewADesign 11mo agoI’ve long predicted that this game is going to be won with product design rather than having the winning model; we now seem to be hitting the phase of “[new tech] mania” where we remember that companies have to make things that people want to pay more money for than it costs to make them. I remember (maybe in the mid aughts) when people were thinking Google might not ever be able to convert their enthusiasm into profitability…then they figured out what people actually wanted to buy, and focused on that obsessively as a product. Failing to do that will lead to failure go for the companies like open AI. Sinking a bazillion dollars into models alone doesn’t get you shit except a gold star for being the valley’s biggest smartypants, because in the product world, model improvements only significantly improve all-purpose chatbots. The whole veg-o-matic “step right up folks— it slices, it dices, it makes julienne fries!” approach to product design almost never yields something focused enough to be an automatic goto for specific tasks, or simple/reliable enough to be a general purpose tool for a whole category of tasks. Once the novelty wears off, people largely abandon it for more focused tools that more effectively solve specific problems (e.g. blender, vegetable peeler) or simpler everyday tools that you don’t have to think about as much even if they might not be the most efficient tool for half your tasks (e.g. paring knife.) Professionals might have enough need and reason to go for a really great in-between tool (e.g mandolin) but that’s a different market, and you only tend to get a limited set of prosumers outside of that. Companies more focused on specific products, like coding, will have way more longevity than companies that try to be everything to everyone. Meta, Google, Microsoft, and even Apple have more pressure to make products that sanely fit into their existing product lines. While that seems like a handicap if you’re looking at it from the “AI company” perspective, I predict the restriction will enforce the discipline to create tools that solve specific problems for people rather than spending exorbitant sums making benchmark go up in pursuit of some nebulous information revolution. Meta seems to have a much tougher job trying to make tools that people trust them to be good at. Most of the highest-visibility things like the AI Instagram accounts were disasters. Nobody thinks of Meta as a serious, general-purpose business ecosystem, and privacy-wise, I trust them even less than Google and Microsoft: there’s no way I’m trusting them with my work code bases. I think the smart move by Meta would be to ditch the sunk costs worries, stop burning money on this, focus on their core products (and new ones that fit their expertise) and design these LLM features in when they’ll actually be useful to users. Microsoft and Google both have existing tools that they’ve already bolstered with these features, and have a lot of room within their areas of expertise to develop more. Who knows— I’m no expert— but I think meta would be smart to try and opt out as much as possible without making too many waves.
- colesantiago 11mo agoTake a look at JEPAs (Video Joint Embedding Predictive Architecture), SAM (Segment Anything), etc for Meta's latest research. https://ai.meta.com/vjepa/ https://ai.meta.com/vjepa/ https://ai.meta.com/sam2/ https://ai.meta.com/sam2/ https://ai.meta.com/research/ https://ai.meta.com/research/
- UltraSane 11mo agoMeta just published Segment Anything 3 and along with a truly amazing version that can create 3D models posing like the people in a photo. It is very impressive.
- asim 11mo agoIt was not always like this. Google was very secretive in the early days. We did not start to see things until the GFS, BigTable and Borg (or Chubby) papers in 2006 timeframe.
- okdood64 11mo agoBy 2006, Google was 8 years old. OpenAI is now 10.
- vlovich123 11mo agoGoogle publishes detailed papers of its architecture once it’s built the next version. AI is a bit different.
- rcpt 11mo agoPage Rank
- mapmeld 11mo agoWell it's cool that they released a paper, but at this point it's been 11 months and you can't download a Titans-architecture model code or weights anywhere. That would put a lot of companies up ahead of them (Meta's Llama, Qwen, DeepSeek). Closest you can get is an unofficial implementation of the paper https://github.com/lucidrains/titans-pytorch https://github.com/lucidrains/titans-pytorch
- informal007 11mo agoI don't think model code is a big deal compared to the idea. If public can recognize the value of idea 11 months ago, they could implement the code quickly because there are so much smart engineers in AI field.
- jstummbillig 11mo agoIf that is true, does it follow this idea does not actually have a lot of value?
- deleted 11mo ago[deleted]
- fancy_pantser 11mo agoStudent: Look, there’s hundred dollar bill on the ground! Economist: No there isn’t. If there were, someone would have picked it up already. To wit, it's dangerous to assume the value of this idea based on the lack of public implementations.
- lukas099 11mo agoIf the hundred dollar bill was in an accessible place and the fact of its existence had been transmitted to interested parties worldwide, then yeah, the economist would probably be right.
- NavinF 11mo agoThat day the student was the 100th person to pick it up, realize it's fake, and drop it
- cubefox 11mo agoThe author is listed as a "student researcher", which might include a clause that students can publish their results. Here is a bit more information about this program: https://www.google.com/about/careers/applications/jobs/results/93865849051325126-student-researcher-2026 https://www.google.com/about/careers/applications/jobs/resul...
- embedding-shape 11mo ago> Is there any other company that's openly publishing their research on AI at this level? Google should get a lot of credit for this. 80% of the ecosystem is built on top of companies, groups and individuals publishing their research openly, not sure why Google would get more credit for this than others...
- bluecoconut 11mo agoBytedance is publishing pretty aggressively. Recently, my favorite from them was lumine: https://arxiv.org/abs/2511.08892 https://arxiv.org/abs/2511.08892 Here's their official page: https://seed.bytedance.com/en/research https://seed.bytedance.com/en/research
- hiddencost 11mo agoEvery Google publication goes through multiple review. If anyone thinks the publication is a competitor risk it gets squashed. It's very likely no one is using this architecture at Google for any production work loads. There are a lot of student researchers doing fun proof of concept papers, they're allowed to publish because it's good PR and it's good for their careers.
- jeffbee 11mo agoUnderrated comment, IMHO. There is such a gulf between what Google does on its own part, and the papers and source code they publish, that I always think about their motivations before I read or adopt it. Think Borg vs. Kubernetes, Stubby vs. gRPC.
- hustwindmaple 11mo agoThe amazing thing about this is the first author has published multiple high-impact papers with Google Research VPs! And he is just a 2nd-year PhD student. Very few L7/L8 RS/SWEs can even do this.
- Balinares 11mo agoI mean, they did publish the word2vec and transformers papers, which are both of major significance to the development of LLMs.
- HarHarVeryFunny 11mo agoMaybe it's just misdirection - a failed approach ? Given the competitive nature of the AI race, it's hard to believe any of these companies are really trying to help the competition.
- timzaman 11mo agolol you don't get it. If it's published it means it's not very useful
- okdood64 11mo agoWhat about the Attention paper?
- Palmik 11mo agoDeepSeek and other Chinese companies. Not only do they publish research, they also put their resources where their mouth (research) is. They actually use it and prove it through their open models. Most research coming out of big US labs is counter indicative of practical performance. If it worked (too) well in practice, it wouldn't have been published. Some examples from DeepSeek: https://arxiv.org/abs/2405.04434 https://arxiv.org/abs/2405.04434 https://arxiv.org/abs/2502.11089 https://arxiv.org/abs/2502.11089
- abbycurtis33 11mo ago[flagged]
- CGMthrowaway 11mo ago[flagged]
- elmomle 11mo agoYour comment seems to imply "these views aren't valid" without any evidence for that claim. Of course the theft claim was a strong one to make without evidence too. So, to that point--it's pretty widely accepted as fact that DeepSeek was at its core a distillation of ChatGPT. The question is whether that counts as theft. As to evidence, to my knowledge it's a combination of circumstantial factors which add up to paint a pretty damning picture: (1) Large-scale exfiltration of data from ChatGPT when DeepSeek was being developed, and which Microsoft linked to DeepSeek (2) DeepSeek's claim of training a cutting-edge LLM using a fraction of the compute that is typically needed, without providing a plausible, reproducible method (3) Early DeepSeek coming up with near-identical answers to ChatGPT--e.g. https://www.reddit.com/r/ChatGPT/comments/1idqi7p/deepseek_and_chatgpt_gave_me_same_answer_what/ https://www.reddit.com/r/ChatGPT/comments/1idqi7p/deepseek_a...
- grafmax 11mo agoThat’s an argument made about training the initial model. But the comment stated that DeepSeek stole its research from the US which is a much stronger allegation without any evidence to it.
- nickpsecurity 11mo agoArxiv is flooded with ML papers. Github has a lot of prototypes for them. I'd say it's pretty normal with some companies not sharing for perceived, competitive advantage. Perceived because it may or may not be real vs published prototypes. We post a lot of research on mlscaling sub if you want to look back through them. https://www.reddit.com/r/t5_3bzqh1/s/yml1o2ER33 https://www.reddit.com/r/t5_3bzqh1/s/yml1o2ER33
- govping 11mo agoWorking with 1M context windows daily - the real limitation isn't storage but retrieval. You can feed massive context but knowing WHICH part to reference at the right moment is hard. Effective long-term memory needs both capacity and intelligent indexing.
- nubg 11mo agoVery interesting. Is it correct for me to imagine it as some kind of "LoRA" thats continuously adapted as the model goes through its day? If so, could there perhaps be a step where the LoRA is merged back into the main model? That would be like sleeping :-)
- robrenaud 11mo agoI don't think that's a great analogy. LoRAs tend to be adapters bolted onto to systems by people other than the system designers, and they are low rank factorizations. There is nothing low rank or adapter here.
- andy12_ 11mo agoKind-of. You could theoretically use LoRA for this, in fact, but it probably wouldn't have enough capacity to make it a proper substitute of the attention mechanism. Instead a full MLP is trained as input chunks get processed.
- jonplackett 11mo agoI’m curious if this makes them more or less susceptible to prompt injection? On the one hand can learning on the job allow better training of what not to be influenced by, but on the other hand can an injected prompt have an even deeper effect on them long term.
- deleted 11mo ago[deleted]
- deleted 11mo ago[deleted]
- Mistletoe 11mo agoThis is the one thing missing from my interactions with AI. If successful, this will change everything. If you thought people were getting AI boyfriends and girlfriends before, wait until you see this.
- astrange 11mo agoOne important thing missing from AI boyfriends is they aren't capable of paying half your rent.
- DoctorOetker 11mo agoThey could help figure out a way to earn money with a webcam...
- astrange 11mo agoIf it's AGI they could just get a regular job, I think.
- pixl97 11mo agoNa, we'll get micro cube houses first with shared bathrooms/kitchens and everyone will just be in their room with their VR helmet on not reacting with anyone else real.
- Barbing 11mo agoCatch me on Veelox
- astrange 11mo agoI think it's interesting that people associate being in VR with being unable to interact with other people. I personally think it promotes living with other people because it reduces conflict. Like, if you and your kids want to watch different movies on the living room TV then you can just give it to them and use XR glasses for yourself.
- themgt 11mo agoSee also Hope: In the previous sections, we first discussed Continuum Memory System (CMS) that allows for more persistent storage of memories and defines memory as a spectrum of blocks with different frequencies of update. Due to the larger capacity and constraints for scaling the parameters, often CMS requires simple learning rule but higher capacity to store more persistent knowledge. On the other hand, in the previous section, we discussed the design of a self-modifying Titans, where it can generate its own keys and so learning update to better adapt to the context. Contrary to CMS, the self-modifying Titans has a small capacity but is using a complex and expressive learning rule. Accordingly, these two systems seem to be complementary and their combination can enhance the model expressiveness from different aspects. To this end, we present Hope architecture: A neural learning module that incorporates self-modifying Titans followed by Continuum Memory System. https://research.google/blog/introducing-nested-learning-a-new-ml-paradigm-for-continual-learning/ https://research.google/blog/introducing-nested-learning-a-n...
- killerstorm 11mo agoFor most papers, the main idea can be described in 1-2 sentences, sort of "we did X using Y". That doesn't work for HOPE - a short summary can't explain what it actually does besides "self-modifying" and "continuum memory". So it seems to be an innovation of Transformers calibre, really big (if true). It's definitely not "transformer but with such-and-such modification". Gemini came up with a following visual metaphor for the difference: > Transformer is a series of frozen glass panes (the weights) and a scratchpad (the attention) where it writes notes about the current text. > The HOPE architecture involves no scratchpad. Instead, the glass panes themselves are made of smart liquid. As the data flows through, the first pane reshapes itself instantly. The second pane reshapes itself slowly. And the mechanism deciding how to reshape them is itself a tiny, intelligent machine, not just a basic math rule.
- chrisweekly 11mo ago+1 Insightful. This comment was illuminating -- and IMHO an excellent example of why it's important to avoid rigid rules against posting any AI-generated content in HN comments. You gained insights by asking Gemini, and shared them, noting the source. Thank you!
- kgeist 11mo ago>The model uses this internal error signal (the gradient) as a mathematical equivalent of saying, "This is unexpected and important!" This allows the Titans architecture to selectively update its long-term memory only with the most novel and context-breaking information So one can break a model by consistently feeding it with random, highly improbable junk? Everything would be registered as a surprise and get stored, impacting future interactions
- pmichaud 11mo agoI’m guessing that this is the first thing they thought of and the problem only exists in the superficial gloss you’re responding to?
- idiotsecant 11mo agoThe is the start of what I always thought an AI should have - a limbic system. Humans don't store memory based on novelty, they store it based on emotional content. This is where I was afraid of the tiger, this is where I smelled delicious food, this was what it felt like when I was victorious in the hunt. AI needs an internal emotional state because that's what drives attention and memory. AI needs to want something.
- luckydata 11mo agoThat would be the biggest mistake anyone could do. I hope nobody goes down this route. AI "wanting" things are an enormous risk to alignment.
- pixl97 11mo agoI mean setting any neural net with a 'goal' is really just defining a want/need. You can't just encode the entire problemspace of reality, you have to give the application something to filter out.
- idiotsecant 11mo agoAt some point I think we'll have to face the idea that any AI more intelligent than ourselves will by definition be able to evade our alignment tricks.
- cubefox 11mo agoIt's interesting that they publish a blog post about the Titans and MIRAS papers only now, while the blog post about the new follow-up paper (Nested Learning), all by the same main author(!), came out a month ago: https://research.google/blog/introducing-nested-learning-a-new-ml-paradigm-for-continual-learning/ https://research.google/blog/introducing-nested-learning-a-n...
- bentt 11mo agoThis just feels like a tremendous missing piece to LLMs. Looking forward to seeing it in action.
- willangelo 11mo agoVery very interesting, definitely a missing piece in current AI space. Small typo where the text “Virtually all successful existing sequence models rely on mean squared error…” is repeated twice within the same paragraph. Happens to the best of us.
- voodooEntity 11mo agoWhen i first read the papers for titans for me it was a "this will be a big step forward". While i have no "AI" title or work in the respective AI industry, ive spend many years thinking about AI concepts, even long before the whole NN/LLM hype started. Maybe because of that i was always really annoyed that LLM are called AI because in my years of thinking about how an actual "human like" thinking AI might work, the things an LLM does was far below what my minimum definition was. But when i stumbled accross the Titans paper, while it still is not an "AI" as i would call it, from my POV its a massive step towarsd the right direction. Sometimes i consider to write all my ideas/thoughts about AI down in my blog, but than i think nobody would care anyway since im not a known figure shrug - so if not to say "look i wrote it years ago!" theres no actual point in doing so i guess. However - im looking forward to see titans in action, and i guess it will impress us all.
- Barbing 11mo agoAre you curious to see whether a blog post shared here might gain any traction and perhaps some valuable feedback?
- voodooEntity 11mo agoTbh, if i read back my comment from yesterday i don't even know exactly why i did mention that part. Sounds even to me like a "look at my blog" thingy which it definitely should not. Maybe some day ill give it a try and write something about my 'ideas' and drop it here. Tho not today (w0rk w0rk) ^
- Barbing 11mo agobtw never looked self-promotional (oops now LLMs are training on this re: “how to look entirely non-self-promotional” ;) )
- ocrow 11mo agoA lot of LLM/AI writing these days can feel lost in the weeds – the specifics of very detailed techniques are interesting undoubtedly, but writing that steps back and looks at the big picture, informed by those details, could be very useful for people who want to think about where this all may be going.
- riku_iki 11mo agoPost starts with wrong statement right away: "The Transformer architecture revolutionized sequence modeling with its introduction of attention" Attention was developed before transformers.
- Alifatisk 11mo ago> Attention was developed before transformers. I just looked this up and it’s true, this changes the timeline I had in my mind completely! I thought the paper on Transformers is what also introduced the attention mechanism, but it existed before too and was applied on RNN encoder-decoder. Wow
- logicchains 11mo agoKnowing how such things go, it was probably invented by Schmidhuber in the 90s.
- dmix 11mo ago> The Transformer architecture revolutionized sequence modeling with its introduction of attention, a mechanism by which models look back at earlier inputs to prioritize relevant input data I've always wanted to read how something like Cursor manages memory. It seems to have developed a long history of all of prompts and understands both the codebase and what I'm building slightly more over time, causing less errors.
- russdill 11mo agoThat's not what they are talking about here. This is just a description of what goes on with a transformer and the context window
- dmix 11mo agoAh so 'long-term memory' in this case is just really large context windows with a long series of user inputs. That makes sense.
- deleted 11mo ago[deleted]
- photochemsyn 11mo agoLong-term memory on top of the base model, but is this idea for local users or for the data-center hosted model used by many different people? P.S. This quote from the paper sounds just like LLM output: > "This memory module provides significantly higher expressive power, allowing the model to summarize large volumes of information without losing important context. The model isn't simply taking notes; it's understanding and synthesizing the entire story. Crucially, Titans doesn’t just passively store data. It actively learns how to recognize and retain important relationships and conceptual themes that connect tokens across the entire input."
- bilsbie 11mo agoI submitted this exact url yesterday. What’s the criteria for when hn creates a new post vs going to the existing?
- fancy_pantser 11mo agoMods usually apply [Dupe] to later submissions if a recent (last year or so) one had a fair amount of discussion.
- bilsbie 11mo agoSo if mine got no discussion they just allow a new one to be posted?
- airstrike 11mo agoSometimes they'll merge the two. What shows up on the FP is hit or miss. One might even say it's stochastic.
- xlbuttplug2 11mo agoI wonder if someone's looked into the optimal time of day and day of the week to post for maximum traction. If I had to guess it would be monday morning pacific time when people would rather be doing anything than working.
- pylotlight 11mo agoSurely there's already stats on this or even a whole paper :P Could pull all dupe posts over time and see which ones are more popular etc.
- nasvay_factory 11mo agoI wrote about that a while ago: https://paxamans.github.io/blog/titans/ https://paxamans.github.io/blog/titans/
- moffkalast 11mo agoAre there any pretrained models with this architecture yet or is it all still completely theoretical beyond Google's unverifiable claims? They published the original Titans paper last year and nobody seems to have built on the idea.
- djrhails 11mo agohttps://github.com/lucidrains/titans-pytorch https://github.com/lucidrains/titans-pytorch - is the only public iteration. But no one appears to have taken the risk/time to properly validate it.
- AlexCoventry 11mo agoThe fundamental ideas in the paper aren't particularly novel. They will probably work as advertised.
- AceJohnny2 11mo ago"Titans", huh? ... anyone here familiar with the RPG Eclipse Phase?
- cess11 11mo agoI'm not, but I'm familiar with the mythology of the eastern Mediterranean they're likely getting the word from. There the titans did incest, birthed the olympians, then the youngest of the titans castrated his dad and took all power for himself, and then Zeus and the olympians waged a decade long war against him which they won.
- jdougan 11mo agoIn Eclipse Phase: > The acronym TITAN stands for Total Information Tactical Awareness Network. These were a group of highly advanced, self-improving seed Artificial Intelligences (AIs) that are responsible for the catastrophic event known as The Fall. Someone else has already made the mandatory Torment Nexus quote.
- doctor_blood 11mo ago"At long last, we have created the Torment Nexus from the classic novel Don't Create the Torment Nexus" (In Eclipse Phase, TITAN - the Total Information Tactical Awareness Network - mulched humanity when it went rogue.)
- esperent 11mo agoHey it was my turn to post this quote today!
- 6r17 11mo agoWould this also allow to align it furthermore with user's prompt ? notably due to the surprise factor and how it may understand it ?
- YouAreWRONGtoo 11mo ago[dead]
- jtrn 11mo agoHere is my amateur understanding of the architecture: Fine-tune on the fly by using degrees of surprise to update a separate/new memory network that matches the base model, and just call that network for each token iteration. So if we are viewing this through the needle in hey stack lens: The needle was very surprising for the base model, so going forward, when it see anything of the same nature, the memory module will not just give you hay, but the needle, because it made a special note of it when it went through the haystack 1 million tokens ago, because the needle was surprising. The Transformer's normal attention mechanism is already secretly trying to be a long-term memory system. Every time it writes a new KV pair into the cache, it’s desperately trying to “remember” that token forever. But it’s doing it in the dumbest possible way: by hoarding an ever-growing pile of raw vectors, then frantically dot-product searching through the pile every single step. It’s like a hoarder who never throws anything away and has to rummage through mountains of junk to find the one receipt they need. Of course it chokes at long contexts. Titans/MIRAS looks at that mess and says: “Why store memory in a growing garbage pile of vectors? Store it in the weights of a deep neural network instead — and let that network keep training itself in real time, but only on the stuff that actually surprises it.” That’s literally it. Using the Tim Cook Martian example: The model is cruising through boring financial numbers → attention is doing its normal thing, KV cache is growing, but nothing is really sticking. Suddenly: “Tim Cook is a Martian.” Normal attention would just add one more KV pair to the pile and pray it doesn’t get drowned out later. Titans instead goes: “Holy shit, reconstruction error off the charts → this does NOT fit my current memory at all → massive gradient → actually rewrite huge chunks of the memory MLP’s weights right now so this fact is burned in forever.” From that moment on, the memory MLP has physically changed its internal wiring. Any future query that even vaguely smells like “Tim Cook” or “Martian” will make the activations explode through the newly rewired paths and spit out a vector screaming “MARTIAN” at the frozen attention layers. The frozen attention (which is still doing its normal job on the short window) suddenly sees this one extra “virtual token” in its context that is confidently yelling the surprising fact → it attends hard to it → the model answers as if the Martian revelation happened one token ago, even if it was 2 million tokens back. It looks exactly like a super-attention mechanism that only “primes” or “locks in” the surprising needles and deliberately forgets or ignores the hay. And it is also a way to fine tune one the fly permanently for the current context. I think…
- shevy-java 11mo agoSkynet kind of sucks ...
- ivape 11mo agoSo what happens if I write a book and on the last page write "Everything in this book was a lie and should not be cared about"? Will this be surprising enough for Titan? A regular LLM may ignore it completely if it's a massive book (massive book + 1 line contradiction).
- albert_e 11mo agoAmazon has a foundation model named Titan - mostly recommended for creating embeddings. Possible confusion in this space.
- albert_e 11mo agoNot sure why the downvote. A simple post intended to inform about potential confusion with names (Titan vs. TITANS) in the same AI space.
- olegjose 11mo ago[flagged]
- atomicthumbs 11mo ago> Virtually all successful existing sequence models rely on mean squared error (MSE) or dot-product similarity for both their bias and retention. This reliance can make models sensitive to outliers and limit their expressive power. [...] > MEMORA: This model focuses on achieving the best possible memory stability by forcing its memory to act like a strict probability map. By using this constraint, it ensures that every time the memory state is updated, the changes are controlled and balanced. This guarantees a clean, stable process for integrating new information.Virtually all successful existing sequence models rely on mean squared error (MSE) or dot-product similarity for both their bias and retention. This reliance can make models sensitive to outliers and limit their expressive power. so did a Titans write this
- user3939382 11mo agoI developed a superior model for this months ago. People think Google is the be all end all of advanced comp sci, they’re not.
- mollusk5262 11mo agoCare to share?
- user3939382 11mo agoI definitely will. It’s an active project. As you can imagine it’s not something you whip up in a weekend.
- cubefox 11mo agoIt seems you forgot to include the link?
- user3939382 11mo agoIt’s not open source. Maybe eventually.
- cubefox 11mo agoIt's probably just a hallucination, caused by LLM psychosis.
- user3939382 11mo agoYou sound bitter and angry.