4 ms·

I reached a similar conclusion: LLMs should only really sit at the terminals of request fulfilment. 1. User request understanding: natural language -> a more r

I reached a similar conclusion: LLMs should only really sit at the terminals of request fulfilment. 1. User request understanding: natural language -> a more rigorous representation, in my case Datalog. 2. Result interpretation: facts and derived facts -> natural language. Between those terminals, the work should be mechanical reasoning over some ontology or formal knowledge structure. That connects to another principle I've been thinking about, which I call Weathering: useful reasoning should change the shape of the system. If an LLM has already had to infer a relation, mapping, rule, or abstraction, repeated use should wear that inference into the system so that the next similar request doesn't require discovering it again from scratch. With continued use, a weathering-capable system should therefore require less and less probabilistic intelligence for recurring work. Put another way, there should be a declining marginal cost of cognition since the products of intelligence harden into structure that can subsequently be reused and evaluated mechanically.

i_eat_rocks 1mo ago
[dead]
tomrod 1mo ago
Bayesian posteriors in the wild. Love it!
alansaber 1mo ago
Theoretically but practically any LLM generated infra/classification set is going to drift due to inaccuracy and harm IR/whatever logical process you're using. I am a big fan of using a loose taxonomy but it's not been revolutionary.
deleted 1mo ago
[deleted]
PcChip 1mo ago
When i hear “weathering” i think of something slowly eroding away
- akk0 1mo ago
  I feel like the thing eroding away in this case is a blocker, a friction. Think of a river cutting a path through the mountains.
  - sim04ful 1mo ago
    Yup, picked that wording to evoke this: https://en.wikipedia.org/wiki/Natural_arch https://en.wikipedia.org/wiki/Natural_arch
- sa46 1mo ago
  Weather is fun choice since it’s a contronym. It has opposing definitions: to endure and to erode. “He weathered the storm” vs “the stone weathered into an arch”.
  - cronin101 1mo ago
    Really (in your example) the timeframe is the only difference. You can endure as long as you want to, but we're all going to erode eventually.
abhgh 1mo ago
What you call "Weathering" has been a constant gripe of mine. We have LLM-driven softwares toward that almost seem to start from scratch every time a request comes in - there are mechanisms to learn or generalize, like writing out a memory, but they are not reliable or reliable in general. There is no convenient lever to be able to say "yes this is in the memory but the request seems like it needs a fresh scan of data, so ignore your memory", or the opposite "you can infer this from stuff in the memory - don't re-reason!". There is some work like Dynamic Cheatsheets [1] and Agentic Context Engineering [2] that have studied this aspect, but we are far from a generally reliable solution. And till we have that, I think the system variations for systems trying to solve this problem are going to be (a) LLM-leaning: create unstructured memory files, with human in the loop as a filter to reject inaccurate responses (b) LLM-as-a-layer: what you describe and the article kind of is doing. [1] https://aclanthology.org/2026.eacl-long.333/ https://aclanthology.org/2026.eacl-long.333/ [2] https://openreview.net/pdf?id=eC4ygDs02R https://openreview.net/pdf?id=eC4ygDs02R
- reacharavindh 1mo ago
  I’ve been thinking about this.. the same idea behind Dynamic cheat sheets paper and the ACE paper. Thanks for those links! Nice to see those ideas academically studied. My problem is the context of today’s models (that I’ve been using - Claude Opus 5 and GPT-Sol) are a black box to a user like me. I cannot tell what they already hold in their context over the duration of a coding session. I cannot tell influencers a little by using/not using MCP tools, AGENTS.md like MD files, but what they _actually_ use at inference time is still opaque. For example, if I insist that GPT-Sol use a size bounded dynamic cheat sheet for my project, and even have those reflector&curator agents refine them concurrently, what is to say GPT-Sol doesn’t have an intrinsic cheat sheet like that already, and it is just reading/curating/managing this one in addition and only getting confused needlessly along the way? Similarly, all those codegraph like tools and semantic search addons.. what is to say the model genuinely uses them instead of relying on its own inference techniques and just “perform” for the user audience as if they are using these fancy tools? This is not to say we shouldn’t try more ways of managing context and just give up to our AI provider overlords. Just thinking through the logic of what we are working with.
sirtaj 1mo ago
I guess efforts like Google's OKF, which teach the llm to manage the ontology directly, are the opposite of what you want. The weathering need is real. An llm equivalent of dbms analysing usage pattern to tune indexes.
furyofantares 1mo ago
The way I put it when working on LLM-enabled things is: Get the hell out of the LLM as soon as possible. And I guess for 2, stay out as long as possible. I guess this is somewhat different; it admits that in some cases you do need the LLM doing more work somewhere in the middle. But it also means sometimes you never enter the LLM to begin with, or never re-enter it for output.

i_eat_rocks 1mo ago

[dead]

tomrod 1mo ago

Bayesian posteriors in the wild. Love it!

alansaber 1mo ago

Theoretically but practically any LLM generated infra/classification set is going to drift due to inaccuracy and harm IR/whatever logical process you're using. I am a big fan of using a loose taxonomy but it's not been revolutionary.

deleted 1mo ago

[deleted]

PcChip 1mo ago

When i hear “weathering” i think of something slowly eroding away

akk0 1mo ago
I feel like the thing eroding away in this case is a blocker, a friction. Think of a river cutting a path through the mountains.
- sim04ful 1mo ago
  Yup, picked that wording to evoke this: https://en.wikipedia.org/wiki/Natural_arch https://en.wikipedia.org/wiki/Natural_arch
sa46 1mo ago
Weather is fun choice since it’s a contronym. It has opposing definitions: to endure and to erode. “He weathered the storm” vs “the stone weathered into an arch”.
- cronin101 1mo ago
  Really (in your example) the timeframe is the only difference. You can endure as long as you want to, but we're all going to erode eventually.

abhgh 1mo ago

What you call "Weathering" has been a constant gripe of mine. We have LLM-driven softwares toward that almost seem to start from scratch every time a request comes in - there are mechanisms to learn or generalize, like writing out a memory, but they are not reliable or reliable in general. There is no convenient lever to be able to say "yes this is in the memory but the request seems like it needs a fresh scan of data, so ignore your memory", or the opposite "you can infer this from stuff in the memory - don't re-reason!". There is some work like Dynamic Cheatsheets [1] and Agentic Context Engineering [2] that have studied this aspect, but we are far from a generally reliable solution. And till we have that, I think the system variations for systems trying to solve this problem are going to be (a) LLM-leaning: create unstructured memory files, with human in the loop as a filter to reject inaccurate responses (b) LLM-as-a-layer: what you describe and the article kind of is doing. [1] https://aclanthology.org/2026.eacl-long.333/ https://aclanthology.org/2026.eacl-long.333/ [2] https://openreview.net/pdf?id=eC4ygDs02R https://openreview.net/pdf?id=eC4ygDs02R

reacharavindh 1mo ago
I’ve been thinking about this.. the same idea behind Dynamic cheat sheets paper and the ACE paper. Thanks for those links! Nice to see those ideas academically studied. My problem is the context of today’s models (that I’ve been using - Claude Opus 5 and GPT-Sol) are a black box to a user like me. I cannot tell what they already hold in their context over the duration of a coding session. I cannot tell influencers a little by using/not using MCP tools, AGENTS.md like MD files, but what they _actually_ use at inference time is still opaque. For example, if I insist that GPT-Sol use a size bounded dynamic cheat sheet for my project, and even have those reflector&curator agents refine them concurrently, what is to say GPT-Sol doesn’t have an intrinsic cheat sheet like that already, and it is just reading/curating/managing this one in addition and only getting confused needlessly along the way? Similarly, all those codegraph like tools and semantic search addons.. what is to say the model genuinely uses them instead of relying on its own inference techniques and just “perform” for the user audience as if they are using these fancy tools? This is not to say we shouldn’t try more ways of managing context and just give up to our AI provider overlords. Just thinking through the logic of what we are working with.

sirtaj 1mo ago

I guess efforts like Google's OKF, which teach the llm to manage the ontology directly, are the opposite of what you want. The weathering need is real. An llm equivalent of dbms analysing usage pattern to tune indexes.

furyofantares 1mo ago

The way I put it when working on LLM-enabled things is: Get the hell out of the LLM as soon as possible. And I guess for 2, stay out as long as possible. I guess this is somewhat different; it admits that in some cases you do need the LLM doing more work somewhere in the middle. But it also means sometimes you never enter the LLM to begin with, or never re-enter it for output.