wenhan_zhou

6 ms·

1.
by wenhan_zhou 4mo ago
Yep. Or even better, compact after a random number of turns. The model must then learn to preserve useful context at arbitrary context lengths.
2.
A Bitter Lesson for Memory(personal-website-3bed.onrender.com)
4 points by wenhan_zhou 4mo ago|3 comments
3.
by wenhan_zhou 4mo ago
If understanding emerges from pre-training, then perhaps memory is what emerges from post-training.
4.
by wenhan_zhou 6mo ago
Currently working on a benchmark!
5.
by wenhan_zhou 6mo ago
Ah, so you are effectively offloading the file exploration mechanism to the INDEX.md in the sub-directories rather than writing a complex prompt?
6.
by wenhan_zhou 6mo ago
How does the agent intelligently synthesize information across different files?
7.
by wenhan_zhou 6mo ago
In theory, yes. Although the privacy setting says otherwise. But in the end, it doesn't really matter; it is public on GitHub, so anyone can use it.
8.
by wenhan_zhou 6mo ago
I just read LLM Wiki in more detail. I have heard about it second-hand before this project. The "no-code" idea was inspired by Karpathy. As I have understood it, in LLM Wiki, the human is very much in the loop in what gets written
9.
by wenhan_zhou 6mo ago
Although I have been working on memory before, ReadMe is very fresh. The moment I saw it running, I published it. So, no continuous running nor LLM ablation studies. Treat it as an MVP, would love to hear how your agent performs!
10.
by wenhan_zhou 6mo ago
I think what's missing is a benchmark that measures how well the memories contribute to future interactions.
11.
by wenhan_zhou 6mo ago
I don't remember such details, but as you suggest, it is a healthy kind of compression. I address it through merging the lower-level memories into more abstracted ones through a temporal hierarchical filesystem. So, days -> months -
12.
by wenhan_zhou 6mo ago
I see your point. A removal mechanism is not (yet) implemented. But in principle, we could adjust the instructions in Update.md so that it does a minor "refactor" of the filesystem each day, then newer abstractions can form, while
13.
by wenhan_zhou 6mo ago
Minimalism is my design philosophy :-) Good question. Since it is just an LLM reading files, it depends entirely on how fast it can call tools, so it depends on the token/s of the model. Haven't done a formal benchmark, but from t
14.
by wenhan_zhou 6mo ago
Yep. Markdown is the future :-)
15.
by wenhan_zhou 6mo ago
Fair concern. ReadMe does support loading memories mid-reasoning! It is simply an agent reading files. Although GPT-5.4 currently likes to explore a lot upfront, and only then responds. But that is more of a model behaviour (adjustable th
16.
by wenhan_zhou 6mo ago
Context bloat is real, but the architecture has the potential to solve it. You need clever naming for the filesystem and exploration policy in AGENTS.md. (not trivial!) The benchmark is definitely the core bottleneck. I don't know any
17.
by wenhan_zhou 6mo ago
The editability is surely an underrated advantage, both for the program itself and the memories it generated. I think in terms of noise, it is less problematic here because not everything is being retrieved. The agent can selectively expl
18.
Show HN: Continual Learning with .md(github.com)
34 points by wenhan_zhou 6mo ago|34 comments