5 ms·

Isn’t this just kicking the can down the road? > but the LLM is rediscovering knowledge from scratch on every question Unless the wiki stays fully in context

by Imanari 6mo ago

Isn’t this just kicking the can down the road? > but the LLM is rediscovering knowledge from scratch on every question Unless the wiki stays fully in context now the LLM hast to re-read the wiki instead of re-reading the source files. Also this will introduce and accumulate subtle errors as we start to regurgitate 2nd-order information. I totally get the idea but I think next gen models with 10M context and/or 1000tps will make this obsolete.

SOLAR_FIELDS 6mo ago
I use a home baked system based on obsidian that is essentially just “obsidian but with structured format on top with schemas” and I deploy this in multiple places with ranges of end users. It is more valuable than you think. The intermediary layer is great for capturing intent of design and determining when implementation diverges from that. There will always be a divergence from the intent of a system and how it actually behaves, and the code itself doesn’t capture that. The intermediate layer is lossy, it’s messy, it goes out of date, but it’s highly effective. It’s not what this person is describing though. A self referential layer like this that’s entirely autonomous does feel completely valueless - because what is it actually solving? Making itself more efficient? The frontier model providers will be here in 3 weeks doing it better than you on that front. The real value is having a system that supports a human coming in and saying “this is how the system should actually behave”, and having the system be reasonably responsive to that. I feel like a lot of the exercises like op are interesting but ultimately futile. You will not have the money these frontier providers do, and you do not have remotely the amount of information that they do on how to squeeze the most efficiency in how they work. Best bet is to just stick with the vanilla shit until the firehose of innovation slows down to something manageable, because otherwise the abstraction you build is gonna be completely irrelevant in two months
- deleted 6mo ago
  [deleted]
- ctxc 6mo ago
  Interesting, I'd love to know more. Are parts of it public?
  - SOLAR_FIELDS 6mo ago
    Indeed, I have it open source, but want to preserve my anonymity here. The main gist of it is Quartz as a static site frontend bundle, backed by Decap as an editor, so that non technical users can edit documents. The validation is twofold - frontmatter is validated by a typical yaml validator library, and then I created markdown body validation using some popular markdown AST libraries, so there are two sets of schemas - one for the frontmatter, one for the body, and documents must conform via ci. I ship it with a basic cli that essentially does validation and has a few other utilities. Not really that much magic, maybe 500 lines of code or so in the CLI and another few hundred lines doing validation and the other utilties. It's all in typescript, so I use the same validation in Decap when people do edits.
nidnogg 6mo ago
It is how I feel when I do it. And it certainly shows over time.
0123456789ABCDE 6mo ago
this solves for now, and this solves for the future. now you get to condense the findings that interest from a handful of papers in the future it solves for condensing your interests in a whole field to a handful of papers or less
khalic 6mo ago
The goal isn’t to keep the context every time, it’s to make the memory queryable. Like a data lake but for your ideas and decisions
lelanthran 6mo ago
> I totally get the idea but I think next gen models with 10M context and/or 1000tps will make this obsolete. We've already got 1m context, 800k context, and they still start "forgetting" things around the 200k - 300k mark. What use is 10M context if degradation starts at 200k - 300k?
dennisy 6mo ago
The “next gen of models” argument is a valid one and one I think of often, but if you truly buy it, it would stop do creating anything - since the next gen of models could make it obsolete.

by Imanari 6mo ago

SOLAR_FIELDS 6mo ago

I use a home baked system based on obsidian that is essentially just “obsidian but with structured format on top with schemas” and I deploy this in multiple places with ranges of end users. It is more valuable than you think. The intermediary layer is great for capturing intent of design and determining when implementation diverges from that. There will always be a divergence from the intent of a system and how it actually behaves, and the code itself doesn’t capture that. The intermediate layer is lossy, it’s messy, it goes out of date, but it’s highly effective. It’s not what this person is describing though. A self referential layer like this that’s entirely autonomous does feel completely valueless - because what is it actually solving? Making itself more efficient? The frontier model providers will be here in 3 weeks doing it better than you on that front. The real value is having a system that supports a human coming in and saying “this is how the system should actually behave”, and having the system be reasonably responsive to that. I feel like a lot of the exercises like op are interesting but ultimately futile. You will not have the money these frontier providers do, and you do not have remotely the amount of information that they do on how to squeeze the most efficiency in how they work. Best bet is to just stick with the vanilla shit until the firehose of innovation slows down to something manageable, because otherwise the abstraction you build is gonna be completely irrelevant in two months

deleted 6mo ago
[deleted]
ctxc 6mo ago
Interesting, I'd love to know more. Are parts of it public?
- SOLAR_FIELDS 6mo ago
  Indeed, I have it open source, but want to preserve my anonymity here. The main gist of it is Quartz as a static site frontend bundle, backed by Decap as an editor, so that non technical users can edit documents. The validation is twofold - frontmatter is validated by a typical yaml validator library, and then I created markdown body validation using some popular markdown AST libraries, so there are two sets of schemas - one for the frontmatter, one for the body, and documents must conform via ci. I ship it with a basic cli that essentially does validation and has a few other utilities. Not really that much magic, maybe 500 lines of code or so in the CLI and another few hundred lines doing validation and the other utilties. It's all in typescript, so I use the same validation in Decap when people do edits.

nidnogg 6mo ago

It is how I feel when I do it. And it certainly shows over time.

0123456789ABCDE 6mo ago

this solves for now, and this solves for the future. now you get to condense the findings that interest from a handful of papers in the future it solves for condensing your interests in a whole field to a handful of papers or less

khalic 6mo ago

The goal isn’t to keep the context every time, it’s to make the memory queryable. Like a data lake but for your ideas and decisions

lelanthran 6mo ago

> I totally get the idea but I think next gen models with 10M context and/or 1000tps will make this obsolete. We've already got 1m context, 800k context, and they still start "forgetting" things around the 200k - 300k mark. What use is 10M context if degradation starts at 200k - 300k?

dennisy 6mo ago

The “next gen of models” argument is a valid one and one I think of often, but if you truly buy it, it would stop do creating anything - since the next gen of models could make it obsolete.