3 ms·

What kind of things you only get with a larger model?

versteegen 1mo ago
IME using 5.6 Luna and DS V4 Flash, I notice that although they are excellent at programming, even Opus-like in the way they try to debug, the thing they are worst at is inferring user intent and making good decisions with little information. They are absolutely terrible at that, will misinterpret small wording ambiguities. I suspect that's an ability you can't add with RL training, that it requires the depth of understanding from vast pre-training.
- johnnyApplePRNG 1mo ago
  So add a pre-ingestion agent to your Pi Coding Agent subagent repertoire, problem solved.
  - NamlchakKhandro 1mo ago
    Woah woah woah buddy... suggesting that people use anything other than ClaudeCode or Codex is simply not allowed around these parts.
    - solarengineer 1mo ago
      Your negative impression is surprising. Could you point to any threads to support your message?
  - solarengineer 1mo ago
    Please, could you point me to any documentation about this pre-ingestion? I’ve looked here: https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/usage.md https://github.com/earendil-works/pi/blob/main/packages/codi...
    - johnnyApplePRNG 1mo ago
      The point is to simply add a subagent that interprets your task and hands that off to the coder set to the appropriate level of thinking, essentially.
onlyrealcuzzo 1mo ago
Similar to the way they asked Sol to solve Erdos problems, that's what I want my model to do for programming. I don't want to try to take my best educated guess at what the best design is BEFORE implementation - especially if you're designing a feature for a codebase you're not an expert in, you don't know like the back of your hand (i.e. one that is mostly or entirely LLM generated). What sounds good on paper - often times becomes unideal in practice when you get to the reality of implementation. It may not be worth re-architecting your entire system to get to a "pure" design that would be the best - all things considered. Instead, I'd like the model to independently design many plausible and coherent good solutions, then implement each of them, then intelligently pick the few winners (after its fixed any bugs that could be causing promising solutions to look artificially bad) - unless there's an obvious one - and then give me the data I need to make an informed decision on which one to go with, all before I even look at the design or implementation. You're not getting this from a one shot prompt from a 30B model today. You can't even really get it from Sol or Fable - IME. But you can get somewhat close.
- alex7o 1mo ago
  This is a harness problem not a model problem, try prime agent it can do that and it will do it well even :P but you need to prompt it in according to its tools and processes.
- ferrouswheel 1mo ago
  You're going to be waiting for a while. Even Fable is bad at this, I would constantly have to fix it going down architectural dead ends or just making obvious mistakes. Which sucks for people that want LLMs to do everything like a genie, but does mean senior engineers have a few more years before they become redundant.
  - johnnyApplePRNG 1mo ago
    I honestly think we've got about a year tops.
    - 1mo ago
redox99 1mo ago
Asking it factual information[1]. You just can't compress the entire human knowledge into a 30GB file. [1] Without searching the internet. And even if you allow it, you'll get much worse results because search means browsing and parsing the top results, and search results are horrible, whereas internal knowledge from training encompasses the entire internet plus all books including very niche stuff.
- Wowfunhappy 1mo ago
  > You just can't compress the entire human knowledge into a 30GB file. ...I'm just asking questions here... how sure are we of this? If you'd asked me six years ago whether we could compress all of human knowledge into a 1 TB file, I would have said no, and yet here we are. If you can do 1 TB, why not 30 GB? It's, like, within one order of magnitude.
  - redox99 1mo ago
    (mumbles) Shannon entropy... Kolmogorov complexity... something, something... On a more serious note, it depends on your cutoff for "entire human knowledge". It's easy to prove for a generous interpretations of "entire human knowledge" that it can't be done, but hard for something like "all useful human knowledge".
    - gen220 1mo ago
      IDK, 30GB is a lot of data when we're talking about text! Moby Dick, uncompressed, is ~1MB. Compressed, it's around 500KB. I feel fairly certain that one could fit all of the textual knowledge required to cultivate a world-class <insert name of preferred professional knowledge worker> in <60,000 Moby Dicks. (Arguably in <5,000 Moby Dicks with intense effort/pruning).
      - redox99 1mo ago
        I think a specialized model could squeeze all you need to know about a certain profession in 30GB. But not all professions at once, which is what these models try to do. Or maybe not, maybe there's a world model needed for human level at any profession that is very hard to quantify and requires more than 30GB by itself.
aka-rider 1mo ago
Smaller models are overconfident and have a hard time to self-correct. If it’s stuck, usually that’s it. Bigger models “understand” better, both the prompt and the contents. If you will try to read a paper together with a smaller model, the difference is immediately obvious. Bigger models will “forget” and drift much less.

IME using 5.6 Luna and DS V4 Flash, I notice that although they are excellent at programming, even Opus-like in the way they try to debug, the thing they are worst at is inferring user intent and making good decisions with little information. They are absolutely terrible at that, will misinterpret small wording ambiguities. I suspect that's an ability you can't add with RL training, that it requires the depth of understanding from vast pre-training.

johnnyApplePRNG 1mo ago
So add a pre-ingestion agent to your Pi Coding Agent subagent repertoire, problem solved.
- NamlchakKhandro 1mo ago
  Woah woah woah buddy... suggesting that people use anything other than ClaudeCode or Codex is simply not allowed around these parts.
  - solarengineer 1mo ago
    Your negative impression is surprising. Could you point to any threads to support your message?
- solarengineer 1mo ago
  Please, could you point me to any documentation about this pre-ingestion? I’ve looked here: https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/usage.md https://github.com/earendil-works/pi/blob/main/packages/codi...
  - johnnyApplePRNG 1mo ago
    The point is to simply add a subagent that interprets your task and hands that off to the coder set to the appropriate level of thinking, essentially.

onlyrealcuzzo 1mo ago

Similar to the way they asked Sol to solve Erdos problems, that's what I want my model to do for programming. I don't want to try to take my best educated guess at what the best design is BEFORE implementation - especially if you're designing a feature for a codebase you're not an expert in, you don't know like the back of your hand (i.e. one that is mostly or entirely LLM generated). What sounds good on paper - often times becomes unideal in practice when you get to the reality of implementation. It may not be worth re-architecting your entire system to get to a "pure" design that would be the best - all things considered. Instead, I'd like the model to independently design many plausible and coherent good solutions, then implement each of them, then intelligently pick the few winners (after its fixed any bugs that could be causing promising solutions to look artificially bad) - unless there's an obvious one - and then give me the data I need to make an informed decision on which one to go with, all before I even look at the design or implementation. You're not getting this from a one shot prompt from a 30B model today. You can't even really get it from Sol or Fable - IME. But you can get somewhat close.

alex7o 1mo ago
This is a harness problem not a model problem, try prime agent it can do that and it will do it well even :P but you need to prompt it in according to its tools and processes.
ferrouswheel 1mo ago
You're going to be waiting for a while. Even Fable is bad at this, I would constantly have to fix it going down architectural dead ends or just making obvious mistakes. Which sucks for people that want LLMs to do everything like a genie, but does mean senior engineers have a few more years before they become redundant.
- johnnyApplePRNG 1mo ago
  I honestly think we've got about a year tops.
  - 1mo ago

redox99 1mo ago

Asking it factual information[1]. You just can't compress the entire human knowledge into a 30GB file. [1] Without searching the internet. And even if you allow it, you'll get much worse results because search means browsing and parsing the top results, and search results are horrible, whereas internal knowledge from training encompasses the entire internet plus all books including very niche stuff.

Wowfunhappy 1mo ago
> You just can't compress the entire human knowledge into a 30GB file. ...I'm just asking questions here... how sure are we of this? If you'd asked me six years ago whether we could compress all of human knowledge into a 1 TB file, I would have said no, and yet here we are. If you can do 1 TB, why not 30 GB? It's, like, within one order of magnitude.
- redox99 1mo ago
  (mumbles) Shannon entropy... Kolmogorov complexity... something, something... On a more serious note, it depends on your cutoff for "entire human knowledge". It's easy to prove for a generous interpretations of "entire human knowledge" that it can't be done, but hard for something like "all useful human knowledge".
  - gen220 1mo ago
    IDK, 30GB is a lot of data when we're talking about text! Moby Dick, uncompressed, is ~1MB. Compressed, it's around 500KB. I feel fairly certain that one could fit all of the textual knowledge required to cultivate a world-class <insert name of preferred professional knowledge worker> in <60,000 Moby Dicks. (Arguably in <5,000 Moby Dicks with intense effort/pruning).
    - redox99 1mo ago
      I think a specialized model could squeeze all you need to know about a certain profession in 30GB. But not all professions at once, which is what these models try to do. Or maybe not, maybe there's a world model needed for human level at any profession that is very hard to quantify and requires more than 30GB by itself.

aka-rider 1mo ago

Smaller models are overconfident and have a hard time to self-correct. If it’s stuck, usually that’s it. Bigger models “understand” better, both the prompt and the contents. If you will try to read a paper together with a smaller model, the difference is immediately obvious. Bigger models will “forget” and drift much less.