3 ms·

It makes intuitive sense. How else could you write a 500-line script top-to-bottom with no backspace key and no arrow keys and get all the imports etc. right up

It makes intuitive sense. How else could you write a 500-line script top-to-bottom with no backspace key and no arrow keys and get all the imports etc. right upfront?

wren6991 3mo ago
...by inferring both the imports and the script body from the same context? I think you're suggesting there's some kind of information flow from the anticipated body of the script back up to the imports, but I don't see why that would be necessary. Infer imports from context, infer body from context + imports. All strictly causal.
- jstanley 3mo ago
  Sure, try it. It's harder than you think. It's not just imports, it's the entire program. > you're suggesting there's some kind of information flow from the anticipated body of the script back up to the imports Yes, I am suggesting this. I don't think it is possible to write programs without either anticipating what you're going to write down below before you get there, or else being able to go back and edit what you already wrote. Of course agent harnesses allow the latter, but raw models without a harness can still do an exceptionally, superhumanly, good job of straight-line programming with no editing. > Infer imports from context, infer body from context + imports. All strictly causal. Of course it's causal, that's kind of a reductive way to look at it. Just infer the entire program from context and then type what you inferred.
  - twister2920 3mo ago
    > It's harder than you think. well, yeah, I'm not a large language model
    - coldtea 3mo ago
      In what ways are you different?
      - chrisjj 3mo ago
        Has a life?
        coldtea 3mo ago
        In what ways regarding the processing involved. How much can it be reduced and simulated and be essentially the same principle?
      - thesz 3mo ago
        Can we use backspace key?
      - twister2920 3mo ago
        see parent comment
- CooCooCaCha 3mo ago
  That suggests a straightforward mapping between context -> program. Maybe for trivial programs but I don’t see how you can argue that for sufficiently complex programs. Reasoning models perform better than non-reasoning models because they’re able to refine their code in multiple steps. That allows any part of the program to influence any other part of the program, not just from start -> end. Human thinking serves a similar purpose. Basically intelligence needs to be able to backtrack if you want better performance.
  - wren6991 3mo ago
    Right, reasoning models are almost more notable for what they don't do, which is: try to generate the first token of the response from a single forward pass through the weights. Non-reasoning models were still surprisingly good at generating working programs. I think it makes sense when you consider that it's still iterative; it doesn't have to generate the entire import block in a single pass, for example, more like decide at the end of each line whether there are more libraries it should import or whether to move on to the body of the program. It's less surprising that a single forward pass is able to make that type of decision IMO. Taking Python as an example, you see LLMs use a lot of scoped imports and I think this is partly because it makes the output more resilient against failure to think forward at the start. Even a reasoning model will write out (the first iteration of) an entire file in one tool call, so once it starts generating it still has to keep going until it finishes, no pausing for breath. Also it's interesting that non-reasoning models can still do quite well in a harness that lets them test and fix up their code. Like you said, it gives them the ability to iterate.
    - chrisjj 3mo ago
      > you see LLMs use a lot of scoped imports and I think this is partly because it makes the output more resilient against failure to think forward at the start. What makes you think an LLM has any such motivation? Or any motivation at all - beyond generating the most likely next token?
      - wren6991 3mo ago

It makes intuitive sense. How else could you write a 500-line script top-to-bottom with no backspace key and no arrow keys and get all the imports etc. right upfront?

wren6991 3mo ago

...by inferring both the imports and the script body from the same context? I think you're suggesting there's some kind of information flow from the anticipated body of the script back up to the imports, but I don't see why that would be necessary. Infer imports from context, infer body from context + imports. All strictly causal.

jstanley 3mo ago
Sure, try it. It's harder than you think. It's not just imports, it's the entire program. > you're suggesting there's some kind of information flow from the anticipated body of the script back up to the imports Yes, I am suggesting this. I don't think it is possible to write programs without either anticipating what you're going to write down below before you get there, or else being able to go back and edit what you already wrote. Of course agent harnesses allow the latter, but raw models without a harness can still do an exceptionally, superhumanly, good job of straight-line programming with no editing. > Infer imports from context, infer body from context + imports. All strictly causal. Of course it's causal, that's kind of a reductive way to look at it. Just infer the entire program from context and then type what you inferred.
- twister2920 3mo ago
  > It's harder than you think. well, yeah, I'm not a large language model
  - coldtea 3mo ago
    In what ways are you different?
    - chrisjj 3mo ago
      Has a life?
      - coldtea 3mo ago
        In what ways regarding the processing involved. How much can it be reduced and simulated and be essentially the same principle?
    - thesz 3mo ago
      Can we use backspace key?
    - twister2920 3mo ago
      see parent comment
CooCooCaCha 3mo ago
That suggests a straightforward mapping between context -> program. Maybe for trivial programs but I don’t see how you can argue that for sufficiently complex programs. Reasoning models perform better than non-reasoning models because they’re able to refine their code in multiple steps. That allows any part of the program to influence any other part of the program, not just from start -> end. Human thinking serves a similar purpose. Basically intelligence needs to be able to backtrack if you want better performance.
- wren6991 3mo ago
  Right, reasoning models are almost more notable for what they don't do, which is: try to generate the first token of the response from a single forward pass through the weights. Non-reasoning models were still surprisingly good at generating working programs. I think it makes sense when you consider that it's still iterative; it doesn't have to generate the entire import block in a single pass, for example, more like decide at the end of each line whether there are more libraries it should import or whether to move on to the body of the program. It's less surprising that a single forward pass is able to make that type of decision IMO. Taking Python as an example, you see LLMs use a lot of scoped imports and I think this is partly because it makes the output more resilient against failure to think forward at the start. Even a reasoning model will write out (the first iteration of) an entire file in one tool call, so once it starts generating it still has to keep going until it finishes, no pausing for breath. Also it's interesting that non-reasoning models can still do quite well in a harness that lets them test and fix up their code. Like you said, it gives them the ability to iterate.
  - chrisjj 3mo ago
    > you see LLMs use a lot of scoped imports and I think this is partly because it makes the output more resilient against failure to think forward at the start. What makes you think an LLM has any such motivation? Or any motivation at all - beyond generating the most likely next token?
    - wren6991 3mo ago