3 ms·

I know it sounds a bit counter-intuitive but try a fresh coding session without skills, agents.md, system prompt and additional tools I think you will be posi

by tosh 3mo ago

I know it sounds a bit counter-intuitive but try a fresh coding session without skills, agents.md, system prompt and additional tools I think you will be positively surprised how good current models like GPT 5.6 Sol are when they are not oversteered and context spammed Here is a task (python templating) with 9 runs with OpenCode, Pi and smol https://smolenv.com/t/nested-template-includes-60636/ https://smolenv.com/t/nested-template-includes-60636/ you can read each run step by step and see what the agents are doing and how the system prompt and available tools are steering their behaviour to take longer and higher cost (disclaimer: I'm working on smol)

hansonkd 3mo ago
Also just routine upkeep. Over time if you keep adding new rules these things can get bloated. I deleted 90% of my spec and things improved dramatically.
tosh 3mo ago
I'm not saying system prompts, agents.md, tools, skills don't have their place actually the opposite: they matter a lot because they do steer the agent with great power comes great responsibility
tosh 3mo ago
Pi is pretty good actually (it has way less context spam, fewer tools and smaller system prompt than the usual suspects) databricks also looked at this: https://earendil.com/posts/pi-autoresearch-and-databricks/ https://earendil.com/posts/pi-autoresearch-and-databricks/
j45 3mo ago
This can work great until the models get tweaked and you realize you’ve been working based on the currents of the model this week. Creating a robust enough setup that can work on different models or different versions of a model is critical.
- tosh 3mo ago
  I agree in principle but in practice this is not easy even with a robust, well thought through setup every model behaves in its own way, some adhere more to a system prompt, another model has more recent cut-off time and knows about new parts in the stdlib some oversteer, some understeer … if you want the best performance unfortunately there is not really a way other than to constantly adapt the harness/clutches/context to the model du jour
  - j45 3mo ago
    I agree in any practice, it's not easy :) In a non-deterministic world the compiler is no longer consistent anyways.
icedchai 3mo ago
I like the smol philosophy. I've been wondering how much of this stuff is closer to meaningless incantations, voodoo with no real benefit.
giancarlostoro 3mo ago
I agree. It's worthwhile to reset your memories and instructions for your models after a major version or two and finetune your needs with the newer model in mind. You can always backup your old instructions / memories. Personally I try to be hands-off with rules and very simple.

by tosh 3mo ago

hansonkd 3mo ago

Also just routine upkeep. Over time if you keep adding new rules these things can get bloated. I deleted 90% of my spec and things improved dramatically.

tosh 3mo ago

I'm not saying system prompts, agents.md, tools, skills don't have their place actually the opposite: they matter a lot because they do steer the agent with great power comes great responsibility

tosh 3mo ago

Pi is pretty good actually (it has way less context spam, fewer tools and smaller system prompt than the usual suspects) databricks also looked at this: https://earendil.com/posts/pi-autoresearch-and-databricks/ https://earendil.com/posts/pi-autoresearch-and-databricks/

j45 3mo ago

This can work great until the models get tweaked and you realize you’ve been working based on the currents of the model this week. Creating a robust enough setup that can work on different models or different versions of a model is critical.

tosh 3mo ago
I agree in principle but in practice this is not easy even with a robust, well thought through setup every model behaves in its own way, some adhere more to a system prompt, another model has more recent cut-off time and knows about new parts in the stdlib some oversteer, some understeer … if you want the best performance unfortunately there is not really a way other than to constantly adapt the harness/clutches/context to the model du jour
- j45 3mo ago
  I agree in any practice, it's not easy :) In a non-deterministic world the compiler is no longer consistent anyways.

icedchai 3mo ago

I like the smol philosophy. I've been wondering how much of this stuff is closer to meaningless incantations, voodoo with no real benefit.

giancarlostoro 3mo ago

I agree. It's worthwhile to reset your memories and instructions for your models after a major version or two and finetune your needs with the newer model in mind. You can always backup your old instructions / memories. Personally I try to be hands-off with rules and very simple.