3 ms·
I would like to have deeper comparison with alternatives like rtk, which are already fast and written in rust, also the previous comments mentioned something th
by alex7o 4mo ago
I would like to have deeper comparison with alternatives like rtk, which are already fast and written in rust, also the previous comments mentioned something that has been a know problem with rtk that it sometimes strips the thing that the llm needs (or expects, causing more work to need to happan not less)
- zdkaster 4mo agoIn term of token saving performance, it should be on par with rtk since it is basically the same idea. The major different is rtk bundled hundreds of filter logic and no room for user to adjust without maintaing user owned fork or opening the pull request while lowfat is using opposite architectural approach by removing almost all filter logic in the binary and seperate user filters as a plugin system
- giancarlostoro 4mo agoYeah I use rtk and would love to see a comparison.
- onlyrealcuzzo 4mo agoNone of these tools measure how effective they are... It's a massive red flag to me when you could get decent data to see if your thing actually works, and they don't even attempt to... Have the LLM use your tool, run it on several of the coding benchmarks. If you're stingy, run it on the ones that don't cost much. Otherwise, I'm going to assume it doesn't actually work. If it did - Claude, Antigravity, Codex, Pi, or some major player would bundle tools like this into the CLI / harness. AFAIK, none of the major players do. That's a sign to me these don't work in general. I've tried building some tools specific to bug fixing. Intelligently feeding context massively helps smaller models. But, what I've found - surprisingly - is that a smaller, much better focused, including a lot of helpful data as well, has almost no impact on larger models compared to what they do by default. You do save some tokens, though, which is what they're claiming - but not ~99%...
- doix 4mo agoIt's too hard to define what "works" even means in this case. Look at the example savings output. A lot of it is kubectl output. Your suggestion to using coding benchmarks doesn't really capture the whole picture. I haven't seen a benchmark using kubectl. > AFAIK, none of the major players do. That's a sign to me these don't work in general. It's a lose/lose for major players. If it works well, it will lower their revenue. Also there's a high risk it'll significantly worsen results for some people, even if it improves results for others.
- no-name-here 4mo ago> I'm going to assume it doesn't actually work. If it did - Claude, Antigravity, Codex, Pi, or some major player would bundle tools like this into the CLI / harness. VS Code launched it as a feature in their bundled AI functionality last month: https://code.visualstudio.com/updates/v1_121 https://code.visualstudio.com/updates/v1_121
- onlyrealcuzzo 4mo agoBundling implies interest... Defaults imply working...
- irthomasthomas 4mo agoMy partial solution to this was to store the full response in a file and prompt the agent to read that if the condensed version had stuff missing.
- taude 4mo agoI don't think frontier model providers are going to be incentivized to invest in this much, yet. Once inference gets more competitive, sure. I haven't looked lately, but won't be surprised if tools like OpenCode do do what you're suggesting, though. Third-party coding harnesses ARE aligned to deliver this type of feature and optimization.
- varispeed 4mo agoYou can't measure effectiveness, because you never know what kind of model will process your prompt. One request you might get full e.g. Opus and another they'll downgrade it to Sonnet or something more basic. I have this with "Opus 4.8" all the time.
- zdkaster 4mo agoI have just put the comparison in the repo in case you want to checkout.