4 ms·
Author of the text here. I will be honest with why I wrote it, the rtk ai looks very odd to me as software engineer, the number of stars, no mention of accuracy
by lackoftactics 4mo ago
Author of the text here. I will be honest with why I wrote it, the rtk ai looks very odd to me as software engineer, the number of stars, no mention of accuracy and how management is pushing that stuff to optimize costs. Now people are wrapping every possible command in rtk and trying to handle all major possible commands and decide which output you should get.
- ianwalter 4mo agoWhy didn’t you offer any real world usage numbers to illustrate your point? I found this unhelpful.
- lloyd-christmas 4mo agoI read another post oddly similar earlier today that has more explicit data on that authors codebase: https://codepointer.substack.com/p/cutting-llm-token-costs-with-rtk https://codepointer.substack.com/p/cutting-llm-token-costs-w... TLDR; ~3-4% savings to actual API costs with rtk, caveman, and headroom combined, but nothing tangible on if those cost reductions came at a cost of quality. By their calculations, rtk saved them $4.96 on a $926 bill.
- bcollins34 4mo ago^recommend reading this one
- lackoftactics 4mo agoThat’s the fair point. The rtk promotional posts point to 60-90% tokens savings and there is no mention how they perform accuracy wise. The commenter below did great job pointing to resource showing caveman, rtk saving just couple bucks on $926 bill. Thanks, Llyoyd Christmas for linking to useful substack
- fumeux_fume 4mo agohttps://en.wikipedia.org/wiki/Brandolini%27s_law https://en.wikipedia.org/wiki/Brandolini%27s_law
- lackoftactics 4mo agoThanks for the link! I thought I know every obscure law and I was so wrong.
- jahala 4mo agoWould sincerely love to hear your thoughts on https://www.github.com/jahala/tilth https://www.github.com/jahala/tilth - it’s a different approach than RTK, benchmarked to reduce cost per correct answer by ~40%
- lackoftactics 4mo agoLooked at your repo, even starred. On the surface, I like your approach a bit better. It looks like your idea sits at the space between semantic search and compressing tokens. I was into semantic search before, but mostly trying to vectorize codebase instead of tree sitter and couldn’t make the semantic search work for me. Thanks for sharing!
- jvican 4mo agoAn ex colleague is working on Headroom, a much more legit alternative to RTK. They provide accuracy benchmarks in the repo and are transparent about the compression algorithms used for the different output types. I liked their approach a lot better than RTK and thought it might be relevant for you. https://github.com/chopratejas/headroom https://github.com/chopratejas/headroom
- polski-g 4mo agoVery cool. I'll probably switch away from AFT to this. Can you add tree-sitter-bash?