4 ms·
the obsession on token discounts recently is pretty funny. If you extrapolate far enough you end up back to where we started, programming languages.
by ares623 6mo ago
the obsession on token discounts recently is pretty funny. If you extrapolate far enough you end up back to where we started, programming languages.
- nunodonato 6mo agoremember TOON? it killed JSON /s just in case
- xandrius 6mo agoThe thing which is 98% JSON and absolutely didn't kill JSON?
- xandrius 6mo agoYeah, I wonder what kind of work people do that they need more than 500k or 1M context window. Even when it's a big project, breaking it down doesn't change the output quality.
- nunodonato 6mo agoit's crazy, I have seen so many projects popping up just focusing on reducing token usage. At least caveman speak is funny! Have to say that since we switched to our own model in a rented GPU, we stopped worrying about tokens and just use the hell out of our AI as much as we want :)
- thegagne 6mo agoHah, I think about this all the time. I think we subtly desire LLMs to be more and more deterministic and efficient. This is why one of the main uses of LLMs is building tools to make their job easier. I made my own project, with one of the goals being discounting tokens, but found that the real goal was just ensuring quality and making things more programmatic. https://ktext.dev https://ktext.dev Basically ends up being agents.md in a schema driven yaml file. Thinking about extending it to also generate or replace skill.md. I think the proliferation of markdown is cool, and lowers the barrier for entry, but it’s also very unpredictable and loose. I think over time we will drive these to be more like config files instead of free text.