3 ms·
That used to matter to me back in the days when the best models still only accepted ~32,000 tokens, but these days even the models that run on my laptop are hap
by simonw 2mo ago
That used to matter to me back in the days when the best models still only accepted ~32,000 tokens, but these days even the models that run on my laptop are happy with ~100,000 and the hosted models I use take ~200,000 or more.
- slowin 2mo agoIf it's one of many tool calls, I'd assume that less is more.
- simonw 2mo agoThe trick there is to use a subagent to read the HTML page and extract the relevant information, than dumping all that HTML into your top-level session. That's effectively using an LLM as an HTML to markdown converter, which is both absurdly wasteful and also surprisingly inexpensive (if you use a model like GPT-5.6 Luna.)
- honr 2mo agoI found HTML itself works [slightly] better with some LLMs (that I happen to frequent). So, when scraping, I parse the resulting html, simplify it, and produce simpler html contents (closer to semantics of what I guessed the real content was). Given processing tools for html are far more mature, I tend to keep content in semantic, low structure html.
- selcuka 2mo agoThey accept more tokens these days, but they are still more accurate with a shorter context [1]. [1] https://arxiv.org/abs/2307.03172 https://arxiv.org/abs/2307.03172