3 ms·
> html Would be willing to bet this is the issue. Adding html files to context for gemini models results in a ton of token use.
by riskassessment 1y ago
> html
Would be willing to bet this is the issue. Adding html files to context for gemini models results in a ton of token use.
- gcr 1y agowhy? EDIT: why must users care?
- SPICLK2 1y agohttps://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags https://stackoverflow.com/questions/1732348/regex-match-open...
- croes 1y agoThe accepted answer is one that doesn’t care about the questioner‘s use case and instead gives a pretty excessive "Don‘t do it"
- lukan 1y agoIt does also give the right solution, using an xml parser.
- croes 1y agoWe don’t know the use case. Maybe the questioner is also in full control of the HTML creation and they don’t need a parser for all possible HTML edge cases.
- SPICLK2 1y agoMaybe they are, but they would also need to ensure a well-defined subset of HTML and also show that the subset is a reglar (Chomsky Type 3) grammar. It seems that even the very conceptually simple example given by the questioner is impossible.
- kulahan 1y agoGotta learn all the quirks of the model before it's replaced in 8 minutes.
- NaomiLehman 1y agoQuirks? like context window?
- kulahan 1y agoI'm saying it's egregious to expect all users to know the fact that an HTML document, for some reason, uses an enormous amount of context in an LLM designed specifically for working with code.