4 ms·
How's the Pipeline? Do you pass all the html to the LLM? Isn't the context window a problem?
by spaniard89277 2y ago
How's the Pipeline? Do you pass all the html to the LLM? Isn't the context window a problem?
- a_wild_dandan 2y agoThere are phenomenal web scraping tools to crudely "preprocess" the document a bit, slashing outer HTML fluff while preserving the small subset of actual data. From there, 8k tokens (or whatever) goes really far.
- LunaSea 2y agoAt a very generous 50 tokens per second doesn't that still leave you with more than two and a half minutes (160s) processing time per document?