4 ms·
Llama-3 70B on my local MacBook works wonderfully for these tasks.
by a_wild_dandan 2y ago
Llama-3 70B on my local MacBook works wonderfully for these tasks.
- spaniard89277 2y agoHow's the Pipeline? Do you pass all the html to the LLM? Isn't the context window a problem?
- a_wild_dandan 2y agoThere are phenomenal web scraping tools to crudely "preprocess" the document a bit, slashing outer HTML fluff while preserving the small subset of actual data. From there, 8k tokens (or whatever) goes really far.
- LunaSea 2y agoAt a very generous 50 tokens per second doesn't that still leave you with more than two and a half minutes (160s) processing time per document?