3 ms·
CSV is a lot lighter on tokens, compared to json, so it can go further, before a LLM gets exhausted.
by predkambrij 1y ago
CSV is a lot lighter on tokens, compared to json, so it can go further, before a LLM gets exhausted.
- finnborge 1y agoIf you haven't already seen the DeepSeek OCR paper [1], images can be profoundly more token-efficient encodings of information than even CSVs! [1]: https://github.com/deepseek-ai/DeepSeek-OCR/blob/main/DeepSeek_OCR_paper.pdf https://github.com/deepseek-ai/DeepSeek-OCR/blob/main/DeepSe...