We minimized hallucinations mainly by tuning the chunk size, document partitioning, and embedding distance thresholds. In our experience, when you provide high-

We minimized hallucinations mainly by tuning the chunk size, document partitioning, and embedding distance thresholds. In our experience, when you provide high-quality and accurate context, hallucinations become quite rare. That said, it heavily depends on the LLM you’re using — hallucinations are more common with models like Gemini or local LLaMA variants. We’re currently working on extracting pre-structured data (like income statements and cash flows) during PDF processing. We aim to populate these structures upfront and then use tools or special functions to query them. That helps us reduce LLM overhead and improve reliability. The current setup is already working quite well. "So have you assessed that llama3.2 (3B, I imagine — also 1B exists) is enough to answer properly to the sought level of accuracy?" It depends on the use case. Our service is designed to support multiple LLMs, but the best results come from models like GPT-4o or DeepSeek-r1. Initially, we focused on local models and used LLaMA 3.2–3B along with Nomic embeddings, but we eventually moved to more powerful cloud-based models. The main driver was the need to extract structured information (like tables/images) from PDFs. We crop those elements and pass them to vision-capable LLMs to retrieve structured JSON. Running models like LLaVA locally on a Mac was too slow for this use case, and traditional OCR didn’t give the accuracy we needed. So while our stack can still run locally (with somewhat lower quality), we now default to general-purpose cloud LLMs for better performance and accuracy