2 ms·
They say the chunker is the most important part, but theirs looks rudimentary: https://github.com/agentset-ai/agentset/blob/main/packages/engine/src/chunk.ts ht
by esafak 1y ago
They say the chunker is the most important part, but theirs looks rudimentary: https://github.com/agentset-ai/agentset/blob/main/packages/engine/src/chunk.ts https://github.com/agentset-ai/agentset/blob/main/packages/e...
That is, there is nothing here that one could not easily write without a library.
- tifa2up 1y agoOP here. We've been working on agentset.ai full-time for 2 months. The product now gets you something working quite well out of the box. Better than most people with no experience in RAG (I'd say so with confidence). Ingestion + Agentic Search are two areas that we're focused on in the short term.
- teraflop 1y agoI'm not sure there is a chunker in this repo. The file you linked certainly doesn't seem to perform any chunking, it just defines a data model for chunks. The only place I see that actually operates on chunks does so by fetching them from Redis, and AFAICT nothing in the repo actually writes to Redis, so I assume the chunker is elsewhere. https://github.com/agentset-ai/agentset/blob/main/packages/jobs/src/tasks/process-document.ts#L185 https://github.com/agentset-ai/agentset/blob/main/packages/j...