4 ms·
Show HN: Vector Embedding Server in under 100 lines of code
- topicseed 3y agoI see you're skipping too long sentences — any thoughts on how you would handle them and chunk them further if they weren't skipped? Good to see non-Go code from you, ha ;)
- mrjn 3y agoAuthor here. I was looking for a Docker-based server, which can expose a simple endpoint to generate vector embeddings for documents. The solution needs to deal with lengthy documents that exceed the 512-token limit enforced by E5 models. Such documents require intelligent chunking, ideally at sentence boundaries, followed by taking a mean of the vectors, to work effectively. Since I couldn't find a solution that met these criteria, I decided to create this setup myself.