metawake

4 ms·

1. 1.▲
   Show HN: Ragprobe – measure RAG domain difficulty before deploying,no embeddings(pypi.org)

   1 points by metawake 7mo ago|0 comments
2. 2.▲
   PageIndex (19k stars) scored 44% on legal docs. Same as vector RAG(medium.com)

   1 points by metawake 7mo ago|0 comments
3. 3.▲
   by metawake 9mo ago

   Great suggestion!! this is exactly the right methodology for establishing confidence intervals. I've added this to the roadmap as `--bootstrap N`: ragtune simulate --queries queries.json --bootstrap 5 # Output: # Rec
4. 4.▲
   by metawake 9mo ago

   Author here. Built RagTune to stop guessing at RAG configs. Surprising findings: 1. On legal text (CaseHOLD), 1024 chunks scored WORST (0.618). The "small" 256 chunks won (0.664). 7% swing. 2. On Wikipedia text? All chunk size
5. 5.▲
   Show HN: RAG chunk size "best practices" failed on legal text – I benchmarked it(medium.com)

   2 points by metawake 9mo ago|3 comments
6. 6.▲
   by metawake 9mo ago

   Thanks! To answer your questions: *Backends:* Currently supports Qdrant, pgvector, Weaviate, Chroma, and Pinecone. Adding more is straightforward since it's just implementing a Store interface. Let me know if I missed some good backend
7. 7.▲
   Show HN: RagTune – EXPLAIN ANALYZE for your RAG retrieval layer(github.com)

   1 points by metawake 9mo ago|1 comments
8. 8.▲
   by metawake 9mo ago

   I am using a vector DB using Docker image. And for debugging and benchmarking local RAG retrieval, I've been building a CLI tool that shows what's actually being retrieved: ragtune explain "your query" --collection
9. 9.▲
   by metawake 1y ago

   I made a small project ( https://github.com/metawake/puppetry-detector ) to detect this type of LLM policy manipulation. It's an early idea using a set of regexp patterns (for speed) and a couple of phases of text a
10. 10.▲
    Exploring spaCy-based prompt compression for LLMs – thoughts welcome(github.com)

    1 points by metawake 1y ago|1 comments
11. 11.▲
    by metawake 1y ago

    Hi HN, I’ve been exploring whether prompt compression — done before sending input to LLMs — can help cut down on token usage and cost without losing key meaning. Instead of using a neural model, I wrote a small open-source tool that uses ha
12. 12.▲
    Anti-fragile web development, preventing “Black Swans”(metawake.tumblr.com)

    1 points by metawake 12y ago|1 comments