zmccormick7

11 ms·

1.
Context is the bottleneck for coding agents now(runnercode.com)
196 points by zmccormick7 1y ago|187 comments
2.
by zmccormick7 1y ago
Good to know. I've heard great things about Context7, but haven't experimented with it yet.
3.
by zmccormick7 1y ago
As in the download itself didn't happen when you clicked the download button, or the installation failed?
4.
by zmccormick7 1y ago
Cool, I hadn't heard of Traycer. That does look quite similar! Completely agree. I basically built Runner to codify the process I was already using manually with Claude Code and Gemini. A lot of developers seem to be settling on a simi
5.
by zmccormick7 1y ago
I agree that's a major problem. It's not something I've solved yet. I suspect a web research sub-agent is likely what's needed, so it can pull in up-to-date docs for whatever library you need to work with.
6.
by zmccormick7 1y ago
Gemini is required for the context management sub-agent. You can use any of OpenAI, Anthropic, or Gemini for the main planning and coding agents, but GPT-5 performs the best in my experience. Claude 4 Sonnet works well too, but it's ab
7.
Show HN: Runner – the anti-vibe coding agent(runnercode.com)
22 points by zmccormick7 1y ago|10 comments
8.
Bootstrapping a Coding Agent(runnercode.com)
2 points by zmccormick7 1y ago|0 comments
9.
by zmccormick7 2y ago
The main thing we need to add is metadata filtering, as that's required for a lot of use cases. We're also thinking about adding hybrid search support and multi-factor ranking.
10.
by zmccormick7 2y ago
We've only done full benchmarking with the FIQA dataset, comparing minDB with Chroma. We're going to try it with Qdrant and Weaviate soon too, since they both have support for quantization, which will be a more apples-to-apples co
11.
Show HN: MinDB – an extremely memory-efficient vector database(github.com)
17 points by zmccormick7 2y ago|5 comments
12.
Show HN: Retrieval engine with SOTA performance on challenging RAG benchmarks(github.com)
2 points by zmccormick7 2y ago|0 comments
13.
by zmccormick7 2y ago
Agreed that thresholds don't work when applied to the cosine similarity of embeddings. But I have found that the similarity score returned by high-quality rerankers, especially Cohere, are consistent and meaningful enough that using a
14.
by zmccormick7 2y ago
Agreed. Retrieval performance is very dependent on the quality of the search queries. Letting the LLM generate the search queries is much more reliable than just embedding the user input. Also, no retrieval system is going to return everyth
15.
Solving the out-of-context chunk problem for RAG(d-star.ai)
260 points by zmccormick7 2y ago|89 comments
16.
Embeddings are not all you need(d-star.ai)
3 points by zmccormick7 2y ago|0 comments
17.
by zmccormick7 2y ago
I had the same issue when searching for specific companies/products. It feels like a pretty basic vector search with no hybrid search component or reranking.
18.
by zmccormick7 2y ago
That should work well with the default parameters, so you shouldn't have to do anything special.
19.
by zmccormick7 2y ago
That description is a little vague, so I need to improve that. The use cases we're focused on are ones with 1) dense unstructured text, like legal documents, financial reports, and academic papers; and 2) challenging queries that go be
20.
by zmccormick7 2y ago
I think the difference is that they're building for end users, not developers.
21.
by zmccormick7 2y ago
Agreed. I've gotten a lot of feedback along those lines today, so that's my top priority now.
22.
by zmccormick7 2y ago
So the point of AutoContext is so you don't have to do that two-step process of first finding the right document, and then finding the right section of that document. I think it's cleaner to do it this way, but it's not neces
23.
by zmccormick7 2y ago
That's a great question. I'll start with a little context: most of the users of our existing hosted platform are no-code/low-code developers who choose us because we're the simplest solution for building what they want t
24.
by zmccormick7 2y ago
I think spRAG should be pretty well suited for that use case. I think the biggest challenge will be generating specific search queries off of more general user inputs. You can look at the `auto_query.py` file for a basic implementation of t
25.
by zmccormick7 2y ago
That's great feedback. I actually went back and forth between those two descriptions. I agree that "dense text, like financial reports and legal documents" is more precise. Those are the kinds of use cases this project is bui
26.
by zmccormick7 2y ago
In our AutoContext implementation, the document title gets included with the generated summary. So if you have files that are organized into nested folders with descriptive names, you can input that full file path as the `document_title`.
27.
by zmccormick7 2y ago
Those are just the defaults, and spRAG is designed to be flexible in terms of the models you can use with it. For AutoContext (which is just a summarization task) Haiku offers a great balance of price and performance. Llama 3-8B would also
28.
Show HN: SpRAG – Open-source RAG implementation for challenging real-world tasks(github.com)
69 points by zmccormick7 2y ago|23 comments