3 ms·

Steve Yegge – RAG to Riches

peter_d_sherman 3y ago
>"Cody’s secret sauce and differentiator has always been Sourcegraph’s deep understanding of code bases, and Cody has tapped into that understanding to create the perfect RAG-based coding assistant, which by definition is the one that produces the best context for the LLM. That’s what RAG (retrieval-augmented generation) is all about. You augment the LLM’s generation by retrieving as much information as a human might need in order to perform some task, and feeding that to the LLM along with the task instructions. Cody’s RAG is a variant of a rec system, recommending relevant code and other context to the LLM for each task or query–a point made recently on Latent Space by our friends there who recently invited Beyang and myself on for a podcast. Good recommenders need numerous models, domain knowledge bases, fast retrieval over large corpora, and other infrastructure that Sourcegraph had already largely built for code. Producing the perfect context is a dark art today , but I think Cody is likely the furthest along here from what I can see. Cody’s Context has graduated from “hey we have vector embeddings” to “hey we have a whole squad of engines.” The key to generating great context is to look at the problem from many different lenses , and each of Cody’s backend context providers is a different lens. On the retrieval side, Cody has many code-understanding lenses: Among them, a high-speed IDE-like code graph, multiple high-performance code search indexes and engines, natural language search, and more, all acting in concert as specialized retrieval backends, both deterministic and model-backed. Over on the context fetching side, for getting info for Cody to slice and dice, Cody is beginning to incorporate multiple context sources beyond just your repos. This gives Cody lenses into metadata and documentation that’s not necessarily evident from the code itself, such as any really critically important information that the author of the code didn’t put into a COMMENT." [...] >"Cody is now Generally Available, and Cody Pro is free until February 2024." [...] >"And it has a free tier for you to just screw around with it" First, I love anything and everything that Steve Yegge writes. Second, here we have the understanding that Context -- is everything in generative LLM AI... Third, we have the concept of Context Providers (which could be called in AI vernacular/AI jargon "Lenses", or perhaps "Context Lenses"...) Anyway, a great article! Steve Yegge's articles are always great!

>"Cody’s secret sauce and differentiator has always been Sourcegraph’s deep understanding of code bases, and Cody has tapped into that understanding to create the perfect RAG-based coding assistant, which by definition is the one that produces the best context for the LLM. That’s what RAG (retrieval-augmented generation) is all about. You augment the LLM’s generation by retrieving as much information as a human might need in order to perform some task, and feeding that to the LLM along with the task instructions. Cody’s RAG is a variant of a rec system, recommending relevant code and other context to the LLM for each task or query–a point made recently on Latent Space by our friends there who recently invited Beyang and myself on for a podcast. Good recommenders need numerous models, domain knowledge bases, fast retrieval over large corpora, and other infrastructure that Sourcegraph had already largely built for code. Producing the perfect context is a dark art today , but I think Cody is likely the furthest along here from what I can see. Cody’s Context has graduated from “hey we have vector embeddings” to “hey we have a whole squad of engines.” The key to generating great context is to look at the problem from many different lenses , and each of Cody’s backend context providers is a different lens. On the retrieval side, Cody has many code-understanding lenses: Among them, a high-speed IDE-like code graph, multiple high-performance code search indexes and engines, natural language search, and more, all acting in concert as specialized retrieval backends, both deterministic and model-backed. Over on the context fetching side, for getting info for Cody to slice and dice, Cody is beginning to incorporate multiple context sources beyond just your repos. This gives Cody lenses into metadata and documentation that’s not necessarily evident from the code itself, such as any really critically important information that the author of the code didn’t put into a COMMENT." [...] >"Cody is now Generally Available, and Cody Pro is free until February 2024." [...] >"And it has a free tier for you to just screw around with it" First, I love anything and everything that Steve Yegge writes. Second, here we have the understanding that Context -- is everything in generative LLM AI... Third, we have the concept of Context Providers (which could be called in AI vernacular/AI jargon "Lenses", or perhaps "Context Lenses"...) Anyway, a great article! Steve Yegge's articles are always great!