2 ms·

Feedback: 1. Description* reeks of AI slop; it extended a surface-level prompt into longer surface-level insights. *: description as in GitHub README 2. #1 cr

Feedback: 1. Description* reeks of AI slop; it extended a surface-level prompt into longer surface-level insights. *: description as in GitHub README 2. #1 creates a situation where I go through reading this long thing, and realize it has no answers to even the first-level questions that would be on anyones mind (what model? where is it run?). For this to become something I'll take the time to integrate into my core workflow and try, it has to be *much* more transparent. 3. Claims in the description are ~impossible. 3b. Up front, I feel your pain, there's a hard set of constraints to navigate here given A) marketing needs to be concise B) people play fast and loose with conciseness vs. accuracy C) you need to sounds as good as the people in B. 3c. That being said, we're crossing into year 3 of post-ChatGPT. People, especially in your target audience, will know when they're reading* that you're reframing "I give text to the LLM which can theoratically do $X" into features, and users expect features to be designed* and intentional. If they are, you should definitely highlight that to differentiate from people who just throw it into the LLM. 3d. Analyzes your entire repository context: impossible, literally, unless you're feeding it to Gemini only. I have about 20KLOC and its multiples of Llama context size. 3e. "Understands code relationships and dependencies" see 3c. 3f. "Contextual Analysis: Reviews code changes within the full repository context": see 3d. 3g. "Language Agnostic: Supports all major programming languages.": see 3c (is there actual work done to do this, or is this just "well, given I just send the text to the LLM, everything is supported"?) 4. nit: Should be "Show HN: LlamaPReview, AI Github PR Reviewer That Learns Your Codebase"

cutemonster 2y ago
> Analyzes your entire repository context: impossible That might be sort of doable, by extracting all function signatures together with brief descriptions, and including that in the context, and maybe a graph showing how they call each other, but none of the actual implementations. Except for the file(s) under review, which would be included in full.
Jet_Xu 2y ago
Thank you for such detailed and thoughtful feedback! Really appreciate the time you took to analyze our claims and point out the areas needing more clarity. You're absolutely right about the marketing copy - we should be more precise and transparent about what we actually do vs. what's aspirational. Regarding "understanding code relationships and dependencies": We're building a knowledge graph of the entire repository that captures code relationships, function calls, and module dependencies. This graph is then used with GraphRAG to fetch relevant context for each PR, allowing the LLM to understand the broader impact of changes. Important to note: We take privacy very seriously. All code analysis happens in-memory during PR reviews - we don't permanently store any source code or build persistent knowledge bases from customer code. The knowledge graph is generated and used on-the-fly for each review session. This approach helps us work around context window limitations while providing meaningful insights. However, I should note that this feature is still under active development - we're continuously improving the graph construction and relevancy matching. Would love to hear your thoughts on this approach. We're committed to building something genuinely useful for developers rather than just another LLM wrapper.

cutemonster 2y ago

> Analyzes your entire repository context: impossible That might be sort of doable, by extracting all function signatures together with brief descriptions, and including that in the context, and maybe a graph showing how they call each other, but none of the actual implementations. Except for the file(s) under review, which would be included in full.

Jet_Xu 2y ago

Thank you for such detailed and thoughtful feedback! Really appreciate the time you took to analyze our claims and point out the areas needing more clarity. You're absolutely right about the marketing copy - we should be more precise and transparent about what we actually do vs. what's aspirational. Regarding "understanding code relationships and dependencies": We're building a knowledge graph of the entire repository that captures code relationships, function calls, and module dependencies. This graph is then used with GraphRAG to fetch relevant context for each PR, allowing the LLM to understand the broader impact of changes. Important to note: We take privacy very seriously. All code analysis happens in-memory during PR reviews - we don't permanently store any source code or build persistent knowledge bases from customer code. The knowledge graph is generated and used on-the-fly for each review session. This approach helps us work around context window limitations while providing meaningful insights. However, I should note that this feature is still under active development - we're continuously improving the graph construction and relevancy matching. Would love to hear your thoughts on this approach. We're committed to building something genuinely useful for developers rather than just another LLM wrapper.