3 ms·

Hi all, Brokk creator here, happy to answer any questions! I made an intro video with a live demo here: https://www.youtube.com/watch?v=Pw92v-uN5xI https://www

Hi all, Brokk creator here, happy to answer any questions! I made an intro video with a live demo here: https://www.youtube.com/watch?v=Pw92v-uN5xI https://www.youtube.com/watch?v=Pw92v-uN5xI

soco 1y ago
Is there something also to read for those of us who will never watch videos?
- lutzleonhardt 1y ago
  Hi, yes there are some blog posts: https://brokk.ai/blog/brokk-under-the-hood https://brokk.ai/blog/brokk-under-the-hood
bchapuis 1y ago
Really cool project! I tried it a couple of weeks ago with an Anthropic API key and will give it another shot. Could you share a bit more about how you handle code summarization? Is it mostly about retaining method signatures so the LLM gets a high-level sense of the project? In Java, could this work with dependencies too, like source JARs? More generally, how’s it been working with Java for this kind of project? Does limited GPU access ever get in the way of summarization or analysis (Jlama)?
- jbellis 1y ago
  That officially makes you an early adopter, thanks! Yes, it's basically just parsing for declarations. (If you doubleclick on any context in the Workspace it will show you exactly what's inside.) You have to import the dependencies via File -> Decompile Dependency and then it gets parsed like the rest of your source, only read-only. I have a love-hate relationship with Java, mostly love lately, the OpenJDK team is doing a great job driving the language forward. It's so much faster than Python, it's nice being able to extend a language in itself and get native performance. Since we're just using Jlama to debounce the LLM requests, we can use a tiny model that runs fine on CPU alone. The latest Jlama supports GPU as well but we're not using that.
neoncontrails 1y ago
I'd be interested to try this out. I'm especially keen on AI tools that implement a native RAG workflow. I've given Cursor documentation links, populated my codebase with relevant READMEs and diagram files that I'm hoping might provide useful context, and yet when I ask it to assist on some refactoring task it often spends 10-20 minutes simply grepping for various symbol names and reading through file matches before attempting to generate a response. This doesn't seem like an efficient way for an LLM to navigate a medium-sized codebase. And for an IDE with first-class LLM tooling, it is a bit surprising that it doesn't seem to provide powerful vector-based querying capabilities out of the box — if implemented well, a Google-like search interface to one's codebase could be useful to humans as well as to LLMs. What does this flow look like in Brokk? Do models still need to resort to using obsolete terminal-based CLI tools in order to find stuff?
- lutzleonhardt 1y ago
  We implemented a multi-step process to find the required context: 1. Quick Context Shows the most relevant files based on a pagerank algorithm (static analysis) and semantic embeddings (JLama inference engine). The input are the instructions and the AI workspace fragments (i.e. files). 2. Deep Scan A richer LLM receives the summaries of the AI workspace files (+instructions) and returns a recommendation of files and tests. It also recommends the type of inclusion (editable, read-only, summary/skeleton). 3. Agentic Search The AI has access to a set of tools for finding the required files. But the tools are not limited to grep/rg. Instead you can: - find symbols (classes, methods, ...) in the project - ask for summaries/skeletons of files - provide class or method implementations - find usages of symbols (where is x used?) - call sites (in/out) ... You can read more about this in the Brokk.ai blog: https://brokk.ai/blog/brokk-under-the-hood https://brokk.ai/blog/brokk-under-the-hood
silverlake 1y ago
No offense, but that video is brutally boring. Even at 1.5x speed I couldn’t get past 10 min. You should transcribe the audio and use an LLM to write a punchy sales pitch.
corysama 1y ago
How large is "Large"? Are we testing on Unreal Engine? :D
- jbellis 1y ago
  no, but I've tested on intellij (~5M loc, takes forever to import b/c of delombok, do not recommend)
- lutzleonhardt 1y ago
  I tested it with Ghidra recently and got very good results
saratogacx 1y ago
Likely not an important note but the name sounds close enough to grok that I assumed this was a spin off of some xAI product. I had to look around to see if it was actually associated (it looks like it isn't) but it may be something to be aware of.

Hi all, Brokk creator here, happy to answer any questions! I made an intro video with a live demo here: https://www.youtube.com/watch?v=Pw92v-uN5xI https://www.youtube.com/watch?v=Pw92v-uN5xI

soco 1y ago

Is there something also to read for those of us who will never watch videos?

lutzleonhardt 1y ago
Hi, yes there are some blog posts: https://brokk.ai/blog/brokk-under-the-hood https://brokk.ai/blog/brokk-under-the-hood

bchapuis 1y ago

Really cool project! I tried it a couple of weeks ago with an Anthropic API key and will give it another shot. Could you share a bit more about how you handle code summarization? Is it mostly about retaining method signatures so the LLM gets a high-level sense of the project? In Java, could this work with dependencies too, like source JARs? More generally, how’s it been working with Java for this kind of project? Does limited GPU access ever get in the way of summarization or analysis (Jlama)?

jbellis 1y ago
That officially makes you an early adopter, thanks! Yes, it's basically just parsing for declarations. (If you doubleclick on any context in the Workspace it will show you exactly what's inside.) You have to import the dependencies via File -> Decompile Dependency and then it gets parsed like the rest of your source, only read-only. I have a love-hate relationship with Java, mostly love lately, the OpenJDK team is doing a great job driving the language forward. It's so much faster than Python, it's nice being able to extend a language in itself and get native performance. Since we're just using Jlama to debounce the LLM requests, we can use a tiny model that runs fine on CPU alone. The latest Jlama supports GPU as well but we're not using that.

neoncontrails 1y ago

I'd be interested to try this out. I'm especially keen on AI tools that implement a native RAG workflow. I've given Cursor documentation links, populated my codebase with relevant READMEs and diagram files that I'm hoping might provide useful context, and yet when I ask it to assist on some refactoring task it often spends 10-20 minutes simply grepping for various symbol names and reading through file matches before attempting to generate a response. This doesn't seem like an efficient way for an LLM to navigate a medium-sized codebase. And for an IDE with first-class LLM tooling, it is a bit surprising that it doesn't seem to provide powerful vector-based querying capabilities out of the box — if implemented well, a Google-like search interface to one's codebase could be useful to humans as well as to LLMs. What does this flow look like in Brokk? Do models still need to resort to using obsolete terminal-based CLI tools in order to find stuff?

lutzleonhardt 1y ago
We implemented a multi-step process to find the required context: 1. Quick Context Shows the most relevant files based on a pagerank algorithm (static analysis) and semantic embeddings (JLama inference engine). The input are the instructions and the AI workspace fragments (i.e. files). 2. Deep Scan A richer LLM receives the summaries of the AI workspace files (+instructions) and returns a recommendation of files and tests. It also recommends the type of inclusion (editable, read-only, summary/skeleton). 3. Agentic Search The AI has access to a set of tools for finding the required files. But the tools are not limited to grep/rg. Instead you can: - find symbols (classes, methods, ...) in the project - ask for summaries/skeletons of files - provide class or method implementations - find usages of symbols (where is x used?) - call sites (in/out) ... You can read more about this in the Brokk.ai blog: https://brokk.ai/blog/brokk-under-the-hood https://brokk.ai/blog/brokk-under-the-hood

silverlake 1y ago

No offense, but that video is brutally boring. Even at 1.5x speed I couldn’t get past 10 min. You should transcribe the audio and use an LLM to write a punchy sales pitch.

corysama 1y ago

How large is "Large"? Are we testing on Unreal Engine? :D

jbellis 1y ago
no, but I've tested on intellij (~5M loc, takes forever to import b/c of delombok, do not recommend)
lutzleonhardt 1y ago
I tested it with Ghidra recently and got very good results

saratogacx 1y ago

Likely not an important note but the name sounds close enough to grok that I assumed this was a spin off of some xAI product. I had to look around to see if it was actually associated (it looks like it isn't) but it may be something to be aware of.