4 ms·
Does anyone else hate reading AI summaries of code? Code can be pithy, but at least its terse compared to prose. When you add how verbose LLMs can be, I often e
by dexwiz 2mo ago
Does anyone else hate reading AI summaries of code? Code can be pithy, but at least its terse compared to prose. When you add how verbose LLMs can be, I often end up reading a paragraph to explain a few lines. Or the opposite happens where the summary skips important edge cases or criteria. "You're right, X also does Y. I missed that in my initial analysis," is much too common of a phrase.
I like the idea of using LLMs to transform code into something more readable, and vice versa. I am not sure if meandering paragraphs and linear lists are the best targets.
- markstos 2mo agoRight, the value is not so clear of having multiple people wade through lengthy LLM responses. Part of the value that a human in the loop adds is figuring out what's correct and important in the response and passing on a more polished result.
- throwaway613746 2mo ago[dead]
- jvuygbbkuurx 2mo agoI don't like the default explanations. But I prompt for small code snippets with explanations of a problem and the solution. This is in the context of extending features, fixing bugs, reviewing new code etc. I still skim the code, but it's nice to have a somewhat thoughtful overview of the key points like database schema, API spec, algorithm or abstraction. It makes it easier to skim a large diff without feeling lost. It also quite often catches some weird choices that might slip through it not carefully reasoning about the code. For me it is hard to understand code I didn't write myself so I have landed on this workflow.
- zahlman 2mo agoI've found that I don't mind getting a lot of text back from an LLM, when I was the one who prompted it. I can easily enough let my eyes flit around in the text and figure out what I need to, and I expect that putting all that text in the context window will help with the rest of the conversation. It's when that text gets copied and pasted into a blog, or a PR, etc. that it really galls.
- fragmede 2mo agoYeah. Human psychology is weird.
- cmoski 2mo agoAI summaries are great for learning which parts of the codebase are load bearing.
- armchairhacker 2mo agoWe need a new specification language for summaries. I think just a (pseudo?-)programming language’s API (modules, interfaces, method signatures), but more concise and descriptive than Java, maybe Haskell or OCaml minus quirks (…the academics were this close to an ideal syntax, maybe they should’ve gave up those languages’ LR parsers…)
- jake_and_fatman 2mo ago[dead]
- bmurphy1976 2mo agoI hate fighting with the AI to get a good summary. Reading the good summary is great. Getting to it is the challenge. First attempt: mountains of meta conversation and not answering the original question. Second attempt: shorter and more concise, cuts out some of the nonsense, but terribly written. Third attempt: goes off the rails, misunderstands what you are asking and tries pushing the work onto you. Fourth attempt: finally something that's concise, reasonably well written, to the point and passable.
- lucascdotnet 2mo agoI'm a bit confused by your use of 'pithy, but at least its terse'. Don't pithy and terse mostly mean similar things? Here it seems like you're contrasting them. Not trying to be pedantic, apologies if it comes off that way.
- superturkey650 2mo agoI think he’s saying that’s it’s dense and _too_ full of meaning. So, while terse, it can also hide extra meaning in its density.
- card_zero 2mo agoGnomic, then (on my mind because I saw it used yesterday, thanks flir).
- jake_and_fatman 2mo ago"Bob can be blunt but at least he's honest." Words can have positive or negative connotations for the same notion.
- card_zero 2mo agoThat's insightful! In a negative way.
- lucascdotnet 2mo agoThats a good point, however, you wouldn't say "Bob can be honest, but at least he's blunt" which is I think more faithful to the initial quote. But point taken.
- deleted 2mo ago[deleted]
- derefr 2mo agoI wonder if a productive avenue might be "doing aspect-oriented programming in reverse": asking the LLM not to "summarize" the code per se, but rather to "clarify" it by transforming it into what a programming blog post would call a "toy example" of what the code is doing, by stripping out all the (non-semantic) error-handling, logging, metrics incrementing, etc — all the things that you might treat as their own "aspects" under AOP.
- redrove 2mo agoSo..pseudo code?
- cadamsdotcom 2mo agoYour AI summary is raw output, like uncooked mince meat. Ask for a rewriting pass! Get it to edit for word count, remove redundancies, remove run-on sentences, and change sentence structure to use active voice. You wouldn't ship your first draft - why let AI do it?
- bigstrat2003 2mo agoI mean... the problem with that, as with all the "you have to go back and forth with the AI" scenarios, is that by the time I finish with that I may as well have just written it myself to begin with. Would take a similar amount of time, actually have my voice in it, and not cost money for tokens.
- cadamsdotcom 2mo agoIf all you need is a readable summary you can understand you should definitely try asking. You might get a surprise! Worst case you lose 10 seconds of your life. Not too bad!
- gitaarik 2mo agoI agree, I also sometimes do stuff manually when it's easier to do manually than to ask AI. Took me years before I got it though.
- jbdamask 2mo agoI use AI explainers a lot in my own projects and have skills that give outputs I like. I'll even pipe the output into NotebookLM to make a video sometimes. This is actually a decent way to grok complex subsystems (YMMV). What I don't care for is the pretty-yet-verbose HTML outputs that I see from some teams. They can be way too long and it's often the case that the engineer who made it didn't read the whole thing.
- paulhebert 2mo agoThose HTML reports are almost always filled with fluff. As someone who likes a nicely designed report document, slapping a coat of paint on slop output isn’t helpful
- jnpnj 2mo agoThe "language model" aspect shows IMO, at least for someone who grew up with the material of 90s and 2000s where we describe things more in mechanical, engineering terms. When gpt4 landed it was somehow amazing to see the output, but yeah nowadays I keep skimming through the explanation. It's like an intermediate dev who has nothing else to do but create long sentences to describe what could be simpler.
- o_m 2mo agoThere is a reason Caveman (https://github.com/JuliusBrussee/caveman https://github.com/JuliusBrussee/caveman) is as popular as it is
- gosolozero 2mo agoI thought Caveman was debunked in the token saving argument? In real sessions it doesn’t really save much longterm and made sessions more forgetful/take longer.
- Garlef 2mo agoAsk for a walkthrough of the highlights (with '...' omissions so that the most important parts are quickly visible) instead of a verbal summary.
- findjashua 2mo agoYou're absolutely right!
- ModernMech 2mo agoYes this is why I ask the agent to summarize the code as a comic of a lizard explaining it to a cat.
- sourdecor 2mo agoI wish LLM summary of code was put aside the actual code, like the Jasmine documentation[0]. [0]: https://jasmine.github.io/tutorials/your_first_suite https://jasmine.github.io/tutorials/your_first_suite
- dyauspitr 2mo agoIt’s an LLM, it does what you want. If you don’t like it verbose tell it you want it to be succinct and it’ll do exactly what you ask of it. What sort of complaint is this?
- AlfeG 2mo agoI use caveman to handle this. Not because to I want to spent less tokens, I want to spent less time reading.
- idk1 2mo agoYeah, I have a rule in my agents MD that essentially says the code doesn't need a comment because it should be readable. And ask me in the exceptionally rare instances where it needs a summary or a comment. I am a Ruby and JavaScript developer, so I suppose that is a lot more readable than some other languages, but broadly, I've found code summaries and code comments to not be necessary if the code is very elegantly written.
- sensanaty 2mo agoWhenever I read any AI-generated documentation of any kind, it feels like my brain immediately blanks and I can't comprehend what I'm reading. It's like my brain blocks out the particular combo of word AIs love to use and 0 information manages to make it in there. I think it's because it's basically always a high-level, rough overview of a topic that isn't quite correct. It's using 10x the words necessary to say nothing at all. It is a large, featureless wall of words that ultimately tells you absolutely nothing. It's simultaneously too verbose while also not actually saying anything useful. Maybe my attention span has just gotten worse or something, but I don't really get the same sensation reading good quality docs that I know were human-generated so it must be something specific to how AIs form their paragraphs of noise.
- sharno 2mo agoI like to ask for a code tour using snippets of the most important types and functions. Much better to read through
- gofreddygo 2mo agoOne thing i hate more is the tendency of people now to rely on such summaries over the code and copy-pasting 400 line summaries of something fundamentally wrong. Code is more precise than english language. If it wasn't everyone would just use english to write programs.