5 ms·
FWIW, when I'm doing a code review, these are the exact kind of comments that I would tell a committer to remove. That is, it's like it generates these kinds o
by hn_throwaway_99 4y ago
FWIW, when I'm doing a code review, these are the exact kind of comments that I would tell a committer to remove.
That is, it's like it generates these kinds of comments:
// initializes the variable x and sets it to 5
let x = 5;
// adds 2 to the variable x and sets that to a new variable y
let y = x + 2;
That is, IMO the whole purpose of comments should be to tell you things that aren't readily apparent just by looking at the code, e.g. "this looks wonky but we had to do it specifically to work around a bug in library X".
Perhaps could be useful for people learning to program, but otherwise people should learn how to read code as code, not "translate" it to a verbose English sentence in their head.
- aaaaaaaaaaab 4y agoYep. Nothing particularly mind-blowing about this. It's just a word-by-word translation of code into English. Heck, you don't even need GPT-3 to do this, except for some variety and grammatical correctness.
- danielvaughn 4y agoWhat I like about it is that it could help me understand a new language. Sometimes it’s easy to follow what’s going on, but sometimes there are just odd language conventions that I’m not used to.
- eyelidlessness 4y agoI don’t think the intent of this tool is to generate comments which you’d then embed into the code it describes. I think it’s meant to explain, in plain language, what the actual behavior is (for whatever confidence level you might assign to “actual” and “is”). To your point about the utility of code comments describing the behavior this way, I agree it’s probably much more valuable for beginners. In fact when I’ve mentored early programmers, I sometimes ask them to write out essentially prose like this in comments before writing a single line of executable code. Now, I’m far from a beginner. I’ve been considered a senior engineer long enough that friends discourage me from disclosing the amount of time, for fear of age discrimination. I can absolutely see the potential of this tool as part of my IDE. I’m on vacation now, but when I return to work I plan to take it for a spin as an aid for refactoring areas of code which clearly work as intended (well, for the most part) but the actual behavior and intent is much less clear. Here’s why I think it’ll be valuable for refactoring: it can help limit the amount of mental context switching necessary to build a mental model of what the code does. I often find myself trying to produce prose much like this for my own reference, but I end up losing context as fast as I acquire it as I follow references into their respective rabbit holes. Having the tool do that for me can help me stay in a single area of focus. It could also be a useful reference for adding and improving type definitions, maybe even regression tests. The best part is that it doesn’t, from what I’ve seen, do anything besides populate ephemeral annotations. It doesn’t try to write code or automate anything other than producing a narrative. Like at least one other commenter, I’m skeptical about the reliability of that. But unlike that commenter, I’m willing to take the risk… probably because I’ve learned to be skeptical of my own reliability performing the same task. If my instinct is right that I can use this tool the way I hope, I’ll still scrutinize it for accuracy. But that’s potentially much better than having only one imperfect, meat-based computer doing the work.
- khitchdee 4y agoWhy could you not create a comment block above the code section in question and fold it. All IDE's nowadays support folding, including comment block folding. To see what a section of code does, you unfold the comment block above it, read it, then fold it back.
- danielvaughn 4y agoBut if it can be generated on the fly, why persist it to the code at all?
- khitchdee 4y agoI guess if the AI is good enough, your point makes some sense. It's a good programming habit to doc your code as you write it. If you do this in a structured way and code folding is used the comments should get out of the way from the main code.
- darkwater 4y agoTo save computational resources could be an answer.
- hn_throwaway_99 4y agoTo be clear, I'm not really arguing that the intent is to embed comments. But I'm just going by the "validateSignature" example they give in the video. All that does is generate a 1-1 mapping of each code line to an English comment. TBH I would expect a senior developer to just be faster reading the code. Perhaps there are some other examples that explain more of a "structural understanding" of the code, but I'm skeptical without more evidence.
- viceroyalbean 4y agoYeah, I find the signature example kind of unimpressive because it can just pick up the meaning of the code from the function name and variables, which an experienced human can do just as easily. If it could get "validates a signature with secret key" when all the variable names are obfuscated it might be useful in some code bases.
- xwowsersx 4y agoVery much agree.
- cdubzzz 4y agoThe tweet and video don’t seem to imply this _should_ be a comment. I have been “learning to program” for 20+ years and would absolutely find this useful as a quick way to get basic information about a chunk of code I’m unfamiliar with. Not that learning to read code isn’t important, just not always necessarily worth the time (:
- daveguy 4y agoYou are grossly over-simplifying what this is doing. Nowhere does it say anything as simple as setting x to y. In nearly each case it takes the context of the variables into account and states the meaning of the function calls, not the meaning of the syntax.
- hn_throwaway_99 4y agoYes, the examples I gave are gross over-simplifications. But look at the attached video. It literally just gives a 1-1 mapping of each code line to an English sentence, and that code is pretty trivial to understand in any case. I mean $data = file_get_contents(filename: 'php://input'); to "We're getting the raw request body" I still argue that developers should be able to read the raw code like that faster than turning it into an English sentence. I'm not saying the text generation isn't extremely impressive, I just don't think it's that useful.
- nextaccountic 4y ago> I just don't think it's that useful. Even for experienced developers, it is useful when navigating a new codebase (where you don't know what each API call does) or even a new programming language (where a particular construct may be unfamiliar like Rust's if let). Well supposing the tool is accurate (incorrect results may make the tool useless or harmful) But if you're discounting the usefulness for beginners I think this is a mistake. The experienced devs of tomorrow are the beginners of today.
- wingerlang 4y agoIronically your example would be a pretty great case for someone like me, who occasionally use PHP for fun but don't really use-use it. $data = file_get_contents(filename: 'php://input'); I would definitely have to google what this means, like why is it opening a file and how is 'php://input' a file? "We're getting the raw request body" Is actually a good explanation. So, someone new to a project, and maybe unfamiliar with the tech, might find this useful.
- version_five 4y agoAnother way to look at it is that it doesnt (afaik) have externsl context, so there are severe limits on what it could possibly infer from some code. Like you say, if something unusual is being done because of a library, or business rules or something, "AI" cannot take this into account. There may be some cases where something non-obvious can be distilled out of the code, though I agree that mostly the stuff you can infer without context is mostly self evident from the code anyway Edit: just thinking, the canonical example would be something like the fast inverse square root from quake. Is it going to summarize or is it going to tell you i = 0x5f3759df - (i >> 1); // Shift i right by one and subtract it from 0x5f... (It would be cool if it does work here, even if it does, when someone makes up a new thing like this, it couldn't possibly comprehend why it's being done)
- Timpy 4y agoFortunately for me, at my current job I don't see a lot of code that makes me say "what the hell is this even trying to do?" The most useful comments are the ones explaining limitations and why this implementation was chosen, or business rules that wouldn't be apparent in the code.
- Izkata 4y agoSummarized by the common advice "comments should explain the why, not the what/how".
- dogcomplex 4y agoSeeing as how its knowledge base is pulled from similar code pulled from the internet, I'd find it very likely to get the reference and interpret that as the quake fast inverse square root. Same with general context/library stuff - though obviously that's conditional on the quality of its training data. e.g. it would probably "understand" common devops/organization/OS/library stuff even if it's not part of the language it's presumably reading - just because other users probably left comments on those lines similarly - but it's not gonna necessarily understand your application-specific business logic beyond what's actually happening. Would need some very specific examples though (by definition lol). Even dumb stuff like interpreting "make the div spin around" from some CSS/JS would probably work, as someone somewhere probably coded that similarly.
- kleiba 4y agoAgree. I find, what you usually want to understand is not the "what" or "how" but the "why", and that is quite a bit harder to automate than translating syntax into natural language statements.
- Gordonjcp 4y agoI caused a bit of an "incident" at a web dev company I worked at many many years ago by removing commit access from one of the "technical managers", whose hobby was removing every single comment he could find. "But the code shouldn't need comments," he'd complain, "it should be obvious what it does otherwise it's just bad code!" Yes, you dobber, it *is* obvious from the code, the *how* is obvious, but the why and the what might not be. The comments explain what it's doing stuff to and why, and in particular why you'd want that and why it's important. Disk space is free, so in a big long comment just write up what that particular bit of business logic is intended to achieve and what it expects as inputs and outputs. or, TL;DR "DOCSTRINGS BITCH DO YOU SPEAK IT"
- more-coffee 4y agoI bet they also pulled out the "but the code will change and then the comments become invalid, or even gasp misleading"?
- Gordonjcp 4y agoAbsolutely, and could not be convinced that if the code and comments got out of sync then it was likely that the function name would no longer make sense, and that's when we get real trouble. There were horror stories with that codebase, like index.php was set as the default error handler, started by immediately returning 200 OK before going anywhere near URL dispatching, and then displaying a page that said 404 despite returning 200 if it couldn't find the URL.
- Quekid5 4y agoYeah, the 'why' often literally doesn't exist as information in the code anywhere, e.g. business rules, domain-specific knowledge, etc. If you're extremely lucky it might exist as 'why comments' :) I do wonder if ML could be applied if you actually also train it on your particular (large) application where if might be able to 'cross-reference' domain knowledge present in commented code to other similar code. (But then you might want to just remove the pseudo-duplication anyway.)
- noufalibrahim 4y agoI completely agree with your point here. The extreme pathological case is when you see things like this i += 2 // Increments i by 1 Now you're screwed. Added bonus - the commit message introducing this change is "Yay! Fixed all the tests!!!!"
- mrtksn 4y agoI'm always amazed by what AI can do but never been amazed by the actual output(amazed within the context that this is computer generated). Not just Copilot but any tool that has too much of intelligence of itself(Dall-e, Midjourneys etc) feels like this because it reminds me of a person with great talent for compositing stuff but doesn't know what they are doing. AI generated papers that got published in prestigious journals situation all over again and again. At glance looks amazing but the machine definitely doesn't have any kind of intelligence and the output is actually worthless. This AI stuff work really well when they do something very specific that is quickly inspectable by human, for example generating interpolated frames in videos or extending a pattern or detecting anomalies kind of stuff. The moment it strays away from human control it fails amazingly well.
- rk06 4y agoMy Rule of comments: 1. Write comment first, code later. 2. Tell the intent of code, not the instructions to achieve it
- deleted 4y ago[deleted]
- alexvoda 4y agoThe only situation where i see an AI generating this type of comments to be useful would be deciphering an obfuscated C challenge program. Or maybe Perl. But otherwise, I also want the Why not the What!