7 ms·

This article is so misleading.... when someone talks about codegen, it usually means AOP, bytecode manipulation, compiler plugins and the sorts. Not "AI tools".

This article is so misleading.... when someone talks about codegen, it usually means AOP, bytecode manipulation, compiler plugins and the sorts. Not "AI tools". What is even worse is that this isn't even a proper article, this is just a thinly concealed advertisement.

diogenes4 3y ago
> This article is so misleading.... when someone talks about codegen, it usually means AOP, bytecode manipulation, compiler plugins and the sorts. Not "AI tools". This is an odd take to me when LLMs are very, very good at generating code, which has got a lot of attention recently. Sure it may be a different beast from what we've currently identified as "codegen" but it remains a descriptive term for the code-generating technology.
- ynx 3y ago
  Words mean things, especially when communicating in professional or semi-professional settings with terminology with longstanding meaning. To not do so is poor communication, and irresponsible depending on the stakes. Codegen's long history of macro-fied (or similar kind of scripting) source writing matters here. It has the very important property of having consistent/deterministic output from a process that can be verified with very high confidence by rudimentary human inspection. It's like calling autocomplete "codegen". I truly love LLM-assisted coding. I would never call it codegen, and think it can even be unethical to do so when the stakes are high, because it gives it the veneer of trustworthiness that lends one to carelessly not audit it. LLM-aided code writing should, IMO, be called "code assist", not "codegen".
  - pvg 3y ago
    I think this is more a matter of familiarity and habit - notice how there are several slightly different definitions of exactly what 'codegen' means in this thread alone. It doesn't really have some super-specific meaning, it's not, dunno 'lexical scoping'. It's still just shorthand for 'code generation' and it's not unreasonable to apply it to, well, most automated processes that generate code.
    - ynx 3y ago
      I don't buy a nihilistic "does anything mean anything anymore?" approach. I'm basing my evaluation on Copilot. It functions as a highly context-sensitive and very useful autocomplete, so the "autocomplete" label is a complete, accurate, and precise description for it. It does not fill the same role as writing repeatable macros, batch scripts in part of the build process, token manipulation, or joining a tabular dataset with templates...all components associated with traditional code generation. In the most strict pedantic sense, yes, LLMs create lines code via their internal processes, and that could be called "code generation" at a technical level. But they serve different needs, with different techniques, and different interfaces. In terms of common parlance, no PM or manager who is not pants-on-head dumb is going to suggest replacing scripts that generate code 1:1 with LLM output, unless the terminology itself has confused them into thinking they fill the same role. Poor communicators, whether because they are overly pedantic or undertrained, may make this mistake. But in a team environment, if "codegen" is considered a valid name for LLM output, an effective communicator is always going to need to clarify which they mean, because the tasks are not interchangeable in the least.
      - pvg 3y ago
        My argument is that it's not a precise technical term of art. There are many of those but this isn't one.The evidence you can easily see in thread and in pretty wide usage on the internet. Your argument is 'this is nihilism' which is mostly vibes and grumpy vibes at that.
- SomeCallMeTim 3y ago
  > LLMs are very, very good at generating code Ummm.... Awful code that often looks right at first glance, maybe. Maybe LLMs can generate the kind of code that's really shallow in its complexity, but for literally everything I would call interesting LLMs have produced hot garbage. From "it doesn't quite do what I want" to "it couldn't possibly work and it's extremely far from being sane," though it always looks reasonable.
  - diogenes4 3y ago
    > Ummm.... Awful code that often looks right at first glance, maybe. > Maybe LLMs can generate the kind of code that's really shallow in its complexity, but for literally everything I would call interesting LLMs have produced hot garbage. From "it doesn't quite do what I want" to "it couldn't possibly work and it's extremely far from being sane," though it always looks reasonable. None of this has any bearing.
verdverm 3y ago
This is an outdated perspective, most people think differently now (and very little of longer established code gen tasks)
mlinsey 3y ago
> when someone talks about codegen, it usually means AOP, bytecode manipulation, compiler plugins and the sorts. Not "AI tools" I have found the opposite to be true, starting this year. You can commiserate with people who are interested in cryptography.
- ReactiveJelly 3y ago
  Every year I commiserate harder - "Dexter" is no longer "Dexter's Lab" (It's Darkly Dreaming Dexter) - "Dark Descent" is no longer "Amnesia" (It's some other game) - "Server" no longer means "server" (It means a Discord guild) - "Serverless" now means "On a server" - "Operating system" sometimes means "web app" - "Powered by" has nothing to do with power And Internet speeds are still measured in bits, in case anyone is not running the latest System/360
  - fragmede 3y ago
    Oh, and clouds are now where other people's computers live, and not full of rain and lightning, both of which aren't friends to computers. Jason isn't my friend anymore, but a data interchange format. Neither is Kate, she's how I orchestrate my containers. Ghosting doesn't involve poltergeists or a Halloween, Zoom isn't a function on my camera, and fishing steals my data and doesn't get me delicious salmon.
  - bigstrat2003 3y ago
    Don't forget "literally" which people have rendered useless by using it to mean "figuratively". And even the dictionary allowed that misuse of the language. :(
    - krapp 3y ago
      Literally has always meant figuratively[0]. That is not a misuse of the language, that is how the language is used and has been used since at least 1769. [0]https://blogs.illinois.edu/view/25/96439 https://blogs.illinois.edu/view/25/96439
leoqa 3y ago
I’m in between: I think of protoc and grpc bindings, or old school Go generics.
- chrsig 3y ago
  Without additional context, this is my default understanding. If it were in a compiler development conversation, I'd think of outputted machine code.
- mikepurvis 3y ago
  The same— I'm in a long-time world of generated message headers for ROS, so "codegen" to me is that add_custom_command/add_custom_target dance you do with CMake to get it to run a non-compiler tool ahead of actual compilation.
robertlagrant 3y ago
> when someone talks about codegen, it usually means AOP, bytecode manipulation, compiler plugins and the sorts This feels like a JVM-land interpretation? I think it's quite domain-specific.
- Pannoniae 3y ago
  Yeah, sorry about that. If I was more familiar with native codegen tools I would have brought examples from them as well. Hopefully I still got my point across.
  - robertlagrant 3y ago
    No worries - all I mean is that you, from Java domain, interpret the word one way. Different domains interpret it very differently.
- nightpool 3y ago
  The examples GP gave were java-specific, but the concept of codegen meaning "deterministic generation of code in a 'compile' phase" is universal—think of OpenAPI libraries, Go pre-generics, Babel, etc.
- ReactiveJelly 3y ago
  Not just JVM. Rust also has a codegen step in its compiler pipeline https://doc.rust-lang.org/rustc/codegen-options/index.html https://doc.rust-lang.org/rustc/codegen-options/index.html
imachine1980_ 3y ago
i don't, not at least this year, but being figma i was thinking it was about asm.js codegen or webassembly code generations.
eyelidlessness 3y ago
Yeah, I was really confused by the article at first. I understand that the term must have evolved, but I hadn’t heard/seen it used this way until now. I’m already skeptical of codegen in the more conventional sense, and definitely skeptical of AI code generation… so I guess now I can carry on as usual, with a slightly elevated “I’m probably not the target audience” filter?
deleted 3y ago
[deleted]
bafe 3y ago
I opened the link expecting an article criticising annotation processors or macros, not a vague writeup on LLM/AI code tools

diogenes4 3y ago

> This article is so misleading.... when someone talks about codegen, it usually means AOP, bytecode manipulation, compiler plugins and the sorts. Not "AI tools". This is an odd take to me when LLMs are very, very good at generating code, which has got a lot of attention recently. Sure it may be a different beast from what we've currently identified as "codegen" but it remains a descriptive term for the code-generating technology.

ynx 3y ago
Words mean things, especially when communicating in professional or semi-professional settings with terminology with longstanding meaning. To not do so is poor communication, and irresponsible depending on the stakes. Codegen's long history of macro-fied (or similar kind of scripting) source writing matters here. It has the very important property of having consistent/deterministic output from a process that can be verified with very high confidence by rudimentary human inspection. It's like calling autocomplete "codegen". I truly love LLM-assisted coding. I would never call it codegen, and think it can even be unethical to do so when the stakes are high, because it gives it the veneer of trustworthiness that lends one to carelessly not audit it. LLM-aided code writing should, IMO, be called "code assist", not "codegen".
- pvg 3y ago
  I think this is more a matter of familiarity and habit - notice how there are several slightly different definitions of exactly what 'codegen' means in this thread alone. It doesn't really have some super-specific meaning, it's not, dunno 'lexical scoping'. It's still just shorthand for 'code generation' and it's not unreasonable to apply it to, well, most automated processes that generate code.
  - ynx 3y ago
    I don't buy a nihilistic "does anything mean anything anymore?" approach. I'm basing my evaluation on Copilot. It functions as a highly context-sensitive and very useful autocomplete, so the "autocomplete" label is a complete, accurate, and precise description for it. It does not fill the same role as writing repeatable macros, batch scripts in part of the build process, token manipulation, or joining a tabular dataset with templates...all components associated with traditional code generation. In the most strict pedantic sense, yes, LLMs create lines code via their internal processes, and that could be called "code generation" at a technical level. But they serve different needs, with different techniques, and different interfaces. In terms of common parlance, no PM or manager who is not pants-on-head dumb is going to suggest replacing scripts that generate code 1:1 with LLM output, unless the terminology itself has confused them into thinking they fill the same role. Poor communicators, whether because they are overly pedantic or undertrained, may make this mistake. But in a team environment, if "codegen" is considered a valid name for LLM output, an effective communicator is always going to need to clarify which they mean, because the tasks are not interchangeable in the least.
    - pvg 3y ago
      My argument is that it's not a precise technical term of art. There are many of those but this isn't one.The evidence you can easily see in thread and in pretty wide usage on the internet. Your argument is 'this is nihilism' which is mostly vibes and grumpy vibes at that.
SomeCallMeTim 3y ago
> LLMs are very, very good at generating code Ummm.... Awful code that often looks right at first glance, maybe. Maybe LLMs can generate the kind of code that's really shallow in its complexity, but for literally everything I would call interesting LLMs have produced hot garbage. From "it doesn't quite do what I want" to "it couldn't possibly work and it's extremely far from being sane," though it always looks reasonable.
- diogenes4 3y ago
  > Ummm.... Awful code that often looks right at first glance, maybe. > Maybe LLMs can generate the kind of code that's really shallow in its complexity, but for literally everything I would call interesting LLMs have produced hot garbage. From "it doesn't quite do what I want" to "it couldn't possibly work and it's extremely far from being sane," though it always looks reasonable. None of this has any bearing.

verdverm 3y ago

This is an outdated perspective, most people think differently now (and very little of longer established code gen tasks)

mlinsey 3y ago

> when someone talks about codegen, it usually means AOP, bytecode manipulation, compiler plugins and the sorts. Not "AI tools" I have found the opposite to be true, starting this year. You can commiserate with people who are interested in cryptography.

ReactiveJelly 3y ago
Every year I commiserate harder - "Dexter" is no longer "Dexter's Lab" (It's Darkly Dreaming Dexter) - "Dark Descent" is no longer "Amnesia" (It's some other game) - "Server" no longer means "server" (It means a Discord guild) - "Serverless" now means "On a server" - "Operating system" sometimes means "web app" - "Powered by" has nothing to do with power And Internet speeds are still measured in bits, in case anyone is not running the latest System/360
- fragmede 3y ago
  Oh, and clouds are now where other people's computers live, and not full of rain and lightning, both of which aren't friends to computers. Jason isn't my friend anymore, but a data interchange format. Neither is Kate, she's how I orchestrate my containers. Ghosting doesn't involve poltergeists or a Halloween, Zoom isn't a function on my camera, and fishing steals my data and doesn't get me delicious salmon.
- bigstrat2003 3y ago
  Don't forget "literally" which people have rendered useless by using it to mean "figuratively". And even the dictionary allowed that misuse of the language. :(
  - krapp 3y ago
    Literally has always meant figuratively[0]. That is not a misuse of the language, that is how the language is used and has been used since at least 1769. [0]https://blogs.illinois.edu/view/25/96439 https://blogs.illinois.edu/view/25/96439

leoqa 3y ago

I’m in between: I think of protoc and grpc bindings, or old school Go generics.

chrsig 3y ago
Without additional context, this is my default understanding. If it were in a compiler development conversation, I'd think of outputted machine code.
mikepurvis 3y ago
The same— I'm in a long-time world of generated message headers for ROS, so "codegen" to me is that add_custom_command/add_custom_target dance you do with CMake to get it to run a non-compiler tool ahead of actual compilation.

robertlagrant 3y ago

> when someone talks about codegen, it usually means AOP, bytecode manipulation, compiler plugins and the sorts This feels like a JVM-land interpretation? I think it's quite domain-specific.

Pannoniae 3y ago
Yeah, sorry about that. If I was more familiar with native codegen tools I would have brought examples from them as well. Hopefully I still got my point across.
- robertlagrant 3y ago
  No worries - all I mean is that you, from Java domain, interpret the word one way. Different domains interpret it very differently.
nightpool 3y ago
The examples GP gave were java-specific, but the concept of codegen meaning "deterministic generation of code in a 'compile' phase" is universal—think of OpenAPI libraries, Go pre-generics, Babel, etc.
ReactiveJelly 3y ago
Not just JVM. Rust also has a codegen step in its compiler pipeline https://doc.rust-lang.org/rustc/codegen-options/index.html https://doc.rust-lang.org/rustc/codegen-options/index.html

imachine1980_ 3y ago

i don't, not at least this year, but being figma i was thinking it was about asm.js codegen or webassembly code generations.

eyelidlessness 3y ago

Yeah, I was really confused by the article at first. I understand that the term must have evolved, but I hadn’t heard/seen it used this way until now. I’m already skeptical of codegen in the more conventional sense, and definitely skeptical of AI code generation… so I guess now I can carry on as usual, with a slightly elevated “I’m probably not the target audience” filter?

deleted 3y ago

[deleted]

bafe 3y ago

I opened the link expecting an article criticising annotation processors or macros, not a vague writeup on LLM/AI code tools