4 ms·

This is difficult to express because I too have enjoyed using an LLM lately and have felt a productivity increase, but I think there is a false sense of securit

by snoman 1y ago

This is difficult to express because I too have enjoyed using an LLM lately and have felt a productivity increase, but I think there is a false sense of security being expressed in your writing and it underlies one of the primary risks I see with LLMs for programming. With minor exceptions, moving from one language to another isn’t a matter of simple syntax, trivia, or swapping standard libraries. Certainly, expert beginners do espouse that all the time, but languages often have fundamental concepts that they’re built that and need to be understood in order to be effective with them. For example: have you ever seen a team move from Java to Scala, js to Java, or C# to Python - all of which I’ve seen - where the developers didn’t try to understand language they were moving to? Non-fail, they tried to force the concepts that were important to their prior language, onto the new one, to abysmal results. If you’re writing trivial scripts, or one-off utils, it probably doesn’t build up enough to matter, and feels great, but you don’t know what you don’t know, and you don’t know what to look for. Offloading the understanding of the concepts that are important for a language to an LLM is a recipe for a bad time.

nojito 1y ago
How about converting a rust library into Haskell? https://ghuntley.com/oh-fuck/ https://ghuntley.com/oh-fuck/ That's from 6 months ago and the tooling today is almost night and day in terms of improvements.
- fleebee 1y ago
  Where can I see the code? Also: > This wasn't something that existed; it wasn't regurgitating knowledge from Stackoverflow. It was inventing/creating something new. Didn't he expressly ask the LLM to copy an established, documented library?
  - simonw 1y ago
    Yes, but in another language. It wasn't regurgitating Haskell code it had seen before.
- snzixjxjxjsn 1y ago
  Translation is the one thing everyone expects AI to be good at. And while I’m not an expert in above posts languages so I can’t review, I’d be willing to bet there’s not obvious mistakes that could end up being pretty significant. The same thing happens with language I’m an expert it. It’s odd the post describes what it’s doing as creating something new - that’s only true in the most literal (intellectually dishonest) sense.
- amelius 1y ago
  It really makes me wonder what is taking us so long to translate all these C python modules like Numpy and SciPy into something that works with one of the GIL-free python variants out there.
  - nojito 1y ago
    I have started pulling all my backlog projects of converting my feature complete python code over to rust and with Claude it's been absolutely phenomenal.
simonw 1y ago
> but languages often have fundamental concepts that they’re built that and need to be understood in order to be effective with them I completely agree. That's another reason I don't feel threatened by non-programmers using LLMs: to actually write useful Go code you need to figure out goroutines, for React you need to understand the state model and hooks, for AppleScript you need to understand how apps expose their features, etc etc etc. All of these are things you need to figure out, and I would argue they are conceptually more complex than for loops etc. But... don't need to memorize the details. I find understanding concepts like goroutines to be a very different mental activity to memorizing the syntax for a Go for loop. I can come back to some Go code a year later and remind myself how goroutines work very quickly, because I'm an experienced software engineer with a wealth of related knowledge about concurrency primitives to help me out.
- pyman 1y ago
  I agree Simon. With the little time I have these days for side projects, LLMs have become my new best friends. I'm more worried about the future of my students than my own. I'm sure I'll grow old gracefully alongside the machines that helped me graduate and build a career. I run Python workshops on weekends, and honestly, I'm speechless at the things students are already doing. But is it realistic to encourage them to study computer science in a year or two, when it's still a 4-year degree? By the time they start uni, LLMs will be 100 times more powerful than they are today. And in 6 years? Any advice?
- danw1979 1y ago
  What are your thoughts on humans learning from LLM output ? I’ve been encouraging the new developers I work with to ensure they read the docs and learn the language to ensure the LLM doesn’t become a crutch, but rather a bicycle. But it occurred to me recently that I probably learned most of what I know from examples of code written by others. I’m certain my less experienced colleagues are doing the same, but from Claude rather than Stack Overflow…
  - skydhash 1y ago
    Not Simon, but here is my take. Code are practical realizations of concepts that exists outside of code. Let's take concurrency as an example. It's something that is common across many domains where 2 (or more) independent actors suddenly need to share a single resource that can't be used by both at the same time. In many cases, it can be resolved (or not) in an ad-hoc manner. And sometimes there's a basic agreement that takes place. But for computers, you need the solution to be precise, eliminating all error cases. So we go one to define primitives and how these interacts with each other and the sequence of the actions. These are still not code. Once we are precise enough, we translate it to code. So if you're learning from code, you are tracing back to the original concept. And we can do so because we are good at grasping patterns. But generated code are often a factor of several patterns at once, and some are not even relevant. Just that in the training phases, the algorithm detected similarity (and we know things can be similar but are actually very different). So I'm wary of learning from LLMs, because it's a land of mirages.
exe34 1y ago
You can write Fortran in any language!
sealeck 1y ago
> With minor exceptions, moving from one language to another isn’t a matter of simple syntax, trivia, or swapping standard libraries. I think this is not true for most programming languages. Most of the ones we use today have converged around an imperative model with fairly similar functionality. Converting from one to another can often be done programmatically!
- kiitos 1y ago
  I mean, absolutely and obviously not, right? Like, languages aren't just different ways to express the same underlying concepts? It's clearly not the case that you can just transliterate a program in language X to language Y and expect to get the same behavior, functionality, etc. in the end?
  - sealeck 1y ago
    > It's clearly not the case that you can just transliterate a program in language X to language Y and expect to get the same behavior, functionality, etc. in the end? You most definitely can; if you have Turing complete languages A and B, you can implement an interpreter for A in B and B in A; you can implement a simple X86/aarch64 interpreter in either A or B and use that to run compiled/interpreted programs in the other; you can also write higher-level routines that map to language constructs in one or the other to convert between the two.
- snoman 1y ago
  The question isn’t whether there exists a function that can translate one language to another. If they’re Turing complete, it’s certainly the case that the same software CAN be written in different languages. The question is whether or not it would produce idiomatic code in the target language and whether or not you/the developer would recognize it as (not) idiomatic when it did. You can’t tell me that precisely the same paradigms, mechanics, patterns and practices will be used to develop software in both Java/Spring and Ruby/Rails, for example. The way you need to (be able to) think about how you write the software using those two languages will be non-trivially different.
diggan 1y ago
> moving from one language to another isn’t a matter of simple syntax, trivia, or swapping standard libraries. [...] Java to Scala, js to Java, or C# to Python I find it kind of funny (or maybe sad?) that you say that, then your examples are all languages that basically offer the same features and semantics, but with slightly different syntax. They're all Algol/C-style languages. I'd understand moving from Java to/from Haskell can be a bit tricky since they're actually very different from each other, but C# to/from Java? Those languages are more similar than they are different, and the biggest changes is just names and trivia basically, not the concepts themselves.
Tokumei-no-hito 1y ago
you have a point, "idiomatic" use of the language. what's unclear is whether idiomatic application of a language is a matter of elegance or functionality and is that idiomatic usage something captured by the training. some elements are necessities to the language while others are more for the human / team.

by snoman 1y ago

nojito 1y ago

How about converting a rust library into Haskell? https://ghuntley.com/oh-fuck/ https://ghuntley.com/oh-fuck/ That's from 6 months ago and the tooling today is almost night and day in terms of improvements.

fleebee 1y ago
Where can I see the code? Also: > This wasn't something that existed; it wasn't regurgitating knowledge from Stackoverflow. It was inventing/creating something new. Didn't he expressly ask the LLM to copy an established, documented library?
- simonw 1y ago
  Yes, but in another language. It wasn't regurgitating Haskell code it had seen before.
snzixjxjxjsn 1y ago
Translation is the one thing everyone expects AI to be good at. And while I’m not an expert in above posts languages so I can’t review, I’d be willing to bet there’s not obvious mistakes that could end up being pretty significant. The same thing happens with language I’m an expert it. It’s odd the post describes what it’s doing as creating something new - that’s only true in the most literal (intellectually dishonest) sense.
amelius 1y ago
It really makes me wonder what is taking us so long to translate all these C python modules like Numpy and SciPy into something that works with one of the GIL-free python variants out there.
- nojito 1y ago
  I have started pulling all my backlog projects of converting my feature complete python code over to rust and with Claude it's been absolutely phenomenal.

simonw 1y ago

> but languages often have fundamental concepts that they’re built that and need to be understood in order to be effective with them I completely agree. That's another reason I don't feel threatened by non-programmers using LLMs: to actually write useful Go code you need to figure out goroutines, for React you need to understand the state model and hooks, for AppleScript you need to understand how apps expose their features, etc etc etc. All of these are things you need to figure out, and I would argue they are conceptually more complex than for loops etc. But... don't need to memorize the details. I find understanding concepts like goroutines to be a very different mental activity to memorizing the syntax for a Go for loop. I can come back to some Go code a year later and remind myself how goroutines work very quickly, because I'm an experienced software engineer with a wealth of related knowledge about concurrency primitives to help me out.

pyman 1y ago
I agree Simon. With the little time I have these days for side projects, LLMs have become my new best friends. I'm more worried about the future of my students than my own. I'm sure I'll grow old gracefully alongside the machines that helped me graduate and build a career. I run Python workshops on weekends, and honestly, I'm speechless at the things students are already doing. But is it realistic to encourage them to study computer science in a year or two, when it's still a 4-year degree? By the time they start uni, LLMs will be 100 times more powerful than they are today. And in 6 years? Any advice?
danw1979 1y ago
What are your thoughts on humans learning from LLM output ? I’ve been encouraging the new developers I work with to ensure they read the docs and learn the language to ensure the LLM doesn’t become a crutch, but rather a bicycle. But it occurred to me recently that I probably learned most of what I know from examples of code written by others. I’m certain my less experienced colleagues are doing the same, but from Claude rather than Stack Overflow…
- skydhash 1y ago
  Not Simon, but here is my take. Code are practical realizations of concepts that exists outside of code. Let's take concurrency as an example. It's something that is common across many domains where 2 (or more) independent actors suddenly need to share a single resource that can't be used by both at the same time. In many cases, it can be resolved (or not) in an ad-hoc manner. And sometimes there's a basic agreement that takes place. But for computers, you need the solution to be precise, eliminating all error cases. So we go one to define primitives and how these interacts with each other and the sequence of the actions. These are still not code. Once we are precise enough, we translate it to code. So if you're learning from code, you are tracing back to the original concept. And we can do so because we are good at grasping patterns. But generated code are often a factor of several patterns at once, and some are not even relevant. Just that in the training phases, the algorithm detected similarity (and we know things can be similar but are actually very different). So I'm wary of learning from LLMs, because it's a land of mirages.

exe34 1y ago

You can write Fortran in any language!

sealeck 1y ago

> With minor exceptions, moving from one language to another isn’t a matter of simple syntax, trivia, or swapping standard libraries. I think this is not true for most programming languages. Most of the ones we use today have converged around an imperative model with fairly similar functionality. Converting from one to another can often be done programmatically!

kiitos 1y ago
I mean, absolutely and obviously not, right? Like, languages aren't just different ways to express the same underlying concepts? It's clearly not the case that you can just transliterate a program in language X to language Y and expect to get the same behavior, functionality, etc. in the end?
- sealeck 1y ago
  > It's clearly not the case that you can just transliterate a program in language X to language Y and expect to get the same behavior, functionality, etc. in the end? You most definitely can; if you have Turing complete languages A and B, you can implement an interpreter for A in B and B in A; you can implement a simple X86/aarch64 interpreter in either A or B and use that to run compiled/interpreted programs in the other; you can also write higher-level routines that map to language constructs in one or the other to convert between the two.
snoman 1y ago
The question isn’t whether there exists a function that can translate one language to another. If they’re Turing complete, it’s certainly the case that the same software CAN be written in different languages. The question is whether or not it would produce idiomatic code in the target language and whether or not you/the developer would recognize it as (not) idiomatic when it did. You can’t tell me that precisely the same paradigms, mechanics, patterns and practices will be used to develop software in both Java/Spring and Ruby/Rails, for example. The way you need to (be able to) think about how you write the software using those two languages will be non-trivially different.

diggan 1y ago

> moving from one language to another isn’t a matter of simple syntax, trivia, or swapping standard libraries. [...] Java to Scala, js to Java, or C# to Python I find it kind of funny (or maybe sad?) that you say that, then your examples are all languages that basically offer the same features and semantics, but with slightly different syntax. They're all Algol/C-style languages. I'd understand moving from Java to/from Haskell can be a bit tricky since they're actually very different from each other, but C# to/from Java? Those languages are more similar than they are different, and the biggest changes is just names and trivia basically, not the concepts themselves.

Tokumei-no-hito 1y ago

you have a point, "idiomatic" use of the language. what's unclear is whether idiomatic application of a language is a matter of elegance or functionality and is that idiomatic usage something captured by the training. some elements are necessities to the language while others are more for the human / team.