4 ms·

> The study involved 47 people with varying levels of experience, including undergraduate students, graduate students, and industry professionals. Honestly, fo

> The study involved 47 people with varying levels of experience, including undergraduate students, graduate students, and industry professionals. Honestly, for things like catching potential sql injections, “industry professional” should be the start. I won’t trust a student of any kind further than I can throw them, for topics like maintainability, security or readable code. Those are too far away from what’s usually taught at school.

BlargMcLarg 4y ago
Many industry professionals ignore or are unaware of known attack vectors. Meanwhile, many schools do teach a semblance. Given how young and emotion-driven the field is, a sweeping statement like this is useless. If security is remotely concerning to you, get a security expert as your baseline.
- torben-friis 4y ago
  > Many industry professionals ignore or are unaware of known attack vectors. Meanwhile, many schools do teach a semblance. It’s not either or. Most industry professionals get there after going through school, or are at some point checked to have achieved equivalent knowledge somewhere else. A professional is school + experience.
ravenstine 4y ago
Even then, be skeptical. The software industry provides positive pressure to improve one's skills, but doesn't provide much if any negative pressure against lacking competence or willingness to learn. If developers want to, they can pretty easily skate by without developing talent, besides when their framework of choice becomes defunct.
- vsareto 4y ago
  This is true for a lot of low-brow work because companies simply need people to grind through requirements for their bespoke CRUD apps and integrations. There's still a boat load of that kind of work to be done, but it's only a fraction of the work available. There are plenty of places where that negative pressure exists though and there's higher standards of quality. I doubt anyone is skating by with hacky code in aviation or space, for instance. So what you're talking about isn't really an issue where it matters.
  - AlotOfReading 4y ago
    People absolutely produce hacky, sketchy code in aerospace and other safety-critical environments.
ykonstant 4y ago
>I won’t trust a student of any kind further than I can throw them, for topics like maintainability, security or readable code. I don't always look for files, but when I do, I `**/(#ia1)name(LK+50mw-1u0)`
jerf 4y ago
Having "industry professionals" in this sort of study actually puts it in the top tier of studies. Most studies don't even have that, they're all undergrad based. (Sometimes people express frustration that we don't pay much attention to "the science" in the programming field, and my response is generally to tell such people to take a closer look at the "science" they're trying to wave around. Studies based on "a class full of juniors in college" top out at zero value, and it's really easy for them to be negative as they can be actively wrong about how the topic under study affects professionals.) In this case, though, I'd submit that one doesn't need to run some sort of enormous study to establish the point that these code assistants are not a magic gateway to quality code at any experience level. I've been banging this drum on HN just from an understanding of how the tech works. Confabulation engines can't help but confabulate. You can't trust them. This GPT stuff is perhaps a better view into human psychology than a useful tool; we've never built an AI that so strongly emits signals of confidence before. They're the best artificial confidence game players ever.
- dmix 4y ago
  > one doesn't need to run some sort of enormous study to establish the point that these code assistants are not a magic gateway to quality code at any experience level You just have to use it a couple times to figure this out. It’s pretty obvious what the limitations are and most programmers are smart enough to understand what it is and what it isn’t. This is why I’m skeptical it will be a problem, it’s not being sold that way and after using it that will be obvious. So anyone dumb enough to treat it like that and trust the output blind probably wasn’t a good programmer before. And if they keep doing it they don’t have proper incentive structures to not produce buggy code (senior devs, bosses, customers etc will notice that output is getting worse at a minimum when the product breaks or doing QA).
- marcosdumay 4y ago
  > This GPT stuff is perhaps a better view into human psychology than a useful tool It would be a great search engine if it cited its sources (but then people would notice it's basically copying code from the internet). It is actually good at getting you the names that come in a certain context, so you can search them. But only if you know enough to establish the context.

BlargMcLarg 4y ago

Many industry professionals ignore or are unaware of known attack vectors. Meanwhile, many schools do teach a semblance. Given how young and emotion-driven the field is, a sweeping statement like this is useless. If security is remotely concerning to you, get a security expert as your baseline.

torben-friis 4y ago
> Many industry professionals ignore or are unaware of known attack vectors. Meanwhile, many schools do teach a semblance. It’s not either or. Most industry professionals get there after going through school, or are at some point checked to have achieved equivalent knowledge somewhere else. A professional is school + experience.

ravenstine 4y ago

Even then, be skeptical. The software industry provides positive pressure to improve one's skills, but doesn't provide much if any negative pressure against lacking competence or willingness to learn. If developers want to, they can pretty easily skate by without developing talent, besides when their framework of choice becomes defunct.

vsareto 4y ago
This is true for a lot of low-brow work because companies simply need people to grind through requirements for their bespoke CRUD apps and integrations. There's still a boat load of that kind of work to be done, but it's only a fraction of the work available. There are plenty of places where that negative pressure exists though and there's higher standards of quality. I doubt anyone is skating by with hacky code in aviation or space, for instance. So what you're talking about isn't really an issue where it matters.
- AlotOfReading 4y ago
  People absolutely produce hacky, sketchy code in aerospace and other safety-critical environments.

ykonstant 4y ago

>I won’t trust a student of any kind further than I can throw them, for topics like maintainability, security or readable code. I don't always look for files, but when I do, I `**/(#ia1)name(LK+50mw-1u0)`

jerf 4y ago

Having "industry professionals" in this sort of study actually puts it in the top tier of studies. Most studies don't even have that, they're all undergrad based. (Sometimes people express frustration that we don't pay much attention to "the science" in the programming field, and my response is generally to tell such people to take a closer look at the "science" they're trying to wave around. Studies based on "a class full of juniors in college" top out at zero value, and it's really easy for them to be negative as they can be actively wrong about how the topic under study affects professionals.) In this case, though, I'd submit that one doesn't need to run some sort of enormous study to establish the point that these code assistants are not a magic gateway to quality code at any experience level. I've been banging this drum on HN just from an understanding of how the tech works. Confabulation engines can't help but confabulate. You can't trust them. This GPT stuff is perhaps a better view into human psychology than a useful tool; we've never built an AI that so strongly emits signals of confidence before. They're the best artificial confidence game players ever.

dmix 4y ago
> one doesn't need to run some sort of enormous study to establish the point that these code assistants are not a magic gateway to quality code at any experience level You just have to use it a couple times to figure this out. It’s pretty obvious what the limitations are and most programmers are smart enough to understand what it is and what it isn’t. This is why I’m skeptical it will be a problem, it’s not being sold that way and after using it that will be obvious. So anyone dumb enough to treat it like that and trust the output blind probably wasn’t a good programmer before. And if they keep doing it they don’t have proper incentive structures to not produce buggy code (senior devs, bosses, customers etc will notice that output is getting worse at a minimum when the product breaks or doing QA).
marcosdumay 4y ago
> This GPT stuff is perhaps a better view into human psychology than a useful tool It would be a great search engine if it cited its sources (but then people would notice it's basically copying code from the internet). It is actually good at getting you the names that come in a certain context, so you can search them. But only if you know enough to establish the context.