4 ms·

All articles like this that end with “Ha! My job is safe!” are drawing the completely wrong conclusion. It’s amazing to me that all those incredibly smart engi

All articles like this that end with “Ha! My job is safe!” are drawing the completely wrong conclusion. It’s amazing to me that all those incredibly smart engineers just can’t see the potential of what’s coming within the set of limitations it currently has. Amazes me even more when those same engineers don’t know how to best talk to GPT. It’s like watching them put into google search box “Good morning google, please I have a question, if you have a moment, could you tell me where I lost my keys because I can’t find them”. And when google is like “wtf?”, they claim “you can’t call it search if it can’t find my keys”. Very arbitrary, blind to limitations, and dismissive of the incredible potential it has as-is. But while they’re off randomly complaining, some of us are building startups…

jchw 4y ago
I think it's a joke. Though, the truth is, having AI write security sensitive code may actually genuinely be "AGI hard". AI breakthroughs are impressive, but I don't think we really know how to prevent the "confident bullshit" problem yet, and it's not clear it will be solved in this current cycle of progress. In many cases this might be an OK problem to deal with manually, but for security code it's probably not worth trying to. So the author might not have a threat to their job security, anyway.
- andrewflnr 4y ago
  I think writing/understanding security specification/policy is probably AGI-hard. The notion of security is (at least in parts) pretty deeply anchored in human experience and desires.
wkat4242 4y ago
To me what stands out is not what it gets wrong but how much it gets right. We use a not super common proprietary system at work which uses a custom query language. I can ask it "In system X, could you give me a query that finds XYZ". And it not only knows what system I'm talking about, it actually gives me a working query that would take me a couple hours to figure out and that's after I've had a few days' worth of training on it. Not to mention a huge IT background. Imagine picking a non-IT person off the street and getting them up to the point of being able to do this. They'd need weeks of training in basic IT concepts to even understand what you're talking about. I find it hard to overstate how amazing it is that a generic automated system can do this. And ChatGPT is not a one-trick pony, IT isn't the only field it knows about. Imagine being an expert on all sub-fields of IT, law, marketing, medical, etc and being able to combine all that knowledge from those fields. A human will never be able to do that. The potential is huge but like any source of information you have to verify. But what really is scary is extrapolating this to the next step forward. And the one after that. Because from what we had before to ChatGPT has been a massive step. I think the next step would be for it to learn and re-evaluate its model every time it gets corrected by someone (like in the linked article where it apologizes for the incorrect info). Right now it's a static model only, so the next time someone asks the original question it will give the wrong answer again. Once it can learn from this it'll improve a lot. The big hazard there is of course people manipulating it with false information. I don't have all the answers and I'm not an AI researcher but I'm very much amazed at the progress so far. And a bit scared. If this tech keeps evolving at this speed, job stability is only a tiny drop in a tidal wave of change. But there's no point in even trying to stop it.
- hutzlibu 4y ago
  "Imagine being an expert on all sub-fields of IT, law, marketing, medical, etc and being able to combine all that knowledge from those fields. A human will never be able to do that. The potential is huge but like any source of information you have to verify." ChatGPT is not an expert in any of these fields, it just has an enormous detail knowledge, but no understanding. A real expert actually understands his field and gives answers that makes sense and clearly states if he does not know something. ChatGPT always gives authorative answers and sometimes they are right. It is a advanced tool, but no competent human is about to be replaced by it anytime soon.
  - wkat4242 4y ago
    I would beg to differ. It can combine information from various sources and make it into a working script or program. I've seen it do that various times with questions much to obscure to have found 1:1 on google. This is not a talking search engine, it's really something much more advanced.
bombela 4y ago
It understands you in any languages, speaks really well any languages. And of course, converses. Like, no big deal. I remember the AI bots of a decade ago. Nothing compares. Skynet is coming.
- 5e92cb50239222b 4y ago
  What languages did you try? I speak Russian fluently and ChatGPT isn't very good at it. It frequently produces nonsensical answers, uses wrong inflections all the time, writes songs and poems that don't rhyme, and its blocks can easily be circumvented to praise Hitler or do anything else. It also writes text at a noticeably slower rate (at least 2× compared to English, maybe more).
  - cypress66 4y ago
    I've asked it to solve certain coding tasks in both English and Spanish and it seems about equally competent in both.
  - Shorel 4y ago
    It's good at Spanish, I did not notice any grammatical mistakes or nonsensical answers caused by the language.
  - bombela 4y ago
    English and French which I both speak/write fluently. I assumed; wrongly; that it must have been good equality with other languages. The poems in French didn't rhyme. I haven't tried poems in English.
zmachinaz 4y ago
One could equally say that mainly the mediocre people are excited, while the smart people actually see the limitations. For people doing non-trivial things, the output of ChatGPT seems currently rather far away from being of any deep value. To replace some mundane tasks, yes, but mundane tasks are anyway just for the mediocre.
- vasco 4y ago
  "The mediocre people" usually are the ones that think there's "mediocre" and "smart" buckets for people.
- williamcotton 4y ago
  > but mundane tasks are anyway just for the mediocre. So when you’re working on something you just let other people take care of the inevitable mundane tasks that arise in daily knowledge work? I’m guessing you’ve never emptied the garbage can at the office either as that would be unworthy of your eminent intellect.
loveparade 4y ago
I think it's just as likely to be the other way around. AI has been doing these hype cycles for for 30+ years now and every time it's "this is amazing and will take people's jobs" - But what always happens is not much change in the short term, and rather subtle changes in the long term. People underestimate how important the edge cases these models aren't good at are. I also think ChatGPT is amazing, but historically it's not any more amazing that many of the previous AI-related developments for their time, none of which have quite lived up to the initial promise after the hype died. Just look at some of the robots from the 80s [0] that were expected to be in every household. And when chess was solved, of course we were just a few years away from AGI. In 20 years, people will look at ChatGPT and make fun of how cute it was, and we'll still be just a few years away from AGI. People aren't blind, they are just realistic. Yes, current AI models will slowly be integrated into services and bring changes behind the scenes, but it's not going to be the big explosion of change a lot of people expect it to be. [0] https://www.youtube.com/watch?v=jkOctWWsj-A https://www.youtube.com/watch?v=jkOctWWsj-A

jchw 4y ago

I think it's a joke. Though, the truth is, having AI write security sensitive code may actually genuinely be "AGI hard". AI breakthroughs are impressive, but I don't think we really know how to prevent the "confident bullshit" problem yet, and it's not clear it will be solved in this current cycle of progress. In many cases this might be an OK problem to deal with manually, but for security code it's probably not worth trying to. So the author might not have a threat to their job security, anyway.

andrewflnr 4y ago
I think writing/understanding security specification/policy is probably AGI-hard. The notion of security is (at least in parts) pretty deeply anchored in human experience and desires.

wkat4242 4y ago

To me what stands out is not what it gets wrong but how much it gets right. We use a not super common proprietary system at work which uses a custom query language. I can ask it "In system X, could you give me a query that finds XYZ". And it not only knows what system I'm talking about, it actually gives me a working query that would take me a couple hours to figure out and that's after I've had a few days' worth of training on it. Not to mention a huge IT background. Imagine picking a non-IT person off the street and getting them up to the point of being able to do this. They'd need weeks of training in basic IT concepts to even understand what you're talking about. I find it hard to overstate how amazing it is that a generic automated system can do this. And ChatGPT is not a one-trick pony, IT isn't the only field it knows about. Imagine being an expert on all sub-fields of IT, law, marketing, medical, etc and being able to combine all that knowledge from those fields. A human will never be able to do that. The potential is huge but like any source of information you have to verify. But what really is scary is extrapolating this to the next step forward. And the one after that. Because from what we had before to ChatGPT has been a massive step. I think the next step would be for it to learn and re-evaluate its model every time it gets corrected by someone (like in the linked article where it apologizes for the incorrect info). Right now it's a static model only, so the next time someone asks the original question it will give the wrong answer again. Once it can learn from this it'll improve a lot. The big hazard there is of course people manipulating it with false information. I don't have all the answers and I'm not an AI researcher but I'm very much amazed at the progress so far. And a bit scared. If this tech keeps evolving at this speed, job stability is only a tiny drop in a tidal wave of change. But there's no point in even trying to stop it.

hutzlibu 4y ago
"Imagine being an expert on all sub-fields of IT, law, marketing, medical, etc and being able to combine all that knowledge from those fields. A human will never be able to do that. The potential is huge but like any source of information you have to verify." ChatGPT is not an expert in any of these fields, it just has an enormous detail knowledge, but no understanding. A real expert actually understands his field and gives answers that makes sense and clearly states if he does not know something. ChatGPT always gives authorative answers and sometimes they are right. It is a advanced tool, but no competent human is about to be replaced by it anytime soon.
- wkat4242 4y ago
  I would beg to differ. It can combine information from various sources and make it into a working script or program. I've seen it do that various times with questions much to obscure to have found 1:1 on google. This is not a talking search engine, it's really something much more advanced.

bombela 4y ago

It understands you in any languages, speaks really well any languages. And of course, converses. Like, no big deal. I remember the AI bots of a decade ago. Nothing compares. Skynet is coming.

5e92cb50239222b 4y ago
What languages did you try? I speak Russian fluently and ChatGPT isn't very good at it. It frequently produces nonsensical answers, uses wrong inflections all the time, writes songs and poems that don't rhyme, and its blocks can easily be circumvented to praise Hitler or do anything else. It also writes text at a noticeably slower rate (at least 2× compared to English, maybe more).
- cypress66 4y ago
  I've asked it to solve certain coding tasks in both English and Spanish and it seems about equally competent in both.
- Shorel 4y ago
  It's good at Spanish, I did not notice any grammatical mistakes or nonsensical answers caused by the language.
- bombela 4y ago
  English and French which I both speak/write fluently. I assumed; wrongly; that it must have been good equality with other languages. The poems in French didn't rhyme. I haven't tried poems in English.

zmachinaz 4y ago

One could equally say that mainly the mediocre people are excited, while the smart people actually see the limitations. For people doing non-trivial things, the output of ChatGPT seems currently rather far away from being of any deep value. To replace some mundane tasks, yes, but mundane tasks are anyway just for the mediocre.

vasco 4y ago
"The mediocre people" usually are the ones that think there's "mediocre" and "smart" buckets for people.
williamcotton 4y ago
> but mundane tasks are anyway just for the mediocre. So when you’re working on something you just let other people take care of the inevitable mundane tasks that arise in daily knowledge work? I’m guessing you’ve never emptied the garbage can at the office either as that would be unworthy of your eminent intellect.

loveparade 4y ago

I think it's just as likely to be the other way around. AI has been doing these hype cycles for for 30+ years now and every time it's "this is amazing and will take people's jobs" - But what always happens is not much change in the short term, and rather subtle changes in the long term. People underestimate how important the edge cases these models aren't good at are. I also think ChatGPT is amazing, but historically it's not any more amazing that many of the previous AI-related developments for their time, none of which have quite lived up to the initial promise after the hype died. Just look at some of the robots from the 80s [0] that were expected to be in every household. And when chess was solved, of course we were just a few years away from AGI. In 20 years, people will look at ChatGPT and make fun of how cute it was, and we'll still be just a few years away from AGI. People aren't blind, they are just realistic. Yes, current AI models will slowly be integrated into services and bring changes behind the scenes, but it's not going to be the big explosion of change a lot of people expect it to be. [0] https://www.youtube.com/watch?v=jkOctWWsj-A https://www.youtube.com/watch?v=jkOctWWsj-A