8 ms·

And you will almost immediately run into the fundamental problem with current iterations of GPT - You can not trust it to be correct or actually do the thing yo

And you will almost immediately run into the fundamental problem with current iterations of GPT - You can not trust it to be correct or actually do the thing you want, only something that resembles the thing you want. The description in this link puts some really high hopes on the ability of AI to simply "figure out" what you want with little input. In reality, it will give you something that sorta kinda looks like what you want if you squint but falls immediately flat the moment you need to put it into an actual production (or even testing) environment.

pbalau 4y ago
Regardless of how successful an AI can get at figuring out what the human operator wants, in the end, all it will manage to do is figure out what the human wants or give similar options to what the human asked for. My experience in working with software and being a client for other craftsmen, is that rarely that's what needs done, do what the human wants. The whole idea of a good craftsman is to figure out what the client needs. That was also my job for the past few years, figuring out what my company needs to build next, either as a product, or internal infra, tooling etc. I did end up building the things myself because there are only 4 engineers in my company and I had to do the building bit. An AI will boost my capabilities (automation does that too, but that still needs me to build it...). Before you tell me that an AI will soon be able to do what I do, we are lifetimes away from that, if it's even possible. That will mean our creation fully understands us, it can understand stupid. If I were religiously inclined, I could even argue that even God failed at such a task.
- xnorswap 4y ago
  If ChatGPT replaces me, I suspect we'll be the ones turning actual client needs into GPT prompts, because as you say, what people ask for, what people want, and what people need are all different things and it's (currently) useful to have a human understand the difference, regardless of whether they're then interfacing with punch cards, an IDE or a chatGPT prompt.
peter303 4y ago
The 5% to 10% of the output that is factually wrong or socially inappropriate could be a legal nightmare.
oceanplexian 4y ago
I keep hearing this assertion, that GPT can be wrong, therefore it’s an unworkable technology. But it’s a bad comparison. LLMs aren’t trying to be computationally correct like a calculator or something, the value is in their ability to semantically process a question. The other issue is assuming that the existing way of doing things is always correct. Engineers frequently get things wrong. If an AI model can complete a task with 95% correctness but let’s say a Jr. Engineer can compete the same task with 85% correctness then it makes sense to use the model instead. I’m not sure why folks can’t see the obvious conclusion of where this is heading.
- groestl 4y ago
  > then it makes sense to use the model instead Especially since a Sr. Engineer (possible with Jr.'s input) using an AI for debugging, might be 99.9% correct _and_ faster.
- kenjackson 4y ago
  And even if the AI model is only 75% correct, if it can generate the output near instantly and give that as a starting point, that's great. There's a reason why templates, wizards, and samples are so popular -- after servicing of code, the hardest part is probably getting started with it.
  - etothepii 4y ago
    This is a fair. However, all our skills and skills at picking people with skills were trained on the set of people that make are surprisingly good at knowing what they don't know.
- krainboltgreene 4y ago
  > If an AI model can complete a task with 95% correctness but let’s say a Jr. Engineer can compete the same task with 85% correctness then it makes sense to use the model instead. I’m not sure why folks can’t see the obvious conclusion of where this is heading. Because this is incredibly shortsighted and also fundamentally misunderstands the return data of an LLM.
throw__away7391 4y ago
Every time I use it for code, it suggests using APIs that don't exist but definitely look like they could. If asked it can go on in great detail talking about the mundane details completely convincingly of APIs that either don't exist or have completely different structure than what is described. On the other hand it is really good at tasks like "turn this XML in JSON and give me a JSON Schema definition for it".
- ProllyInfamous 4y ago
  Treat LLM systems like "Toddlers who know everything, but have NO COMMON SENSE." —Anna Bernstein
naasking 4y ago
> You can not trust it to be correct or actually do the thing you want, only something that resembles the thing you want. So, just like people then?
RC_ITR 4y ago
An interesting take on AI is that it's just a tool that overcomes some of the quirks of capitalism, and we are impressed with that because we are so deeply entrenched in capitalism. Put differently - every website needs a back-end. 95%+ of websites don't differentiate on their back-end, but they still need to build from scratch since there's no incentive for businesses to share knowledge with unaffiliated businesses. One way this problem is solved is neutral platforms like AWS that sell the 'good enough' turn-key solution (keep in mind, at one point, the cloud had nearly as much hype as AI does now). Another way to solve the problem is an AI that 'makes' the back-end code 'from scratch,' but is really just returning the code (cribbed from its training dataset) that probabilistically answers your question in the best way possible, based on the results of its training. The AI option seems really impressive to us right now, because we haven't seen it before (much like photoshop in the 90's), but eventually we get used to it. Once we get to that phase, we will either regulate AI until it looks like a marketplace business (the creators of the training dataset maybe should be compensated) or we will just see 'generating code from a training dataset' as so basic that we move on to other, harder problems that have no training dataset yet (in the same way Quickbooks has largely replaced book-keepers, but digital advertisers for small business are increasingly relevant).
- ProllyInfamous 4y ago
  >[using probability it] answers your question in the best way possible, based on the results of its training. In medical school, we were taught Differential Diagnosis, which is the manner which GPs MUST utilize to solve symptoms. This is a probabilistically-determined ranking of what is MOST LIKELY to be the cause, based on how the patient presents [medical symptoms]. A LLM like ChatGPT is already demonstrating can, with a over-worked (&underpaid) GP, filter through many of the "first guess" diagnosis, and prevent unnecessary testing using information that extend beyond the single minds of patient and doctor. These datasets know how every. other. body. has. responded. to treatment... and if they don't now, they will. The ability for the "Democratization of mental healthcare" has already arrived, and positive, motivations responses that people are already getting from these system (e.g. finding purpose by asking "what GPT thinks [AUTHOR]'s POV on 'the meaning of life'?") is absolutely profound; and absolutely unavailable to the large swaths of men which society so-readily ignored (e.g. veterans). ...until now. I am glad I did not finish medical school, because the writing was on the was fifteen years ago, and one of the few justification for an expensive doctor's salary now is [from a hospital administrator's view] just a way to distribute liability among the FEW humans that can remain in competition simply by putting-together absolutely esoteric connections. Peace.

pbalau 4y ago

Regardless of how successful an AI can get at figuring out what the human operator wants, in the end, all it will manage to do is figure out what the human wants or give similar options to what the human asked for. My experience in working with software and being a client for other craftsmen, is that rarely that's what needs done, do what the human wants. The whole idea of a good craftsman is to figure out what the client needs. That was also my job for the past few years, figuring out what my company needs to build next, either as a product, or internal infra, tooling etc. I did end up building the things myself because there are only 4 engineers in my company and I had to do the building bit. An AI will boost my capabilities (automation does that too, but that still needs me to build it...). Before you tell me that an AI will soon be able to do what I do, we are lifetimes away from that, if it's even possible. That will mean our creation fully understands us, it can understand stupid. If I were religiously inclined, I could even argue that even God failed at such a task.

xnorswap 4y ago
If ChatGPT replaces me, I suspect we'll be the ones turning actual client needs into GPT prompts, because as you say, what people ask for, what people want, and what people need are all different things and it's (currently) useful to have a human understand the difference, regardless of whether they're then interfacing with punch cards, an IDE or a chatGPT prompt.

peter303 4y ago

The 5% to 10% of the output that is factually wrong or socially inappropriate could be a legal nightmare.

oceanplexian 4y ago

I keep hearing this assertion, that GPT can be wrong, therefore it’s an unworkable technology. But it’s a bad comparison. LLMs aren’t trying to be computationally correct like a calculator or something, the value is in their ability to semantically process a question. The other issue is assuming that the existing way of doing things is always correct. Engineers frequently get things wrong. If an AI model can complete a task with 95% correctness but let’s say a Jr. Engineer can compete the same task with 85% correctness then it makes sense to use the model instead. I’m not sure why folks can’t see the obvious conclusion of where this is heading.

groestl 4y ago
> then it makes sense to use the model instead Especially since a Sr. Engineer (possible with Jr.'s input) using an AI for debugging, might be 99.9% correct _and_ faster.
kenjackson 4y ago
And even if the AI model is only 75% correct, if it can generate the output near instantly and give that as a starting point, that's great. There's a reason why templates, wizards, and samples are so popular -- after servicing of code, the hardest part is probably getting started with it.
- etothepii 4y ago
  This is a fair. However, all our skills and skills at picking people with skills were trained on the set of people that make are surprisingly good at knowing what they don't know.
krainboltgreene 4y ago
> If an AI model can complete a task with 95% correctness but let’s say a Jr. Engineer can compete the same task with 85% correctness then it makes sense to use the model instead. I’m not sure why folks can’t see the obvious conclusion of where this is heading. Because this is incredibly shortsighted and also fundamentally misunderstands the return data of an LLM.

throw__away7391 4y ago

Every time I use it for code, it suggests using APIs that don't exist but definitely look like they could. If asked it can go on in great detail talking about the mundane details completely convincingly of APIs that either don't exist or have completely different structure than what is described. On the other hand it is really good at tasks like "turn this XML in JSON and give me a JSON Schema definition for it".

ProllyInfamous 4y ago
Treat LLM systems like "Toddlers who know everything, but have NO COMMON SENSE." —Anna Bernstein

naasking 4y ago

> You can not trust it to be correct or actually do the thing you want, only something that resembles the thing you want. So, just like people then?

RC_ITR 4y ago

An interesting take on AI is that it's just a tool that overcomes some of the quirks of capitalism, and we are impressed with that because we are so deeply entrenched in capitalism. Put differently - every website needs a back-end. 95%+ of websites don't differentiate on their back-end, but they still need to build from scratch since there's no incentive for businesses to share knowledge with unaffiliated businesses. One way this problem is solved is neutral platforms like AWS that sell the 'good enough' turn-key solution (keep in mind, at one point, the cloud had nearly as much hype as AI does now). Another way to solve the problem is an AI that 'makes' the back-end code 'from scratch,' but is really just returning the code (cribbed from its training dataset) that probabilistically answers your question in the best way possible, based on the results of its training. The AI option seems really impressive to us right now, because we haven't seen it before (much like photoshop in the 90's), but eventually we get used to it. Once we get to that phase, we will either regulate AI until it looks like a marketplace business (the creators of the training dataset maybe should be compensated) or we will just see 'generating code from a training dataset' as so basic that we move on to other, harder problems that have no training dataset yet (in the same way Quickbooks has largely replaced book-keepers, but digital advertisers for small business are increasingly relevant).

ProllyInfamous 4y ago
>[using probability it] answers your question in the best way possible, based on the results of its training. In medical school, we were taught Differential Diagnosis, which is the manner which GPs MUST utilize to solve symptoms. This is a probabilistically-determined ranking of what is MOST LIKELY to be the cause, based on how the patient presents [medical symptoms]. A LLM like ChatGPT is already demonstrating can, with a over-worked (&underpaid) GP, filter through many of the "first guess" diagnosis, and prevent unnecessary testing using information that extend beyond the single minds of patient and doctor. These datasets know how every. other. body. has. responded. to treatment... and if they don't now, they will. The ability for the "Democratization of mental healthcare" has already arrived, and positive, motivations responses that people are already getting from these system (e.g. finding purpose by asking "what GPT thinks [AUTHOR]'s POV on 'the meaning of life'?") is absolutely profound; and absolutely unavailable to the large swaths of men which society so-readily ignored (e.g. veterans). ...until now. I am glad I did not finish medical school, because the writing was on the was fifteen years ago, and one of the few justification for an expensive doctor's salary now is [from a hospital administrator's view] just a way to distribute liability among the FEW humans that can remain in competition simply by putting-together absolutely esoteric connections. Peace.