6 ms·

> The danger of "AI" is that we actually believe the plausible fabrications it produces are "intelligent". 100% this! I'm fed up with seemingly rational peopl

> The danger of "AI" is that we actually believe the plausible fabrications it produces are "intelligent". 100% this! I'm fed up with seemingly rational people who just can't comprehend that the AI hype is just sales talk. I had to convince a customer the other day we cannot write whole apps with ChatGPT. As far as I know, not a single example exists of a full app written by ChatGPT. It just can't be done, because ChatGPT is not reasoning!!! It is not intelligent. It can just spit out seemingly coherent, but often incorrect, snippets of code. I think it's best described as a word-calculator, or autocomplete on crack, in the sense that it is great on guessing what comes next. But it has no reasoning behind it's predictions, only statistics. EDIT: The AI apologetics are downvoting me LOL

_fizz_buzz_ 3y ago
Why didn't the customer write the whole app themselves with chatgpt?
- shzhdbi09gv8ioi 3y ago
  They tried to show me what they meant. ChatGPT proceeded to spit out a snippet of rust code, starting with a dependecy outdated by 4 years, proceeeded to mix API from several versions of the dependency and completely ignored most of the request. Customer do not know rust so it looks amazing to them. ChatGPT is a great con-man for the gullible.
  - ben_w 3y ago
    > ChatGPT proceeded to spit out a snippet of rust code, starting with a dependecy outdated by 4 years, proceeeded to mix API from several versions of the dependency and completely ignored most of the request. If that's what you meant, I think everyone here, including the people disagreeing with examples (e.g. me), would have agreed. What I mean with my examples is: it can do all the bits. Is it perfect? Nope. Do you still need a human in the loop to understand the failure modes? Yup. That MS paint app you asked for 45 minutes ago? Features are decomposable, so it can do this step by step, so long as you're OK adding one feature at a time instead of trying to do it all in one go — 75% of the results are fully functional and do what you asked for (which isn't always what you wanted, "tool selector" can be the CSS or the widget itself).
    - shzhdbi09gv8ioi 3y ago
      Some people used to write crap code stitching together snippets from stack overflow, now the same people write crap code stitching together what ChatGPT spit out after trained on said stackoverflow clusterfuck of buggy or flat out incorrect solutions. You do you.
      - ben_w 3y ago
        And those people are now your competitors and customers. Worse (for all of us in this industry) sometimes there's a bug or three, but it can generally fix those bugs by you giving the source code (if the source didn't already come from the LLM) and simply describing the issue. Even when the bug is graphical in nature, which is pretty wild for a text based chat engine that can't see (like 3.5 is). And that's without applying my experience of any language or frameworks, e.g. when asking it for solutions in languages I don't have meaningful professional experience in, like JavaScript. When I can apply my experience, I cringe at the choices it comes up with, and find I want to use it as a rough guide rather than a complete answer. But you know who doesn't care about cringy code? Everyone who isn't a developer. Even QA only cares if it passes the tests.
        shzhdbi09gv8ioi 3y ago
        > And those people are now your competitors and customers. For now, yes. > When I can apply my experience, I cringe at the choices it comes up with, and find I want to use it as a rough guide rather than a complete answer. But you know who doesn't care about cringy code? Everyone who isn't a developer. Even QA only cares if it passes the tests. This is a race to the bottom, surely you must understand that? I expect some of the biggest tech companies to start limiting developer use of LLM:s soon due to the monumental technical debt they are creating. More context: https://visualstudiomagazine.com/articles/2024/01/25/copilot-research.aspx https://visualstudiomagazine.com/articles/2024/01/25/copilot...
robbiep 3y ago
I’ve actually used a mates replit chained gpt-4 app to build short single function apps from a single command that function on first run, so it’s definitely possible, and will only get better - but building code is just following a logical set of instructions down a pathway
- shzhdbi09gv8ioi 3y ago
  "Single function app".. how is that different from a snippet? Show me a proper app generated by ChatGPT. You can't, because ChatGPT can't.
- player1234 3y ago
  It will only get better, based on what? A true believer just knows.
  - shzhdbi09gv8ioi 3y ago
    Reminds me of the latest excuse from OpenAI: https://news.yahoo.com/ai-needs-nuclear-power-breakthrough-104214982.html https://news.yahoo.com/ai-needs-nuclear-power-breakthrough-1... (its not the tech that are wrong, we just don't have enough energy in the world).
    - ben_w 3y ago
      Sam Altman is also a non-trivial investor in various nuclear energy R&D companies, e.g. $375 million into Helion, so take that "need" with a pinch of salt. He made a bet, presumably because he already believes, but it's human nature to try to justify your decisions after the fact and not just before.
soulofmischief 3y ago
Are we using the same ChatGPT? It cannot write a full app, but that is a matter of context size. It absolutely can "reason" within bounds, attention is a much more powerful system than you realize. Chain-of-Thought is a well-established technique for LLM prompting which can produce great results. And the snippets of code GPT-4 generates for me are usually on point, at least they were until OpenAI dumbed the model down in the last few months with GPT-4-Turbo. I get it not only to write novel code, but explain existing code all the time and it reasons about my code very well.
- shzhdbi09gv8ioi 3y ago
  Ahh.. I didn't pay for the premium version, that is why I am not convinced. /s
  - soulofmischief 3y ago
    I don't know what to tell you if you think you're entitled to a completely free cutting-edge service that takes multiple dedicated GPUs to serve your requests.
kombookcha 3y ago
>autocomplete on crack I have had some really frustrating conversations about this very topic with people insisting on using it for complex tasks like designing business strategies. It doesn't know how to actually answer your question, it's just spitting out probable words associated with the topic in a plausible sequence! It's like Quora with spellcheck and an aura of legitimacy which makes it infinitely more seductive.
JimDabell 3y ago
> I'm fed up with seemingly rational people who just can't comprehend that the AI hype is just sales talk. The reason why you see rational people believe the AI hype is that AI is a lot better than you give it credit for. They are right. > I had to convince a customer the other day we cannot write whole apps with ChatGPT. As far as I know, not a single example exists of a full app written by ChatGPT. It just can't be done It can. When it first came out, I wanted a macOS menu bar app so that I could access ChatGPT conversations from the menu bar. I’d never written a macOS app before. I told it what I wanted, it wrote the whole thing in one go. There was one minor compile error (a type signature had changed from a previous version, if I remember correctly), which was a one-line fix. I iterated a couple of times, telling ChatGPT what improvements I wanted to make to the app. It did them. Would I use it to build a complex app? No. But it is capable of building a whole app.
- shzhdbi09gv8ioi 3y ago
  > The reason why you see rational people believe the AI hype is that AI is a lot better than you give it credit for. They are right. I am sorry, but you are mistaken. There are no AI. The "AI" that is being mongered today is just a salespeople buzz word for Large Language Model technology, which is just that, a language model. It can be used to generate seemingly coherent text, but it can not reason. > Would I use it to build a complex app? No. But it is capable of building a whole app. It only works for you as you accept lowering the bar of that would pass as an "app". I mean of course any kind of normal app. Say Microsoft Paint for example. Show me ChatGPT able to create a full app like that based on business logic you prompted.
  - JimDabell 3y ago
    > It only works for you as you accept lowering the bar of that would pass as an "app". I asked it to create an app, it created the app I wanted. That’s not “lowering the bar”. You’re just redefining words to prop up your incorrect claims.
    - shzhdbi09gv8ioi 3y ago
ben_w 3y ago
"""That's one of those irregular verbs, isn't it? I give confidential security briefings. You leak. He has been charged under section 2a of the Official Secrets Act.""" - Yes Minister. To rephrase for the subject: We're only human, mistakes are to be expected; They are a idiots, mistakes are to be expected; that thing is just a glorified calculator, mistakes are to be expected. (There's a short story I found I couldn't bring myself to finish, “Zero for Conduct” by Greg Egan, where the lead character is bullied by someone who has a similar disregard for her intelligence; I know one cannot use fiction to learn about reality, so I will instead say that this disregard of human intelligence by other humans happens a lot in real life too, the racism and xenophobia of βαρβαρίζω can still be found today in all the people who insist that ancient structures like the pyramids couldn't possibly have been built by the locals and therefore it must have been aliens). > AI hype is just sales talk But where does the hype end and the reality begin? > I had to convince a customer the other day we cannot write whole apps with ChatGPT. As far as I know, not a single example exists of a full app written by ChatGPT. It just can't be done, because ChatGPT is not reasoning!!! It is not intelligent. I will agree ChatGPT does indeed make incoherent solutions — one test project was making a game in JS, it (eventually) gave me a vector class with methods for multiply(scalar) etc., but then tried to use mul(scalar). But ironically, I've also made a functioning (basic, but functioning) ChatGPT API interface… by bolting together the output of ChatGPT. I won't claim it's amazing or anything, because I'm an iOS developer and the thing "I" made is a web app, but it works well enough for my needs (just don't paste HTML into the query section, because I stopped adding to it when it was good enough for my needs and therefore only have a very basic solution to code being shown in the chat list, there's a lot of stuff that would be improved by using simple libraries but I didn't want to). > I think it's best described as a word-calculator, or autocomplete on crack. And that's the same error in the opposite direction. If I understand right, GPT-3 is literally the complexity of a mid-sized rodent. Thus the metaphor I use is: Imagine a ferret was generically modified to be immortal, had every sensory input removed except smell, which was wired up to a computer. The ferret and computer then spend 50,000 years going through 10% of all the text on the internet, where every sequence is tokenised and those tokens are turned into a pattern of olfactory nerves to stimulate, and the ferret is rewarded or punished based on how well it imagined the next token. You're annoyed that this specific ferret's jokes are derivative, their code doesn't always compile, that they make mistakes when trying to solve algebraic problems, that their pecan pie recipe needs work, and that they make mistakes when translating Latin into Hindi. I'm amazed the ferret can do any of these things.
williamcotton 3y ago
What matters is if the tool is useful or not. You cannot claim that this tool is not useful to people and use cases you are entirely unfamiliar with. Debates of what is real intelligence are akin to running around in circles.

_fizz_buzz_ 3y ago

Why didn't the customer write the whole app themselves with chatgpt?

shzhdbi09gv8ioi 3y ago
They tried to show me what they meant. ChatGPT proceeded to spit out a snippet of rust code, starting with a dependecy outdated by 4 years, proceeeded to mix API from several versions of the dependency and completely ignored most of the request. Customer do not know rust so it looks amazing to them. ChatGPT is a great con-man for the gullible.
- ben_w 3y ago
  > ChatGPT proceeded to spit out a snippet of rust code, starting with a dependecy outdated by 4 years, proceeeded to mix API from several versions of the dependency and completely ignored most of the request. If that's what you meant, I think everyone here, including the people disagreeing with examples (e.g. me), would have agreed. What I mean with my examples is: it can do all the bits. Is it perfect? Nope. Do you still need a human in the loop to understand the failure modes? Yup. That MS paint app you asked for 45 minutes ago? Features are decomposable, so it can do this step by step, so long as you're OK adding one feature at a time instead of trying to do it all in one go — 75% of the results are fully functional and do what you asked for (which isn't always what you wanted, "tool selector" can be the CSS or the widget itself).
  - shzhdbi09gv8ioi 3y ago
    Some people used to write crap code stitching together snippets from stack overflow, now the same people write crap code stitching together what ChatGPT spit out after trained on said stackoverflow clusterfuck of buggy or flat out incorrect solutions. You do you.
    - ben_w 3y ago
      And those people are now your competitors and customers. Worse (for all of us in this industry) sometimes there's a bug or three, but it can generally fix those bugs by you giving the source code (if the source didn't already come from the LLM) and simply describing the issue. Even when the bug is graphical in nature, which is pretty wild for a text based chat engine that can't see (like 3.5 is). And that's without applying my experience of any language or frameworks, e.g. when asking it for solutions in languages I don't have meaningful professional experience in, like JavaScript. When I can apply my experience, I cringe at the choices it comes up with, and find I want to use it as a rough guide rather than a complete answer. But you know who doesn't care about cringy code? Everyone who isn't a developer. Even QA only cares if it passes the tests.
      - shzhdbi09gv8ioi 3y ago
        > And those people are now your competitors and customers. For now, yes. > When I can apply my experience, I cringe at the choices it comes up with, and find I want to use it as a rough guide rather than a complete answer. But you know who doesn't care about cringy code? Everyone who isn't a developer. Even QA only cares if it passes the tests. This is a race to the bottom, surely you must understand that? I expect some of the biggest tech companies to start limiting developer use of LLM:s soon due to the monumental technical debt they are creating. More context: https://visualstudiomagazine.com/articles/2024/01/25/copilot-research.aspx https://visualstudiomagazine.com/articles/2024/01/25/copilot...

robbiep 3y ago

I’ve actually used a mates replit chained gpt-4 app to build short single function apps from a single command that function on first run, so it’s definitely possible, and will only get better - but building code is just following a logical set of instructions down a pathway

shzhdbi09gv8ioi 3y ago
"Single function app".. how is that different from a snippet? Show me a proper app generated by ChatGPT. You can't, because ChatGPT can't.
player1234 3y ago
It will only get better, based on what? A true believer just knows.
- shzhdbi09gv8ioi 3y ago
  Reminds me of the latest excuse from OpenAI: https://news.yahoo.com/ai-needs-nuclear-power-breakthrough-104214982.html https://news.yahoo.com/ai-needs-nuclear-power-breakthrough-1... (its not the tech that are wrong, we just don't have enough energy in the world).
  - ben_w 3y ago
    Sam Altman is also a non-trivial investor in various nuclear energy R&D companies, e.g. $375 million into Helion, so take that "need" with a pinch of salt. He made a bet, presumably because he already believes, but it's human nature to try to justify your decisions after the fact and not just before.

soulofmischief 3y ago

Are we using the same ChatGPT? It cannot write a full app, but that is a matter of context size. It absolutely can "reason" within bounds, attention is a much more powerful system than you realize. Chain-of-Thought is a well-established technique for LLM prompting which can produce great results. And the snippets of code GPT-4 generates for me are usually on point, at least they were until OpenAI dumbed the model down in the last few months with GPT-4-Turbo. I get it not only to write novel code, but explain existing code all the time and it reasons about my code very well.

shzhdbi09gv8ioi 3y ago
Ahh.. I didn't pay for the premium version, that is why I am not convinced. /s
- soulofmischief 3y ago
  I don't know what to tell you if you think you're entitled to a completely free cutting-edge service that takes multiple dedicated GPUs to serve your requests.

kombookcha 3y ago

>autocomplete on crack I have had some really frustrating conversations about this very topic with people insisting on using it for complex tasks like designing business strategies. It doesn't know how to actually answer your question, it's just spitting out probable words associated with the topic in a plausible sequence! It's like Quora with spellcheck and an aura of legitimacy which makes it infinitely more seductive.

JimDabell 3y ago

> I'm fed up with seemingly rational people who just can't comprehend that the AI hype is just sales talk. The reason why you see rational people believe the AI hype is that AI is a lot better than you give it credit for. They are right. > I had to convince a customer the other day we cannot write whole apps with ChatGPT. As far as I know, not a single example exists of a full app written by ChatGPT. It just can't be done It can. When it first came out, I wanted a macOS menu bar app so that I could access ChatGPT conversations from the menu bar. I’d never written a macOS app before. I told it what I wanted, it wrote the whole thing in one go. There was one minor compile error (a type signature had changed from a previous version, if I remember correctly), which was a one-line fix. I iterated a couple of times, telling ChatGPT what improvements I wanted to make to the app. It did them. Would I use it to build a complex app? No. But it is capable of building a whole app.

shzhdbi09gv8ioi 3y ago
> The reason why you see rational people believe the AI hype is that AI is a lot better than you give it credit for. They are right. I am sorry, but you are mistaken. There are no AI. The "AI" that is being mongered today is just a salespeople buzz word for Large Language Model technology, which is just that, a language model. It can be used to generate seemingly coherent text, but it can not reason. > Would I use it to build a complex app? No. But it is capable of building a whole app. It only works for you as you accept lowering the bar of that would pass as an "app". I mean of course any kind of normal app. Say Microsoft Paint for example. Show me ChatGPT able to create a full app like that based on business logic you prompted.
- JimDabell 3y ago
  > It only works for you as you accept lowering the bar of that would pass as an "app". I asked it to create an app, it created the app I wanted. That’s not “lowering the bar”. You’re just redefining words to prop up your incorrect claims.
  - shzhdbi09gv8ioi 3y ago

ben_w 3y ago

"""That's one of those irregular verbs, isn't it? I give confidential security briefings. You leak. He has been charged under section 2a of the Official Secrets Act.""" - Yes Minister. To rephrase for the subject: We're only human, mistakes are to be expected; They are a idiots, mistakes are to be expected; that thing is just a glorified calculator, mistakes are to be expected. (There's a short story I found I couldn't bring myself to finish, “Zero for Conduct” by Greg Egan, where the lead character is bullied by someone who has a similar disregard for her intelligence; I know one cannot use fiction to learn about reality, so I will instead say that this disregard of human intelligence by other humans happens a lot in real life too, the racism and xenophobia of βαρβαρίζω can still be found today in all the people who insist that ancient structures like the pyramids couldn't possibly have been built by the locals and therefore it must have been aliens). > AI hype is just sales talk But where does the hype end and the reality begin? > I had to convince a customer the other day we cannot write whole apps with ChatGPT. As far as I know, not a single example exists of a full app written by ChatGPT. It just can't be done, because ChatGPT is not reasoning!!! It is not intelligent. I will agree ChatGPT does indeed make incoherent solutions — one test project was making a game in JS, it (eventually) gave me a vector class with methods for multiply(scalar) etc., but then tried to use mul(scalar). But ironically, I've also made a functioning (basic, but functioning) ChatGPT API interface… by bolting together the output of ChatGPT. I won't claim it's amazing or anything, because I'm an iOS developer and the thing "I" made is a web app, but it works well enough for my needs (just don't paste HTML into the query section, because I stopped adding to it when it was good enough for my needs and therefore only have a very basic solution to code being shown in the chat list, there's a lot of stuff that would be improved by using simple libraries but I didn't want to). > I think it's best described as a word-calculator, or autocomplete on crack. And that's the same error in the opposite direction. If I understand right, GPT-3 is literally the complexity of a mid-sized rodent. Thus the metaphor I use is: Imagine a ferret was generically modified to be immortal, had every sensory input removed except smell, which was wired up to a computer. The ferret and computer then spend 50,000 years going through 10% of all the text on the internet, where every sequence is tokenised and those tokens are turned into a pattern of olfactory nerves to stimulate, and the ferret is rewarded or punished based on how well it imagined the next token. You're annoyed that this specific ferret's jokes are derivative, their code doesn't always compile, that they make mistakes when trying to solve algebraic problems, that their pecan pie recipe needs work, and that they make mistakes when translating Latin into Hindi. I'm amazed the ferret can do any of these things.

williamcotton 3y ago

What matters is if the tool is useful or not. You cannot claim that this tool is not useful to people and use cases you are entirely unfamiliar with. Debates of what is real intelligence are akin to running around in circles.