4 ms·

I've been saying exactly this from the beginning. I don't like being lied to, and calling ChatGPT AI was always a lie, from minute one. It is not intelligent in

I've been saying exactly this from the beginning. I don't like being lied to, and calling ChatGPT AI was always a lie, from minute one. It is not intelligent in any sense of the word, whatsoever, nor may it aspire to intelligence. It's a highly adept predictive text model. It generates words based upon words it has been trained on as being the right words to minimize the errors in the words it generates. Not useless, mind you, and frankly tons of B2B salespeople especially would benefit from using it to clean up their nearly incoherent rambling emails; but not intelligence. The way you can "talk to" these machines is certainly clever, and the way they generate their replies is fascinating technology and certainly beats the shit out of a traditional chatbot, all of these things are true and I think they have a bright future as such. Natural language UI is a cool concept and it's cool to see some serious progress there. But that's all it is, and all it really can be. I don't know for certain what the path is to true emergent intelligence from the machine, but I assure you that shoving billions of words scraped from the internet through an LLM is not that. Now, if for whatever reason you want ream upon ream of pretty dull and repetitive content churned out on a given subject, that's far longer than it needs to be to convey the limited information within as apparently tons of online businesses do, written damn near to perfection? ChatGPT's your hookup, no question. However you will never get blinding insights, you will never get new ideas, and it will never be your friend. Sorry.

baq 3y ago
It isn't a person. It also isn't a computer as we were used to. It's a new thing. Being able to query the latent space of a large subset of human knowledge using natural language is the closest to 'intelligent computer' as we've ever come to. Yes it is lossy compression, it confabulates the bits that it lost, but it's still useful.
- ToucanLoucan 3y ago
  > Being able to query the latent space of a large subset of human knowledge using natural language is the closest to 'intelligent computer' as we've ever come to. I would agree with this statement, but it's still laughably far from it. As the other comment said, it lacks the ability to rank the veracity of the information it presents: it will present it in a grammatically correct way, but it has no way of knowing if the information it's presenting is accurate which explains ChatGPT's tendency to just make some shit up when you ask it things. And in the odd event it is correct, it's simply digesting information from other sources, likely search engines either it's own or not. Again, this is not useless and not not-clever: however in that way it does remain pretty useless because of the above issues of accuracy. The best it can do is get you the most highly ranked information on a topic based on it's pool of knowledge, and that may or may not be the most correct one. Also I'd add: these two issues compound one another, because the LLM is obscuring the source of it's information. You can have it add the link to it's citations, that would be helpful, but you don't know where it got those citations or why it picked them, and the LLM can't explain that for you. And, those choices were based on factors that are definitively not the accuracy or reliability of those sources, merely their ranking in whatever search algorithm is being used. Even if you presume this AI was trained on data relevant to the topic you're asking about, that doesn't mean it (and in fact, as I understand LLM, it's basically incapable of) understands that topic in a way that makes sense to a subject matter expert. If you asked an engineer what size lumber would be required to construct a floor that could support a small vehicle, that engineer would probably have a rough guess based on their previous experiences, and further, could then do the work: research the weight of the vehicle, the situation the structure will be installed in, the environment it will exist in, etc. and come back with a solid answer to your question. An LLM, by comparison, would consult the writings it was trained on for similar questions, and create an answer it thinks matches yours. But what does "similar" mean here? Is the wood and the dimensions the same? Is what it's looking at based on a small vehicle or a cement truck? Is it looking for articles about structures in similar climates or did it pick some that were closer matched to the structure but in completely different environments? On and on. Using an LLM this way takes the problematic aspects of googling things and basically factors them out, because now you can't see what it's googling, which results it's picking or why, and can't verify their relevance. How many times have you searched specific things and left a google result completely befuddled as to why on earth it included it? Now imagine that, except there's essentially zero chance the AI caught that, and it incorporated that irrelevant data anyway.
  - disqard 3y ago
    Yes, it lacks veracity. However, it does have verisimilitude. IMO, that quality (of appearing to be correct) is so attractive to us humans, that all other considerations are being discarded.
- dawnerd 3y ago
  Except it’s not fact checking and answers can’t be relied upon as fact. It’s creating an answer which to the machine seems correct.
  - idontpost 3y ago
    [dead]
  - lupire 3y ago
    Sounds like an AI to me. People seem to conflate "artificial intelligence" aka smart like a person , with Singularity, aka "artificial omniscient god". Just because it can't do everything any human could do or conceive of being done, doesn't mean it isn't doing things that humans do but rocks and even dogs can't.
    - dawnerd 3y ago
      I think most of us here know that but that’s not how it’s being marketed.
tempest_ 3y ago
Having discussions about what is "AI" without first defining it explicitly is usually a fruitless effort. Eventually someone will claim the goal posts are being moved and they will probably be right because there were never any to start with.
- ToucanLoucan 3y ago
  > Having discussions about what is "AI" without first defining it explicitly is usually a fruitless effort. Oh nonsense. AI: Artificial Intelligence. Artifical - made or produced by human beings rather than occurring naturally, especially as a copy of something natural. Intelligence - the ability to acquire and apply knowledge and skills. ChatGPT is certainly artificial, but it lacks the ability to apply knowledge and skills. It will do it, KIND OF, on the knowledge front, if you ASK it to, but we wouldn't call a car that moves when the accelerator is pressed a creature, nor would we call google intelligent because it can respond to queries when prompted. It's a machine. Firmly in the realm of a machine that responds to input from other actors to accomplish a task presented it. You can leave a ChatGPT instance running for a thousand years and it will never do anything until prompted. That's not intelligence. There's simply no debate here to be had. It's a CLEVER machine, and it does things a lot of previous machines could not, but a machine it remains nonetheless.
  - jeremyjh 3y ago
    Yes. A lot of people get confused about goalposts because we had one idea for gauging or measuring intelligence (Turing test), and that idea turned out to be wrong. We don't know how to define intelligence, but we know it when we see it. ChatGPT has fooled a lot of people, but that is nothing new.
    - JHonaker 3y ago
      The Turing test has been a poor goalpost almost since its inception. It turns out, it doesn't require much to fool people. People thought ELIZA was going to eliminate the need for therapists and psychiatrists. Hell, ELIZA still beat ChatGPT 3.5 in terms of % of people that believed they were interacting with a human in a recent study! [1] Notice that we only properly identified humans about 60% of the time too. [1] https://arstechnica.com/information-technology/2023/12/real-humans-appeared-human-63-of-the-time-in-recent-turing-test-ai-study/ https://arstechnica.com/information-technology/2023/12/real-...
owisd 3y ago
Researchers use “Artificial General Intelligence” to mean what you’re talking about. AI has always been a bit of a nebulous term with moving goalposts that tends to be used to mean “something that humans were significantly better than computers at until recently”. Not that I’m suggesting marketers haven’t deceptively confused the two.
swatcoder 3y ago
> However you will never get blinding insights, you will never get new ideas, and it will never be your friend I've been 100% in the "exciting tech being painfully misrepresented" camp all along too, and I think more people are saying that as time passes, but I think you might surprised what little people need for insights, ideas, and friends. Rubber duck debugging, Brian Eno's Oblique Strategies, John Cage's I Ching use, journaling, etc already delivered a useful degree of those things with far less sophistication, and these generative tools really can take many of those to another level.
- joloooo 3y ago
  Yeah its great as a mirror. Simon Wardley's thread comes to mind. " Think of it like Myers Briggs or Astrology. Gibberish that is useful for self reflection. " https://twitter.com/swardley/status/1717457831736103159 https://twitter.com/swardley/status/1717457831736103159

baq 3y ago

It isn't a person. It also isn't a computer as we were used to. It's a new thing. Being able to query the latent space of a large subset of human knowledge using natural language is the closest to 'intelligent computer' as we've ever come to. Yes it is lossy compression, it confabulates the bits that it lost, but it's still useful.

ToucanLoucan 3y ago
> Being able to query the latent space of a large subset of human knowledge using natural language is the closest to 'intelligent computer' as we've ever come to. I would agree with this statement, but it's still laughably far from it. As the other comment said, it lacks the ability to rank the veracity of the information it presents: it will present it in a grammatically correct way, but it has no way of knowing if the information it's presenting is accurate which explains ChatGPT's tendency to just make some shit up when you ask it things. And in the odd event it is correct, it's simply digesting information from other sources, likely search engines either it's own or not. Again, this is not useless and not not-clever: however in that way it does remain pretty useless because of the above issues of accuracy. The best it can do is get you the most highly ranked information on a topic based on it's pool of knowledge, and that may or may not be the most correct one. Also I'd add: these two issues compound one another, because the LLM is obscuring the source of it's information. You can have it add the link to it's citations, that would be helpful, but you don't know where it got those citations or why it picked them, and the LLM can't explain that for you. And, those choices were based on factors that are definitively not the accuracy or reliability of those sources, merely their ranking in whatever search algorithm is being used. Even if you presume this AI was trained on data relevant to the topic you're asking about, that doesn't mean it (and in fact, as I understand LLM, it's basically incapable of) understands that topic in a way that makes sense to a subject matter expert. If you asked an engineer what size lumber would be required to construct a floor that could support a small vehicle, that engineer would probably have a rough guess based on their previous experiences, and further, could then do the work: research the weight of the vehicle, the situation the structure will be installed in, the environment it will exist in, etc. and come back with a solid answer to your question. An LLM, by comparison, would consult the writings it was trained on for similar questions, and create an answer it thinks matches yours. But what does "similar" mean here? Is the wood and the dimensions the same? Is what it's looking at based on a small vehicle or a cement truck? Is it looking for articles about structures in similar climates or did it pick some that were closer matched to the structure but in completely different environments? On and on. Using an LLM this way takes the problematic aspects of googling things and basically factors them out, because now you can't see what it's googling, which results it's picking or why, and can't verify their relevance. How many times have you searched specific things and left a google result completely befuddled as to why on earth it included it? Now imagine that, except there's essentially zero chance the AI caught that, and it incorporated that irrelevant data anyway.
- disqard 3y ago
  Yes, it lacks veracity. However, it does have verisimilitude. IMO, that quality (of appearing to be correct) is so attractive to us humans, that all other considerations are being discarded.
dawnerd 3y ago
Except it’s not fact checking and answers can’t be relied upon as fact. It’s creating an answer which to the machine seems correct.
- idontpost 3y ago
  [dead]
- lupire 3y ago
  Sounds like an AI to me. People seem to conflate "artificial intelligence" aka smart like a person , with Singularity, aka "artificial omniscient god". Just because it can't do everything any human could do or conceive of being done, doesn't mean it isn't doing things that humans do but rocks and even dogs can't.
  - dawnerd 3y ago
    I think most of us here know that but that’s not how it’s being marketed.

tempest_ 3y ago

Having discussions about what is "AI" without first defining it explicitly is usually a fruitless effort. Eventually someone will claim the goal posts are being moved and they will probably be right because there were never any to start with.

ToucanLoucan 3y ago
> Having discussions about what is "AI" without first defining it explicitly is usually a fruitless effort. Oh nonsense. AI: Artificial Intelligence. Artifical - made or produced by human beings rather than occurring naturally, especially as a copy of something natural. Intelligence - the ability to acquire and apply knowledge and skills. ChatGPT is certainly artificial, but it lacks the ability to apply knowledge and skills. It will do it, KIND OF, on the knowledge front, if you ASK it to, but we wouldn't call a car that moves when the accelerator is pressed a creature, nor would we call google intelligent because it can respond to queries when prompted. It's a machine. Firmly in the realm of a machine that responds to input from other actors to accomplish a task presented it. You can leave a ChatGPT instance running for a thousand years and it will never do anything until prompted. That's not intelligence. There's simply no debate here to be had. It's a CLEVER machine, and it does things a lot of previous machines could not, but a machine it remains nonetheless.
- jeremyjh 3y ago
  Yes. A lot of people get confused about goalposts because we had one idea for gauging or measuring intelligence (Turing test), and that idea turned out to be wrong. We don't know how to define intelligence, but we know it when we see it. ChatGPT has fooled a lot of people, but that is nothing new.
  - JHonaker 3y ago
    The Turing test has been a poor goalpost almost since its inception. It turns out, it doesn't require much to fool people. People thought ELIZA was going to eliminate the need for therapists and psychiatrists. Hell, ELIZA still beat ChatGPT 3.5 in terms of % of people that believed they were interacting with a human in a recent study! [1] Notice that we only properly identified humans about 60% of the time too. [1] https://arstechnica.com/information-technology/2023/12/real-humans-appeared-human-63-of-the-time-in-recent-turing-test-ai-study/ https://arstechnica.com/information-technology/2023/12/real-...

owisd 3y ago

Researchers use “Artificial General Intelligence” to mean what you’re talking about. AI has always been a bit of a nebulous term with moving goalposts that tends to be used to mean “something that humans were significantly better than computers at until recently”. Not that I’m suggesting marketers haven’t deceptively confused the two.

swatcoder 3y ago

> However you will never get blinding insights, you will never get new ideas, and it will never be your friend I've been 100% in the "exciting tech being painfully misrepresented" camp all along too, and I think more people are saying that as time passes, but I think you might surprised what little people need for insights, ideas, and friends. Rubber duck debugging, Brian Eno's Oblique Strategies, John Cage's I Ching use, journaling, etc already delivered a useful degree of those things with far less sophistication, and these generative tools really can take many of those to another level.

joloooo 3y ago
Yeah its great as a mirror. Simon Wardley's thread comes to mind. " Think of it like Myers Briggs or Astrology. Gibberish that is useful for self reflection. " https://twitter.com/swardley/status/1717457831736103159 https://twitter.com/swardley/status/1717457831736103159