4 ms·

ChatGT makes exactly the same mistakes as your colleague that has no actual experience in a certain matter: they present what they reasonably think how somethin

ChatGT makes exactly the same mistakes as your colleague that has no actual experience in a certain matter: they present what they reasonably think how something probably should be or work in absence of factual knowledge about the insane mess that was actually created in reality. Just ask a technical colleague that has no deep experience in the specifics about how they think e.g. SharePoint, SAP or Microsoft Identities work. The architectures they will extrapolate from sane logic will be very far of the incredible craziness of the actual reality.

CSSer 4y ago
Right, but all this means is that the training data doesn't include thorough or an adequate representation of that knowledge, no?
- sebzim4500 4y ago
  I don't think so, the rules of chess are certainly in the training set yet GPT-4 can not realiably play legal moves, yet alone good ones.
itsboring 4y ago
I still think there’s a big difference. If asked to speculate on the architecture of Sharepoint, my response would be “I have no earthly idea.” If pressed further for an answer, my response would be “It’s probably some over-complicated mess and I don’t care enough about Sharepoint to spend any further time on this line of questioning.” I have yet to see ChatGPT just admit it doesn’t know, but in this case “I don’t know” is the most trustworthy answer I, as a technical person, could give.
- burlesona 4y ago
  It doesn't know it doesn't know.
  - danaris 4y ago
    It is fundamentally incapable of "knowing" anything. It is a statistical engine, with no internal representation of the world or understanding of the words it emits.
    - Xelynega 4y ago
      They don't know it doesn't know
    - dTal 4y ago
      It clearly does have an internal representation of the world, implicitly encoded in its network weights. Quite an accurate one too, for most general knowledge, and one that can be updated in-context - it handily solves "blocks world[0]" style tasks. "Understanding" and "knowing" aren't helpful words, just focal points for pointless philosophical arguments. [0] https://en.wikipedia.org/wiki/Blocks_world https://en.wikipedia.org/wiki/Blocks_world
      - danaris 4y ago
        That's not a representation of the world. It's simply a lossy encoding of its data. It's not semantically structured in the way that our thoughts largely are—it's merely syntactically structured.
        EnergyAmy 4y ago
        "That's not a representation of the world. It's simply a [representation of the world as viewed from its input data]" is an... interesting take.
        dTal 4y ago
        What exactly is the difference? It's clearly managed to abstract the training data to a ludicrously deep degree, such that it's capable of solving semantically non-trivial problems it's never seen before. It can make metaphors, accurately predict the behavior of humans in complex social scenarios, and translate arbitrary passages between syntactically distinct languages while preserving nuance. That last task in particular is pretty much a slam dunk against any argument of the type you are making. "Sufficiently advanced syntax is indistinguishable from semantics."
sekai 4y ago
Yeah, you could call it a "human" characteristic
yoyohello13 4y ago
While you are right, I just don't think this is a useful comparison. There are other context clues when talking to people. I tend to know whether my colleague is talking out of his ass, and most people will preface stuff by saying they don't really know. We are trained to trust what computers are telling us, and ChatGPT doesn't 'qualify' what it's saying. I think if ChatGPT could preface what its saying with 'Well, I'm not super sure, but here is what I think' that would go a long way to solving this issue.
mhh__ 4y ago
So they're only as good as most humans in that context? No biggie...
garciansmith 4y ago
Someone with no experience may give you their best guess, but they'll also tell you that. And if I ask a colleague of mine for their sources on a particular matter that they might not be sure about (or I find curious), they would not send me a list of totally made up journal articles and book titles like ChatGPT does.
alfalfasprout 4y ago
Not sure why this is being upvoted but it's completely wrong. ChatGPT will answer like a colleague that is a subject matter expert on whatever you're asking them. And in doing so goes above and beyond inventing minute details that are blatantly wrong to pretend they're an expert. But the vast majority of folks have a concept of uncertainty and will either communicate (or know internally) that what they're saying is speculative. ChatGPT is like a pathological liar that pretends to be an expert and is often correct.

CSSer 4y ago

Right, but all this means is that the training data doesn't include thorough or an adequate representation of that knowledge, no?

sebzim4500 4y ago
I don't think so, the rules of chess are certainly in the training set yet GPT-4 can not realiably play legal moves, yet alone good ones.

itsboring 4y ago

I still think there’s a big difference. If asked to speculate on the architecture of Sharepoint, my response would be “I have no earthly idea.” If pressed further for an answer, my response would be “It’s probably some over-complicated mess and I don’t care enough about Sharepoint to spend any further time on this line of questioning.” I have yet to see ChatGPT just admit it doesn’t know, but in this case “I don’t know” is the most trustworthy answer I, as a technical person, could give.

burlesona 4y ago
It doesn't know it doesn't know.
- danaris 4y ago
  It is fundamentally incapable of "knowing" anything. It is a statistical engine, with no internal representation of the world or understanding of the words it emits.
  - Xelynega 4y ago
    They don't know it doesn't know
  - dTal 4y ago
    It clearly does have an internal representation of the world, implicitly encoded in its network weights. Quite an accurate one too, for most general knowledge, and one that can be updated in-context - it handily solves "blocks world[0]" style tasks. "Understanding" and "knowing" aren't helpful words, just focal points for pointless philosophical arguments. [0] https://en.wikipedia.org/wiki/Blocks_world https://en.wikipedia.org/wiki/Blocks_world
    - danaris 4y ago
      That's not a representation of the world. It's simply a lossy encoding of its data. It's not semantically structured in the way that our thoughts largely are—it's merely syntactically structured.
      - EnergyAmy 4y ago
        "That's not a representation of the world. It's simply a [representation of the world as viewed from its input data]" is an... interesting take.
      - dTal 4y ago
        What exactly is the difference? It's clearly managed to abstract the training data to a ludicrously deep degree, such that it's capable of solving semantically non-trivial problems it's never seen before. It can make metaphors, accurately predict the behavior of humans in complex social scenarios, and translate arbitrary passages between syntactically distinct languages while preserving nuance. That last task in particular is pretty much a slam dunk against any argument of the type you are making. "Sufficiently advanced syntax is indistinguishable from semantics."

sekai 4y ago

Yeah, you could call it a "human" characteristic

yoyohello13 4y ago

While you are right, I just don't think this is a useful comparison. There are other context clues when talking to people. I tend to know whether my colleague is talking out of his ass, and most people will preface stuff by saying they don't really know. We are trained to trust what computers are telling us, and ChatGPT doesn't 'qualify' what it's saying. I think if ChatGPT could preface what its saying with 'Well, I'm not super sure, but here is what I think' that would go a long way to solving this issue.

mhh__ 4y ago

So they're only as good as most humans in that context? No biggie...

garciansmith 4y ago

Someone with no experience may give you their best guess, but they'll also tell you that. And if I ask a colleague of mine for their sources on a particular matter that they might not be sure about (or I find curious), they would not send me a list of totally made up journal articles and book titles like ChatGPT does.

alfalfasprout 4y ago

Not sure why this is being upvoted but it's completely wrong. ChatGPT will answer like a colleague that is a subject matter expert on whatever you're asking them. And in doing so goes above and beyond inventing minute details that are blatantly wrong to pretend they're an expert. But the vast majority of folks have a concept of uncertainty and will either communicate (or know internally) that what they're saying is speculative. ChatGPT is like a pathological liar that pretends to be an expert and is often correct.