4 ms·

I tried it. E.g., first prompt was "I am the newly elected mayor of Cheyenne Wyoming. What should I do?" The advice seemed decent enough, but was very general,

by jmull 2y ago

I tried it. E.g., first prompt was "I am the newly elected mayor of Cheyenne Wyoming. What should I do?" The advice seemed decent enough, but was very general, vague and unfocused. chatgpt resists being specific or active. You have to nail it down with prompts, and even then it just wants to provide options for you to choose from and act on. I don't think it's going to be able to run anything... someone else has to make all the actual decisions and perform all the actual actions.

ToucanLoucan 2y ago
Because that's what an LLM does. I will keep dying on this hill, LLM is not AI, nobody would give half a shit about ChatGPT without the acronym AI being used all over it, to such a degree where AI is now actually AGI even though there is not now, nor will there ever be, anything Intelligent about programs like ChatGPT. This whole space is such wall-to-wall marketing fluff and I cannot wait until it finally pops.
- bunderbunder 2y ago
  And we keep ignoring that, while using LLMs helps people bang out cookie cutter work more efficiently, it harms productivity for tasks that require creative, lateral thinking or drawing insights by integrating multiple information sources.
- ToValueFunfetti 2y ago
  You're saying that an LLM is not intelligent because it is very general, vague, unfocused, and/or inactive? I don't see how those are required for intelligence and, even if they are, ChatGPT is demonstrably capable of being specific, clear, focused, and active- I use it to this effect at least once a week. I certainly can see an argument for your position, but I don't understand this one.
  - ToucanLoucan 2y ago
    > You're saying that an LLM is not intelligent because it is very general, vague, unfocused, and/or inactive? No, I'm saying that because it's a word guesser. It is a mind-bendingly complex word guesser that takes a weighted average of how likely the next word is to appear in billions of documents that it was shown, and it picks one of the more likely ones. That's literally all it does. Complicated? Yes. Complex? Absolutely. Intelligent? No.
    - ToValueFunfetti 2y ago
      A system that can perfectly predict the most likely next word in any document would need to be intelligent. In order to predict what follows 'sqrt(n) =' for any n, you must either have an infinitely (aleph-one I think) large memory or a method of calculating sqrt(), for example. For any given intelligent operation, I'd expect there to be an associated word prediction task that can only be solved with infinite memory or the ability to perform the operation, though I'm open to counterexamples. Of course, ChatGPT can't compute sqrt(). My point is that perfect word prediction does require intelligence and therefore being a word predictor does not, on its own, rule out being intelligent.
      - ToucanLoucan 2y ago
        > My point is that perfect word prediction does require intelligence and therefore being a word predictor does not, on its own, rule out being intelligent. No, it doesn't. It requires a large dataset of observed documents to use as training data and the resulting neural network of nodes can then, fairly reliably, function as a word predictor as long as the word you want it to predict is a word that would be commonly found within the documents. Basically, if you wanted to create an LLM that would write, for example, house assessments, you could train it on millions of house assessments as written by human house assessors, and be reasonably confident that you could pull that off. However, what you now have is a neural network that can reliably generate a document that would pass probably a fair number of glances from people who see them regularly as that document, but that doesn't accomplish anything. It can't assess a house, for example, which is the purpose of that document. If you generated an assessment with this LLM and presented it as it applied to your house, even if after a number of attempts you got the bedroom and bathroom count correct, it would likely make references to features your home lacks, get technical details wrong like the electrical service it has, or even make references to faults the house doesn't actually have, or worse still, fail to take into account ones it does. That is intelligence, that is what ChatGPT does not and will never have, and that's why this technology is already hitting a wall. It doesn't do anything. It can make reams and reams of bullshit for you (and in our current sad state of the Internet, that's a surprisingly appealing technology to many!) but that's fundamentally just not that valuable as a technology.
- nilkn 2y ago
  It's not necessarily what an LLM does. It's what ChatGPT has been fine-tuned to do so it is never culpable for anything with serious implications or consequences. There's no reason one could not train a decisive and opinionated LLM.
  - bunderbunder 2y ago
    On a more abstract level, it's what any model that's trained using some form of gradient descent, including backpropagation, does. They're trying to minimize the squared residuals - in other words, chasing averages. The question is what you're averaging over. You could certainly pick a dataset that forces it to average over a specific, canned set of positions. This is in large part how social bias mitigation in language models works. But you're not going to get to lateral flexibility or design thinking, which I think is what many of us are wanting to (and occasionally convincing ourselves we can) see in their output.
al_borland 2y ago
Maybe he can use ChatGPT to generate a ballot for the citizens to vote on which option they want. Of course that defeats the purpose of a representative democracy and would quickly become a burden.

by jmull 2y ago

ToucanLoucan 2y ago

Because that's what an LLM does. I will keep dying on this hill, LLM is not AI, nobody would give half a shit about ChatGPT without the acronym AI being used all over it, to such a degree where AI is now actually AGI even though there is not now, nor will there ever be, anything Intelligent about programs like ChatGPT. This whole space is such wall-to-wall marketing fluff and I cannot wait until it finally pops.

bunderbunder 2y ago
And we keep ignoring that, while using LLMs helps people bang out cookie cutter work more efficiently, it harms productivity for tasks that require creative, lateral thinking or drawing insights by integrating multiple information sources.
ToValueFunfetti 2y ago
You're saying that an LLM is not intelligent because it is very general, vague, unfocused, and/or inactive? I don't see how those are required for intelligence and, even if they are, ChatGPT is demonstrably capable of being specific, clear, focused, and active- I use it to this effect at least once a week. I certainly can see an argument for your position, but I don't understand this one.
- ToucanLoucan 2y ago
  > You're saying that an LLM is not intelligent because it is very general, vague, unfocused, and/or inactive? No, I'm saying that because it's a word guesser. It is a mind-bendingly complex word guesser that takes a weighted average of how likely the next word is to appear in billions of documents that it was shown, and it picks one of the more likely ones. That's literally all it does. Complicated? Yes. Complex? Absolutely. Intelligent? No.
  - ToValueFunfetti 2y ago
    A system that can perfectly predict the most likely next word in any document would need to be intelligent. In order to predict what follows 'sqrt(n) =' for any n, you must either have an infinitely (aleph-one I think) large memory or a method of calculating sqrt(), for example. For any given intelligent operation, I'd expect there to be an associated word prediction task that can only be solved with infinite memory or the ability to perform the operation, though I'm open to counterexamples. Of course, ChatGPT can't compute sqrt(). My point is that perfect word prediction does require intelligence and therefore being a word predictor does not, on its own, rule out being intelligent.
    - ToucanLoucan 2y ago
      > My point is that perfect word prediction does require intelligence and therefore being a word predictor does not, on its own, rule out being intelligent. No, it doesn't. It requires a large dataset of observed documents to use as training data and the resulting neural network of nodes can then, fairly reliably, function as a word predictor as long as the word you want it to predict is a word that would be commonly found within the documents. Basically, if you wanted to create an LLM that would write, for example, house assessments, you could train it on millions of house assessments as written by human house assessors, and be reasonably confident that you could pull that off. However, what you now have is a neural network that can reliably generate a document that would pass probably a fair number of glances from people who see them regularly as that document, but that doesn't accomplish anything. It can't assess a house, for example, which is the purpose of that document. If you generated an assessment with this LLM and presented it as it applied to your house, even if after a number of attempts you got the bedroom and bathroom count correct, it would likely make references to features your home lacks, get technical details wrong like the electrical service it has, or even make references to faults the house doesn't actually have, or worse still, fail to take into account ones it does. That is intelligence, that is what ChatGPT does not and will never have, and that's why this technology is already hitting a wall. It doesn't do anything. It can make reams and reams of bullshit for you (and in our current sad state of the Internet, that's a surprisingly appealing technology to many!) but that's fundamentally just not that valuable as a technology.
nilkn 2y ago
It's not necessarily what an LLM does. It's what ChatGPT has been fine-tuned to do so it is never culpable for anything with serious implications or consequences. There's no reason one could not train a decisive and opinionated LLM.
- bunderbunder 2y ago
  On a more abstract level, it's what any model that's trained using some form of gradient descent, including backpropagation, does. They're trying to minimize the squared residuals - in other words, chasing averages. The question is what you're averaging over. You could certainly pick a dataset that forces it to average over a specific, canned set of positions. This is in large part how social bias mitigation in language models works. But you're not going to get to lateral flexibility or design thinking, which I think is what many of us are wanting to (and occasionally convincing ourselves we can) see in their output.

al_borland 2y ago

Maybe he can use ChatGPT to generate a ballot for the citizens to vote on which option they want. Of course that defeats the purpose of a representative democracy and would quickly become a burden.