4 ms·

This as an assistant/teacher could be incredibly empowering for people learning to program. I'm sceptical of AI writing programs for people, but I can easily i

• layer8 4y ago
  It’s a double-edged sword. From the article: GPT-3 doesn’t actually know anything about anything at all. It’s a huge pattern generator. You can’t trust anything it says, because all it does is group words together into convincing looking shapes based on text that it’s seen before. Once again, I’m reminded that tools like GPT-3 should be classified in the “bicycles for the mind” category. You still have to know how to pedal! They’re fantastic tools for thinking, but to actually use their output effectively requires VERY deep knowledge—both of the subject matter in question, and of the way that the AI tools themselves work.
  • l33t2328 4y ago
    What is thinking if not pattern generating?
    • trention 4y ago
      It's not enough to state that the human mind is a pattern matcher as pretty much everyone of the "scale is all you need" crowd does. You have to prove it too. Otherwise it's just a variant of Russell's teapot.
      • abrax3141 4y ago
        Even if you prove it, it's not useful.
    • abrax3141 4y ago
      You aren't actually serious, are you?
    • Swenrekcah 4y ago
      We use far more variety of inputs than GPT-3. GPT-3 has more textual training data than any human could read in a thousand lifetimes, but we base our judgment on much more than that.
    • layer8 4y ago
      I agree that in a sense it’s probably only a matter of degree. However, take for example its explanation of the xm modifiers, where it only explains the x, but gives the impression that it explains both modifiers. It doesn’t seem to have any awareness that it provides an incomplete and potentially misleading explanation. It would be interesting to see how it reacts to “What you explained is only the x, what about the m?”.
    • skybrian 4y ago
      It's missing the part where you're not sure what the answer is, so you look it up, do an experiment, or ask someone. That's a bit much to ask of a fiction generator, though. It doesn't know it's not okay to make things up. All its training is about making things up whether it knows the answer or not.
      • xboxnolifes 4y ago
        > It doesn't know it's not okay to make things up. All its training is about making things up whether it knows the answer or not. Sounds very human to me.
        • skybrian 4y ago
          Humans are sometimes curious enough to look for answers.
          • solardev 4y ago
            This model has already spent more time learning than most humans ever would... Imagine if we gave it more power and funding. Maybe redirect all of Congress's salary to it?
      • goodside 4y ago
        Look into RETRO, which greatly reduces the model’s tendency to confabulate by teaching it to query a document database known to be truthful, and justify its answers with specific references: https://www.deepmind.com/publications/improving-language-models-by-retrieving-from-trillions-of-tokens https://www.deepmind.com/publications/improving-language-mod...
      • skjoldr 4y ago
        That looks like "if confidence < %some number%, then try to expand the dataset using known related sources". Load a library of short book descriptions into such an AI, and it should reproduce this behavior. Though it also needs the capacity to learn. Otherwise you could put a snippet that automatically queries it for better materiel about the subject when its confidence is low, so it just tells you where it thinks you should look. That seems doable now.
    • shadowofneptune 4y ago
      Things we have that current AI models do not: Intent: We can have a goal which directs our thinking. This changes what we come up with much more flexibly than experience, even if experience is often needed for good results. Temperment: We can be angry, tired, excited, relaxed, etc. while thinking. This isn't always good, but it is a way we differ. Self-awareness: A very difficult term to define, but we can (hopefully) take a step back and discard our current thought if we realize we are falling into one of our usual bad patterns of thinking.
      • skjoldr 4y ago
        Intent and self-awareness do come from experience. The goals we want to achieve are tied to what has been going on throughout our lives thus far. We don't invent our own values out of a vacuum, we tend to adopt them from other people, and rank them depending on how our nervous system works. Self-awareness is the ability to reason about the process of reasoning, plus, in your example, a recollection of past examples of incorrect reasoning that disrupted the process of achieving our goals. So basically current models need a memory and the ability to change their own weights in response to new data to even start approximating humans, but that looks like a doable task, in principle.
    • astrobe_ 4y ago
      Indeed the brain is pretty good at this, so much so that it is annoying when you are trying to generate a variety of textures by combining a small set of textures. This of course extends to audio patterns, temporal event patterns (e.g. post hoc, ergo propter hoc) etc. That may be the cornerstone of "intelligence", but that's only one part of it. I think another one is making or finding relations between the parts of the whole. Life is the continuous adjustment of internal relations to external relations [1] Once you have internalized relations of the real world, you can start to hack them in your head, run simulations of the results and take action. [1] https://www.thoughtco.com/famous-education-quotations-herbert-spencer-31420 https://www.thoughtco.com/famous-education-quotations-herber...
    • Double_Cast 4y ago
      My take is that pattern-matching is thinking. But it's low-quality thinking. High-quality thinking is logic. And higher still is causality, which is to logic what calculus is to algebra. I.e. if logic studies the relationship between x and y, then causality studies the relationship between dx and dy. And causality is what we actually want, because causality is power. E.g. causality is what lands astronauts on the moon. When folks like Judea Pearl complain that current AI isn't truly thinking, they're complaining that current AI can't reason logically/causally.
  • duskwuff 4y ago
    And, in particular, GPT-3 will be very easily misled by plausible-but-incorrect inputs -- like programs written by amateur developers. I definitely wouldn't trust it to find subtle logical errors in a student's computer program, for example.
  • Transfinity 4y ago
    The types of error made by the model in the article are exactly the sort of error that I (senior engineer / decade of experience) would make if asked to describe how something works and I didn't want to spend too much time on the details. The difference is, I can usually give a decent estimate of how sure I am of something, and know how to zero in on the truth when it's important. Maybe a confidence level for a given explanation, along with some sort of "here's where you can go to learn more" would be useful? No idea if language models would be good at that kind of meta reasoning.
    • ShamelessC 4y ago
      There’s already a per-token confidence score given by GPT-3 actually. As for citing actual sources, retrieval architectures like RETRO are becoming popular for this as they can tell you which sources they used for the current prediction and are actually faster thanks to including a fast KNN lookup over the dataset.
    • meken 4y ago
      Dunning-Kreuger notwithstanding? ;)
• AkshatJ27 4y ago
  I agree, while i do not think AI is at the level where it can replace developers, and i doubt it will ever be, I am sure that AI can increase productivity and make development easier, like copilot