7 ms·

If even half of what we're hearing about GPT 4 comes to fruition then we're going to know in the next year or two tops if the scaling hypothesis is true or not.

• smy20011 4y ago
  2 AI winter so far!
• siva7 4y ago
  AI winter isn't an issue anymore and won't be - the breakthrough in public perception already happened for its first time at this scale.
  • russdill 4y ago
    I am curious if AI is successful if an AI backlash is coming. Perhaps one of the existing political parties or a new political party taking up the mantle of regulating or outright banning of AI. Or even just attacks on AI researchers and institutions.
    • ethanbond 4y ago
      I mean… the fear/goal of AI success, depending on who you ask, is social and economic upheaval. I suspect there will be a lot of resistance as it “succeeds.”
      • russdill 4y ago
        Could make "AI revolution" take on an entirely different meaning
      • pksebben 4y ago
        it's already happening. Stable diffusion forums regularly get brigaded, and there are lawsuits in the works against stability ai and midjourney. Those lawsuits are nominally about copyright, but one gets the sense that it's really about putting the breaks on developing the tech. I kind of get it, too. If I made my money making art this would scare me. What I wonder is what's gonna happen when it's [ artists, accountants, copywriters, bloggers, designers, programmers, etc, etc, etc]. I see problems coming.
        • p1esk 4y ago
          The resistance if futile. The amount of effort currently going into making generative models like Stable Diffusion or ChatGPT is insane. It's the fastest changing subfield in ML. These models are getting better and better every month, with no sign of plateauing in the near future. I expect decent music and video generation models appearing within the next 2 years. There's no way to stop the progress.
  • ethanbond 4y ago
    Why is that a perception question rather than a delivery question? We still don’t really know how/if the current crop of AI is going to deliver. Even if there are a lot of great applications for these things (I think there are), it’s not clear they’re going to be truly transformative?
    • osigurdson 4y ago
      >> truly transformative For me, it is already pretty transformative. I use ChatGPT at least as much as the regular internet - probably more.
      • ethanbond 4y ago
        Out of curiosity: to do what? Not just as a hobby/toy?
        • recursive 4y ago
          Today it told me how to write a powershell command to make a git commit that includes the current timestamp in the commit message. For example. I could have figured it out elseways, but would have taken >20x the time.
          • ethanbond 4y ago
            I don’t see how this type of usage (which is real/valuable) adds up to “as much or more” than the actual internet.
            • p1esk 4y ago
              "The actual internet" allowed people to get information 20x times faster than before (e.g. vs physically going to a library, or writing a letter to an expert). If we now have a technology that lets us get answers 20x times faster than what we had before it, then yes - I see this technology as a huge advancement over the "old" internet.
        • 4y ago
  • dangero 4y ago
    Public interest wanes quickly if the technology doesn’t continue to be magic https://en.m.wikipedia.org/wiki/Gartner_hype_cycle https://en.m.wikipedia.org/wiki/Gartner_hype_cycle
  • spion 4y ago
    Its not the first time. Chess was the first time.
• BulgarianIdiot 4y ago
  What kind of a "winter" would this be when literally the world already runs on AI? Search engines run on AI, stock markets trade on AI, social networks distribute content by AI, your photos and content are sorted on your phone by AI, vaccines and medicines are produced by AI, precedent law is implemented increasingly via AI, our weapons are AI (drones) and so on and so on and so on. And all this up there was BEFORE GPT and DALL-E. No, no winter. We're in the singularity.
  • deleted 4y ago
    [deleted]
  • anticensor 4y ago
    Technological singularity is the point no one is able to predict what happens after the point from what comes before. We have not arrived that yet. And yes, strong AI will pave road to technological singularity.
    • BulgarianIdiot 4y ago
      If the definition of singularity has such low requirements, I think we've probably reached singularity somewhere back at the dawn of human civilization when we started making lots of predictions, for sure, and the overwhelming majority of them terrible. We carry this old tradition with ourselves to this day.
  • woeirua 4y ago
    You're using the term AI pretty generously there. Let's not delude ourselves into believing that deterministic algorithms are general artificial intelligence.
    • BulgarianIdiot 4y ago
      I didn't say "general" artificial intelligence. Nor did I refer to "deterministic algorithms".
    • deleted 4y ago
      [deleted]
• midlightdenight 4y ago
  This has always made me curious about multi modality and especially Google’s Palm. At least how google presents Palm in diagrams. The way I’ve interpreted the scaling hypothesis is that we will see emergent intelligence with larger nets through automated training alone. If we want a model to learn images we throw a larger net at it with training data. The way I’ve interpreted some of these newer techniques to multi modality is that they are stitched together models. If we want a model to learn images we decide this and teach one model images and then connect it to the core model. There’s not a lot of emergent behavior due to scaling in this scenario. With that perception I don’t see how gpt4 says anything about the scaling hypothesis. However I am not in this field and would be grateful to learn more.
  • concinds 4y ago
    The brain is composed of distinct regions that specialise in specific tasks. It's reasonable to assume AGI would be the same. So the goal should be: we've created a "language module" (LLMs) and a "visual perception module" (computer vision), but we also need to add a "logic module", a "reasoning module", an "empathy module", etc, while continuing to improve each. I just don't see how you could get an LLM, no matter how advanced, to recognize a car. Even if it can describe cars (wheels, windshield, doors) it doesn't know what any of those components look like. It's like that old joke about philosophers being unable to define a chair beyond "I'll know it when I see it".
    • avereveard 4y ago
      it's even more foundamental than that, the brain is constantly learning and retaining new knowledge and solve problems with a mix of old knowledge and knowledge learned in the context of the problem, either by experimentation or research these net can replicate at most the first step for now, even tuning by reingofrcement learning is more of a set up batch than an ongoing thing, and certainly not something they will be able to do in the context of a single problem but as part of a retraining agi is still a fair bit away, I'm unsure if these super large architecture will ever get to replicate the second part of our brain, the flexibility while on the job, because of their intrinsic training mechanism.
• osigurdson 4y ago
  I recall watching a video where Sam Altman was downplaying expectations for GPT4 so I'm not expecting Star Trek. However, I don't think we are due for another AI winter anytime soon. Even if technology were to plateau at GPT3.5 for several years it would not constitute a typical AI winter as I don't think we've found the edges of the current technology yet.
  • tpoacher 4y ago
    surely you mean GPT3.1 followed by GPT95, 98, XP, etc
  • woeirua 4y ago
    We can have an AI "winter" without actually having a solid freeze in terms of development if expectations fail to match the reality. Right now, the hype around ChatGPT is unwarranted in my opinion, and if nothing else comes from it we will end up in at least a cooling off period as people realize that.
    • p1esk 4y ago
      A lot of people find Siri (and other AI assistants) helpful, and use it every day. Some would even say Siri is good enough for the limited set of tasks it currently performs. ChatGPT is like Siri on steroids - it's much better in every way, with the important caveat that it - rarely - can provide a wrong/bad answer. If Siri is level 2 autonomous driving, then ChatGPT is full level 3. People hope that GPT-4 will be equivalent to level 4 driving, but I don't think anyone has promised that. Most of the hype I hear today is about what ChatGPT can already do (more or less). I actually think most people haven't even caught up with what it can do today.
• zeofig 4y ago
  Spoiler: it's false
• concinds 4y ago
  I've looked for papers, but can't figure this out: do our current neural networks (LLM, vision) qualify as complex systems? Are they composed of autonomous agents that exhibit: - diversity (in behavior) - high connectedness - interaction among agents - adaptation (agents changing their behavior based on the behavior of other agents)? Because if current NNs don't fit that definition of complexity, they it seems unreasonable to expect them to be intelligent, no matter their scale. Intelligence will likely be an emergent property of the above four criteria. If anyone knows of relevant books or papers, would love to hear it. BTW: "AI winter" refers to a collapse in funding; but I think we're well past that point. Previous winters only happened because academia led AI research, and there were no commercial opportunities. Now, the economic benefits are far more tangible. And if funding doesn't dry up, there's no reason to think innovation will dry up either.
  • pksebben 4y ago
    they're gonna. I've seen a lot of material on wiring these models together to have them do more interesting cross-functional work. Microsoft is about to put out a model you can talk to that makes and recognizes images - and there's already multi-model frameworks that do things like speech-to-text to LLM to text-to-speech, which means you can hold a verbal conversation. Once the continuous learning problem is solved, it'll be pretty trivial to just tell your compound model to "go learn everything you can about x" and have it autonomously do so. If it were me, the first thing I'd ask it to do was learn how to make other AI, then give it some hardware and the directive to build the best thing it could and test it until it can build a model better than itself.
• jokoon 4y ago
  Artificial intelligence will always be artificial. If science can neither have a better definition of intelligence, or analyze and understand the resulting black box of a neural net, I don't see how AI would improve or get nearer general intelligence. I have much more curiosity about neurology and how psychology define intelligence, than how programmers are trying to build things that digest data. Proper science requires understanding. ML is not about understanding, it's only "sophisticated statistical methods". It is still artificial, thus but genuine.