6 ms·
You note that this will change the world, but then talk about GPT-4 specifically. The leap from GPT-2 to 3 was enormous. 3 to 4 was enormous, and we’re not eve
by sillysaurusx 4y ago
You note that this will change the world, but then talk about GPT-4 specifically.
The leap from GPT-2 to 3 was enormous. 3 to 4 was enormous, and we’re not even using 32k context yet nor image input. 4 to 5 will likely be as disruptive if not more.
This isn’t about 4. We’re in the iPhone 1 era of LLMs. This is about what the world will look like in one or two decades. And there’s a good chance this comment might age poorly.
That’s a scary thought. I was skeptical of AI, and still am. But it seems undeniable that the world is in for a big awakening. This might be as big of a transformation to society as the introduction of microprocessors.
- alchemist1e9 4y ago> one or two decades More like one or two years at this rate.
- hammyhavoc 4y agoRemember the crypto hype cycle?
- alchemist1e9 4y agoAnd how is that going? Died down? It’s really amusing when I encounter tech savvy individuals who think crypto is hype, little do they realize it’s stronger than ever and central bankers are becoming increasingly concerned.
- deleted 4y ago[deleted]
- UncleEntity 4y agoConcerned the scams and fraud might bleed over into the real economy.
- alchemist1e9 4y agohttps://www.coindesk.com/policy/2023/03/24/federal-reserve-says-custodias-crypto-focused-business-model-is-inconsistent-with-approval/ https://www.coindesk.com/policy/2023/03/24/federal-reserve-s... I don’t think that’s what they are actually worried about. I would also like to point out that the biggest scams, FTX for example, are simply traditional Ponzi schemes with a crypto front, they have all been executed entirely using regular banking systems and due to the incompetence of those regulators. Bitcoin itself is rock solid and constantly gaining users and influence.
- hammyhavoc 4y agoI invoked the crypto hype cycle, not crypto itself. Don't conflate the two topics of discussion.
- alchemist1e9 4y agoUnderstood. However I’d still argue there is a reason for the hype and I believe that is the case with the GPT LLMs also.
- idopmstuff 4y agoExcept crypto lacked utility for most people, even early on. I've spent the last couple of days creating python scripts to automate parts of my business. I'm not a developer (though technical enough to help point GPT in the right direction sometimes when it's getting stuck on problems) and have written <100 lines of python in my life. I'm using image generation AI regularly to create images for my marketing emails, and when I've got writer's block it helps with the text too. Right now the iPhone 1 is a great analogy - it was cool but it was really subpar for using a lot of the internet, because it wasn't mobile optimized. GPT takes some coaxing to get it where you want, like you had to do a lot of pinching to zoom in on websites on your phone. In a few generations, this is going to be as seamless to use as the iPhone 5 was compared to the first gen.
- krainboltgreene 4y ago> Except crypto lacked utility for most people, even early on. Every crypto person said the opposite. They said exactly what ChatGPT-hype people are saying now.
- eli 4y agochatgpt has already provided more value to users than all crypto combined
- ChatGTP 4y agonot if you're a teenager who is into using banned substances on the weekend
- UncleEntity 4y agoWhen I was a teenager, back in the 80s, it didn’t take magic internet money to buy drugs. In fact, every drug buy I ever heard of (second hand, of course) involved fiat currencies.
- ChatGTP 4y agoYou are no longer a teenager in the 80s my friend.
- hammyhavoc 4y agoSource?
- dpkirchner 4y agoWhat sort of source are you looking for? What would satisfy your question?
- hammyhavoc 4y agoA source that quantifies value with cited examples and statistics. I ask because the crypto crowd argue there's utility and value there too.
- sillysaurusx 4y agoYet the decades will come. I think progress is sigmoidal rather than exponential, and it’s very hard to tell the difference in the early stages. But even sigmoidal progress with smartphones was enough to completely upend online society. We adapted, of course, but it looks nothing like it did in 2003. We’re all still using the internet; that’s basically it. Point is, it could slow down, assuming that AGI isn’t waiting like a cat in a corner. But it’ll still displace a tremendous amount of intellectual work.
- hattmall 4y agoThis sentiment pops up with most somewhat new technology, but in my experience the plateaus come quickly. Going with the iphone. The first was transformative, and it continued to improve but the functional difference between iPhone 4 to iPhone 14 is not that great. Think of the wow factor of showing one or the other to someone from 100 years ago.
- thethimble 4y agoThe entrenchment of smartphones in society dramatically increased between iPhone 4 and 14. Technical capability is just one axis. Still, I think LLMs are different than phones in terms of scaling. Faster processor speeds don’t necessarily result in more user value for phones, but scaling up LLMs seem to predictability improve performance/accuracy.
- bryanrasmussen 4y ago>but scaling up LLMs seem to predictability improve performance/accuracy. what exactly is performance/accuracy in slogan generation?
- je42 4y agogpt does a lot more than slogans... currently using it like driving a junior programmer. after gpt has written some functions to my specs in natural language. I can say for example: - "add unit tests". It writes for all functions tests. Not perfect but not bad for short instruction like this. - rewrite x to include y etc
- bryanrasmussen 4y agothe original post way back was talking about marketing, they were underwhelmed. I recently generated some slogans. They sucked. When someone mentioned predictability/accuracy how does that apply to marketing slogans. I know how it applies to writing unit tests. The unit tests writing comes pretty close to the original posters definition of GPT as filling out templates. The sucky slogans I got were also very template like. Would accuracy be if slogans did not suck? At any rate there seems to be a lot of things people want to use it for where the terms accuracy / predictability don't make much sense. So making claims based on those qualities naturally causes me to ask how do they apply to all these cases - such as slogan generation where accuracy predictability are not normally metrics that apply.
- ben_w 4y agoI wasn't impressed or motivated by the original iPhone when it was new. I don't know about the future, but by analogy with the past I would say that GPT-3 was the original iPhone (neat tech demo but I didn't really care), ChatGPT is the iPhone 3G, and GPT-4 is the 3GS. Looking at the sales graphs on Wikipedia (Q1 2012) I think it took until the 4S to transition from "the next big thing" to "the big thing". Analogies only rhyme rather than replicate, so don't assume GPT needs exactly two more versions to do the same; might be more, might be less, and Uncanny Valley might trigger a Butlerian Jihad at the last possible moment before fully-general AGI.
- ghaff 4y agoI would have said the 3GS but, yeah, sometime around then. Similarly, the original iPod was a not obviously remarkable pocket music player in a fairly crowded field.
- croes 4y agoWasn't the leap from GPT-2 to GPT-3 bigger than from GPT-3 to GPT-4? Like self driving cars, the leap were enormous until they hit a wall and we still don't have full self driving
- UncleEntity 4y agoIf someone were to ask me (which nobody ever does) GPT-4 was more about cashing in on the hypetrain than pushing the boundaries. A little better but a lot “safer” to cut down on the articles on how it’s trying to steal someone’s man.
- macrolime 4y agoThe first few days I didn't think there was much difference, but after using GPT-4 a lot I think the leap is huge. For things where I would previously use Stack Overflow or some other Stack Exchange, I now use GPT-4 and get a working answer almost every single time, with GPT-3.5 the frequency of working answers was much smaller.
- jasmer 4y agoDecreasing marginal returns though. We were a bit blown away with 'Siri' - I mean, it could understand what you said, and 'get the weather'. I think we're going to start feeling the limits of this soon. It will be pervasive though.
- bitwize 4y agoI have a sinking suspicion we're not in the iPhone era of LLMs, we're in the "in mice" era of LLMs. I can't elucidate why, but this strikes me as the sort of thing that could either blow up (like the iPhone) or fizzle out (like Theranos), or blow up and then fizzle out (like crypto). And it's too early to know yet which it'll be. Hackernews is making lots of change-everything predictions, but Hackernews was like that about Theranos, crypto, and self-driving cars too.
- ModernMech 4y ago> 3 to 4 was enormous... This isn’t about 4. We’re in the iPhone 1 era of LLMs. GPT3 is great, but I can't reasonably say that 4 is such a huge advance over 3 in my experience so far. Apparently it's better at some things according to the marketing, but for actual usage I can't qualitatively say 4 is an "enormous" advance over 3. It seems to face the same major shortcomings, and it produces qualitatively the same results. That brings me to the iPhone bit. Yes, the iPhone was a huge advance, but today looking at an iPhone 14, it largely has the same form/function/features as early iPhones. If you looked at the trajectory of iPhones in 2005, you'd conclude that in 2023 they would be 1mm think and transparent with a holodisplay or something. But instead, in the year 2023, my iPhone 14 looks and functions largely like my old iPhone 4. I mean, it does more stuff better, but I'm still using it to browse the net, text, take pictures, and use the maps app -- the same stuff that made the original iPhone revolutionary.
- coldtea 4y ago>This isn’t about 4. We’re in the iPhone 1 era of LLMs. Well, on the other hand, iPhone 14 isn't that different. Same how a 60s car and a modern Tesla aren't that different. Evolutionary marginally better yes. More convenient, yes. But nothing life changing or necessary. Which is why some folks can even get by reverting to a dumb phone (whereas they wouldn't dream of going pre-electricity or pre-antibiotics). Also, we were hearing the same about VR in the early 90s, and again in the mid 2010s. Still crickets.
- ROTMetro 4y agoHuh? The iPhone 1 was a toy and lots of people laughed at the users. Today a modern phone is a requirement to be a member of society. It is how I pay for things. It is needed for most of my interactions with friends/family. It is the diary of my life, and the repository of my good memories with its near unlimited video/image storage at a quality only dreamed of when the iPhone 1 came out. Take away a person's iPhone1 and they weren't phased much, taking away a person's iPhone 14 is a LIFE CHANGING experience today. In 10 years taking away your AI will be on the same level, you will function but at a much more frustrating level.
- coldtea 4y ago>Huh? The iPhone 1 was a toy and lots of people laughed at the users. Nothing "toy" about it, it was the most advanced phone on the market. The people who laughed were just the handful of idiots that would laugh because "Apple, har har har" and then go buy the same thing from another vendor. The same kind of Zune buying crowd. >Today a modern phone is a requirement to be a member of society. You'd be surprised. >It is how I pay for things. It is needed for most of my interactions with friends/family. It is the diary of my life, and the repository of my good memories with its near unlimited video/image storage at a quality only dreamed of when the iPhone 1 came out. None of those are essential, even for a 21st century level lifestyle, some of those are indulgent, others are detrimental. In any case, nothing revolutionary, except if one thinks "I can pay by taking out my phone and pointing it at the gizmo at the cashier" is something far great than "I can pay by getting out my credit card and pointint it at the gizmo at the cashier" (or, god forbid, giving cash and not being tracked).
- eternalban 4y agoYou are working under an assumption that this tech is an O(n) or better computational regime. Ask ChatGPT: “Assume the perspective of an expert in CS and Deep Learning. What are the scaling characteristic (use LLMs and Transformer models if you need to be specific) of deep learning ? Expect answer in terms of Big O notation. Tabulate results in two rows, respectively “training” and “inference”. For columns, provide scaling characteristic for CPU, IO, Network, Disk Space, and time. ” This should get you big Os for n being the size of input (i.e. context size). You can then ask for follow up with n being the model size. Spoiler, the best scaling number in that entire estimate set is quadratic. Be “scared” when a breakthrough in model architecture and pipeline gets near linear.
- gremlinsinc 4y agomicroprocessors? Geoffrey Hinton(I think that's his first name), the grandfather of ai recently said this is like the invention of the wheel, agriculture, or electricity. ie even more foundational for everything that's coming in the future. ai will be as essential as electricity.