14 ms·
Interesting, both Karpathy and Sutskever are gone from OpenAI now. Looks like it is now the Sam Altman and Greg Brockman show. I have to admit, of the four, Ka
by zoogeny 2y ago
Interesting, both Karpathy and Sutskever are gone from OpenAI now. Looks like it is now the Sam Altman and Greg Brockman show.
I have to admit, of the four, Karpathy and Sutskever were the two I was most impressed with. I hope he goes on to do something great.
- yu3zhou4 2y agoJakub Pachocki is still in OpenAI though
- gdiamos 2y agoI don’t think people give Dario enough credit
- deleted 2y ago[deleted]
- HarHarVeryFunny 2y agoYeah, I think him leaving was a huge blow to OpenAI that they have maybe not yet recovered from. Clearly there is no moat to transformer-based LLM development (other than money), but in terms of pace of development (insight as to what is important) I think Anthropic have the edge, although Reka are also storming ahead at impressive pace.
- nabla9 2y agoTop 6 science guys are long gone. Open AI is run by marketing, business, software and productization people. When the next wave of new deep learning innovations sweeps the world, Microsoft eats whats left of them. They make lots of money, but don't have future unless they replace what they lost.
- fnordpiglet 2y agoI don’t feel that OpenAI has a huge moat against say Anthropic. And I don’t know OpenAI needs Microsoft nearly as much as Microsoft needs OpenAI
- gunalx 2y agoI Don't know. Being able to get azure credits has payed out really well for openai as a business in constant need of computer.
- fnordpiglet 2y agoWhich is a very short term advantage. And Anthropic gets aws credits which would you rather have?
- Closi 2y agoNever discount the value of short term advantages. Being first at the start (i.e. first mover advantage) is huge.
- luma 2y agoGiven Amazon's no show in the AI space? Azure. By a mile.
- cthalupa 2y agoExcept if you're Anthropic or OpenAI you don't care about what your compute provider has done in the AI space - you care about the compute power they can give you.
- luma 2y agoThat's exactly what I'm talking about: https://i.imgur.com/sZ3tniY.jpeg https://i.imgur.com/sZ3tniY.jpeg
- cthalupa 2y agoBut how many of those are ordered specifically for OpenAI, and are on order as a result of them to begin with? Do you think if we were in a parallel universe where OpenAI ended up partnering with Google or Amazon instead, the GPU shipments would look the same? I think they would reflect wherever OpenAI ended up doing all their compute showing a pretty similar lion's share. Your claim was that people should care about compute based on what the provider has done in the AI space, but Microsoft was pretty far behind on that side until OpenAI - Google was really the only player in town. Should they have wanted GCP credits instead? Do you care about their AI results or the ex post facto GPU shipments? Or, if what you actually want to argue is that Anthropic would be able to get more GPUs with Azure than AWS or GCP then this is a different argument which is going to require different evidence than raw GPU shipments.
- ralfd 2y agoHow important are top science guys though? OpenAI has a thousand employees and almost unlimited money, and llm are better understood, I would guess continous development will beat singular genius heroes?
- mrklol 2y agoThey definitely need them to find new approaches which you won’t find normally.
- AdamN 2y agoAgreed - it's good to have some far thinking innovation but really that can be acquired as needed so you really just need a few people with their pulse on innovation which there will always be more of outside a given company than within it. Right now it's all about reducing transaction costs, small-i innovating, onboarding integrations, maintaining customer and stakeholder trust, getting content, managing stakeholders, and selling.
- benterix 2y ago> OpenAI has a thousand employees and almost unlimited money You could say the same about Google - and yet they missed the consequences of their own discovery and got behind instead of being leaders. So you need specific talent to pull this off even if in theory you can hire anybody.
- wg0 2y agoI am just curious how it happened to Google? Like who were the product managers or others who didn't see an opportunity here exactly where the whole thing was invented and they had huge amounts of data already, whole web basically and the amount of video that no one else can ever hope to have?
- andy99 2y agoData volume isn't that important, that's becoming clearer now. What OpenAI did was paid for a bunch of good labelled data. I'm convinced that's basically the differentiator. It's not a academic or fundamental thing to do which is why google didn't do it, it's a pure practical product thing.
- deleted 2y ago[deleted]
- loldomsa 2y agoI honestly think that is the best course of actions for humanity. Even less chance to see AGI anytime soon if he leaves.
- dkjaudyeqooe 2y ago> Open AI is run by marketing, business, software and productization people. AKA 'the four horsemen of enshitification'.
- chx 2y ago> When the next wave of new deep learning innovations sweeps the world, that won't happen, the next scam will be different it was crypto until FTX collapsed then the usual suspects led by a16z leaned on OpenAI to rush whatever they had on market hence the odd naming of ChatGPT 3.5. When the hype is finally realized to be just mass printing bullshit -- relevant bullshit, yes, which sometimes can be useful but not billions of dollars of useful -- there will be something else. Same old, same old. The only difference is there is no new catchy tunes. Yet? https://youtu.be/I6IQ_FOCE6I https://youtu.be/I6IQ_FOCE6I https://locusmag.com/2023/12/commentary-cory-doctorow-what-kind-of-bubble-is-ai/ https://locusmag.com/2023/12/commentary-cory-doctorow-what-k...
- trashtester 2y agoCrypto currencies has the potential to grow the world economy by about 1-3%, as banking fees go down. Add other uses of crypto may double or triple that, but that's really speculative. AI, on the other hand, has a near infinite potential. It's conceivable that it will grow the global economy by 2% OR MORE per MONTH for decades or more. AI is going to be much more impactful than the internet. Probably more than internal combustion, the steam engine and electricity combined. The question is about the timescale. It could take 2 years before it really starts to generate profits, or it could take 10 or even more.
- jedrek 2y agoAI? Yes. LLMs pretending to be AI? No.
- trashtester 2y agoWhat you call "AI" is generally named AGI. LLM's are alredy a kind of AI, not just generic enough to fully replace all humans. We don't know if full AGI can be built using just current technology (like transformers) given enough scale, or if 1 or more fundamental breakthroughs are needed beyond just the scale. My hypothesis has always been that AGI will arrive roughly when the compute power and model size matches the human brain. That means models of about 100 trillion params, which is not that far away now.
- fsloth 2y agoIf we look at history of innovation and invention it’s very typical the original discovery and final productization are done by different people. For many reasons, but a lot of them are universal I would say. E.g. Oppenheimer’s team created the bomb, then following experts finetuned the subsequent weapon systems and payload designs. Etc.
- fprog 2y agoExcept OpenAI hasn’t yet finished discovery on its true goal: AGI. I wonder if they risk plateauing at a local maximum.
- Zambyte 2y agoI'm genuinely curious: what do you expect an "AGI" system to be able to do that we can't do with today's technology?
- jagrsw 2y agoSome first ideas coming to mind: Engineering Level: Solve CO2 Levels End sickness/death Enhance cognition by integrating with willing minds. Safe and efficient interplanetary travel. Harness vastly higher levels of energy (solar, nuclear) for global benefit. Science: Uncover deeper insights into the laws of nature. Explore fundamental mysteries like the simulation hypothesis, Riemann hypothesis, multiverse theory, and the existence of white holes. Effective SETI Misc: End of violent conflicts Fair yet liberal resource allocation (if still needed), "from scarcity to abundance"
- hojleorier23423 2y agoWhat an absurd thing to say. John Schulman is still at OpenAI. As are many others.
- kinnth 2y agoAI has now evolved beyond just the science and it's biggest issue is in the productization. Finding use cases for what's already available ALONG with new models will be where success lies. ChatGPT is the number 1 brand in AI and as such needs to learn what it's selling, not how its technology works. It always sucks when mission and vision don't align with the nerds ideas, but I think it's probably the best move for both parties.
- itsoktocry 2y ago>ChatGPT is the number 1 brand in AI and as such needs to learn what it's selling, not how its technology works. I'm not as in tune as some people here so: don't they need both? With the rate at which things are moving, how can it be otherwise?
- vasco 2y agoI guess their point is you already have a lot out there to create new products, and you can still read papers you just won't be writing them.
- throwthrowuknow 2y agoThey do need both but it seems like they have enough engineering talent to keep improving. Time will tell now that Ilya is out but I expect they have enough cultural cache to attract excellent engineers even if they aren’t as famous as Ilya and Karpathy. They have a strong focus on making the existing models fast and cheap without sacrificing capability which is music to the ears of those looking to build with them.
- Zacobia 2y agoI love the way you used "cultural cache" here Lol. In any case I do hope whatever Ilya is building is some sort of AI competition to stop this OpenAI monopoly.
- godelski 2y ago> With the rate at which things are moving Things have been moving fast because we had a bunch of top notch scientists in companies paired with top notch salesmen/hype machines. But you need both in combination. Hypemen make promises that can't be kept, but get absurd amounts of funding for doing so. Scientists fill in as many of the gaps as possible, but also get crazy resources due to the aforementioned funding. Obviously this train can't go forever, but I think you might understand that one of these groups is a bit more important than the other while one of these groups is more of a catalyst (makes things happen faster) for the other.
- renegade-otter 2y ago"Productization". You mean "enshitification".
- boringg 2y agoDepends on who is controlling product.
- ookdatnog 2y ago> They make lots of money Will they though? Last I heard OpenAI isn't profitable, and I don't know if it's safe to assume they every will be. People keep saying that LLMs are an existential threat to search, but I'm not so sure. I did a quick search (didn't verify in any way if this is a feasible number) to find that Google on average makes about 30 cents in revenue per query. They make a good profit on that because processing the query costs them almost nothing. But if processing a query takes multiple seconds on a high-end GPU, is that still a profitable model? How can they increase revenue per query? A subscription model can do that, but I'd argue that a paywalled service immediately means they're not a threat to traditional ad-supported search engines.
- FuriouslyAdrift 2y agoJakub Pachocki is taking over as chief scientist. https://analyticsindiamag.com/meet-jakub-pachocki-openais-new-chief-scientist/ https://analyticsindiamag.com/meet-jakub-pachocki-openais-ne...
- davedx 2y agoI love Karpathy. He's like a classical polymath, a scholar and a teacher.
- albertzeyer 2y agoGreg Brockman is a very good engineer. And that's maybe even more important in the current situation.
- larodi 2y agoKarpathy is still a mountain in the area of ML/AI, one of the few people worth following closely on Twitter/X.