6 ms·
I actually think OpenAI is on an accelerated path because it knows it's days are numbered. If the tech were truly superior, we would see Apple, Google and Meta
by glitchc 3y ago
I actually think OpenAI is on an accelerated path because it knows it's days are numbered.
If the tech were truly superior, we would see Apple, Google and Meta rushing to license their tech. Yet they're not, instead, they're all building their own version. There's no secret sauce left to building an LLM. It's all public knowledge. And while ChatGPT has an edge right now, it's not a substantial one.
- yieldcrv 3y agosynthetic datasets ftw! they might have had a library when nobody was looking, but everything created after GPT-3 is analyzed by everyone and everything before GPT is extracted by getting GPT and other LLM's to talk oh noes not the terms of service!
- bombcar 3y agoApple is apparently throwing largish sums of money to license all its training data, so it may be that OpenAI ends up exploding because of the “move fats and break things” so common in SV.
- ponderings 3y agoWith enough training anyone can be a belly dancer.
- boringuser2 3y agoNah, none of the other LLMs are particularly useful at much of anything, but GPT-4 is profoundly useful.
- potsandpans 3y agoI've seen a lot of things come from gtp4, none of them "profound." Could you share something that you se and think "profound"?
- boringuser2 3y agoI said profoundly "useful". GPT-4 helps me write reams of code every day at my job.
- DirkH 3y agoThe fact that on my day job and also as per most engineers I know ChatGPT is now the first line of defense for finding quick code snippets where for over a decade Google held that distinction is profound.
- Anon1096 3y agoYou seem to be very very conveniently ignoring the fact that Microsoft spent 10 billion to basically gain exclusive use of OpenAI's tech... exactly what you're saying Apple, Google, and Meta should be doing.
- glitchc 3y agoMicrosoft funded the R&D, their return on investment was not guaranteed. It's apples to oranges.
- spott 3y agoMicrosoft invested $10B after GPT4 came out… their initial investment of $1B might have been for R&D, but their $10B investment was for tech that they knew about.
- FourHumboldt 3y agoWell, the MSFT investment is mostly in the form of cloud services spend/usage credits (vs. actual cash) which of course is one of the most important aspects (and massive expensive item) for OpenAI in developing and productizing their offering, given the nature of generative AI. MSFT has absolutely no secret sauce with Azure. AWS and Google (GCP) have those capabilities as well and can (and do) apply the same investment model. See also the recent AWS/Anthropic engagement.
- xnx 3y agoIsn't Microsoft's $10B investment likely more of a "pay as you go" rented integration of GPT4 into Bing and a few other places rather than a $10B wire transfer into OpenAI's bank account?
- londons_explore 3y agoChatGPT has been ahead the whole time, and according to the lmsys leaderboard it's not a small margin. Nobody else has yet beaten what openAI released in march last year. Google tried to beat it with Gemini Ultra, but as well as being unreleased, the stats in the paper don't lend much confidence it will beat gpt-4-0314
- scarface_74 3y agoIt doesn’t matter how good the technology is. It matters how well they can productize it.
- dr_kiszonka 3y agoGemini ultra is nowhere close to GPT-4 for anything I tried.
- berniedurfee 3y agoWhat if you took away 3/4 of the training data though? If NYT et al wins their case, training data won’t be free anymore. Content owners will decide on the price and even who to license to. Content owners will be asking exorbitant amounts for licensing fees and will likely strike exclusive deals with LLM owners. Maybe Microsoft actually bought GitHub for the content?
- berniedurfee 3y agoI also think that profit for LLM-based businesses without massive data troves is now solidly capped. Content owners are moving quickly to monetize their hordes of data. The days of free training data are over. If you don’t own it already, you probably can’t afford it. I think the interesting side effect is that LLMs will end up as bifurcated as the internet. Each only being trained on some subset of content based on which subset the LLM builder chooses or is able to license for training. LLM agents will all be hamstrung and biased in various ways based on fragmented training sets. There will be no singularity. There will be many LLM agents that learned to think based on the information their creators could afford or chose to provide it. These agents will have biased, inaccurate and incomplete world views, yet will be very confident they know everything. How very human!
- pegasus 3y agoIf inference costs keep going down, I expect pirate LLMs trained on pirated and much more complete text libraries will proliferate.
- berniedurfee 3y agoGovernments have the most to gain. The NSA’s LLMs will be light years ahead of anything commercially produced if more data makes better models.