4 ms·
It's too late for monopolizing large language models. The technology for making them is known and the cost keeps coming down. There are things to worry about,
by Animats 3y ago
It's too late for monopolizing large language models. The technology for making them is known and the cost keeps coming down.
There are things to worry about, but this doesn't seem to be one of them.
- jszymborski 3y agoTwo thoughts on that: 1. GPU prices aren't as high as they were during the mining peak, but you can still buy a car with the money it takes to buy a pod of A100s. Further to that point, Google et al. are making the next generation of ML accelerators and they won't even sell the best ones. 2. Compute is only half the problem. Access to data (e.g. in the form of email and smart assistant queries and ring door bells) is a much under appreciated second half of that equation.
- neodypsis 3y agoThe biggest cost, other than capital costs, is energy. Some geographies have competitive advantages w.r.t. energy.
- echelon 3y ago> It's too late for monopolizing large language models. The technology for making them is known and the cost keeps coming down. Not so sure non-institutional folks can catch up. OpenAI is trying to regulatory capture LLMs to prevent the Amazons and the well-funded unicorns from following. Everyone left in the lurch after these players crystalize is SOL. You, average HN reader, cannot possibly compete with OpenAI. They're trying to stem venture dollars from standing up more well-funded competition so they take as much of the pie as possible. The hype/funding cycle for LLMs is mostly over now. You're not Anthropic and you're more than likely too late to get in on that game.
- fnordpiglet 3y agoAmazon excels at regulatory compliance. This isn’t a moat against Amazon. It’s a moat against you.
- bugglebeetle 3y agoThe same is true of Microsoft, who more or less own OpenAI now.
- nextaccountic 3y agoTraining is still very expensive.. and if it becomes cheaper, then a next-gen model with better performance will be expensive again. It may be very hard to train cutting edge models if you only have consumer hardware
- chrismarlow9 3y agoWhy do you need cutting edge models if you can solve your problems with less?
- Palmik 3y agoThe gap between the value added by GPT 3.5 and GPT 4 to my every day tasks is immense. Most people, outside of these circles, don't realize that such a large gap exists.
- tome 3y agoCould I ask what exactly you get from 4 that you don't get from 3.5?
- upbeat_general 3y agoPersonally it’s not one specific function but the answer I’m looking for [e.g, code that just works] is significantly better from GPT-4 imo. The better the output is, the less time I spend and the more likely I am to default to using the model in the first place.
- tome 3y agoI see. So even though 3.5 would attempt to answer your question, the results from 4 are significantly better?
- samus 3y agoThey are required if there is an arms-race between actors wielding models. For example, to detect increasingly authentic deepfakes.
- rhizome 3y agoCost of technology doesn't matter for this. It's not the hammers, it's the building permits.
- desmond-grealy 3y agoIt is though. The groundwork is already being laid to restrict the legality of training large models. See the recent executive order. There is a rather small cutoff size being established for needing to report your training runs and data centers to the government. It is also being ordered that X-risk and CBRN risk testing frameworks and secure environments be developed specifically so they can be imposed as requirements of model developers, including in the private sector. You may argue that the knowledge required to develop LLMs is open source and will be accessible to all (and in practice this is not-quite but mostly true). However the federal government is currently in the process of making it so only certain people are authorized to do so legally.
- samus 3y agoThe same can be said about guns. Because of this, gun making is a licensed industry, even though in theory a lot of people can legally acquire the tools to make guns in their garage. Edit: drugs and licensed services like law and healthcare also apply