4 ms·
Yes, they are doing the improving, but then you need loads of money to do the learning no university can afford. So now big tech is hiring promising university
by jona-f 3y ago
Yes, they are doing the improving, but then you need loads of money to do the learning no university can afford. So now big tech is hiring promising university researchers for good money to scale up their research.
This could be solved by massive decentralization where millions of users provide compute with their gpus and i think it will be at some point, cause i believe foss is more powerful than this openai bs.
There are people working on this, but afaik the techniques aren't quite there. You need a different kind of model with much more parallelization then what is currently used.
- wiz21c 3y agoI knew for the lack of money. But the parallelization idea, I didn't. Thanks for posting !
- DeathArrow 3y ago> There are people working on this, but afaik the techniques aren't quite there. You need a different kind of model with much more parallelization then what is currently used. What if crypto is switching from mindless hashing as proof-of-work to training AI models as proof-of-work? That would mean suddenly big computing resources are available.
- jona-f 3y agoThat would indeed be neat, no idea if that is possible safely. Blockchain technology sure looks like a good fit for the organization of such a decentralized model. The reaction of the people of hn to 'blockchain-ai', probably won't be kind though, lol.
- pas 3y agoBlockchains depend on small inputs and outputs, and easy verifiability of the computation. AI training is none of that. It's a massively interconnected problem, Google and NVIDIA built traditionalish fast-interconnect supercomputers to do it. Maybe it's possible to implement it in a decentralized way with not-completely-useless performance, but by then OpenAI and others will be even more ahead. (Sure, it'd be good to have such an implementation, so maybe enthusiasts will do it eventually, but that's mostly for fun.)
- jona-f 3y agoYeah, that's the current state of course, I'm betting on future developments. Neural networks are still in their infancy. Stable diffusion showed everyone how much more powerful open models are (at generating porn /s). Chatgpt is severely limited, cause Openai fear public outrage, litigation and regulation.
- mensetmanusman 3y agoYeah I keep telling people the universe played a joke on us with the crypto winter happening right before GPT-4 being released. Flip the timeline and crypto from the start would have had this as a goal.
- jodrellblank 3y ago“What if running part of the paperclip maximiser that destroys humankind was profitable?” is both a bit of a nightmare sci-fi dystopia, and a reasonable description of companies in the modern economy.
- DennisP 3y agoFor blockchains using proof-of-work you need two things: 1) The work has to be very hard to do (and quantifiably so), but very easy to verify. 2) The block's transactions have to be an input to the computation, and it has to be impossible to get the same output with a modified set of transactions. Cryptographic hash functions fulfill both of these requirements. Almost nothing else does. However, if blockchains switch to proof-of-stake, then the GPUs previously dedicated to that blockchain are available for other purposes. But the biggest GPU-based blockchain already did that, and Bitcoin uses specialized hardware that can't do anything other than Bitcoin's hash function.