5 ms·
IMO it highlights just what a commodity they weights are. If all one needs is the weights to reproduce the work, then where is the value? I mean there is very l
by marktangotango 4y ago
IMO it highlights just what a commodity they weights are. If all one needs is the weights to reproduce the work, then where is the value? I mean there is very little moat here. Further, what does it say about consciousness and individuality if we all are simply the values of the weights in our wet neural networks? Or whatever the biological equivalent is?
- akiselev 4y agoNothing. It says nothing about consciousness and individuality. Numerical weights and physical neurons aren’t even remotely comparable.
- quonn 4y agoEven if they were, any data is also only bytes - numbers - and therefore any knowledge, books, any kind of sound or picture would likewise be trivial. But isn‘t.
- TeMPOraL 4y agoThere's nothing "simply" here. The weights in question are a particular configuration of several gigabytes worth of data. They're not random. Getting anything comparable by randomly generating a number this long is a "total atoms in the universe to the power of total atoms in the universe" kind of a deal. In abstract terms, those weights are by far the most dense form of meaning we've ever dealt with.
- microtonal 4y agoI think the point is more that the primary edge that they (Meta, OpenAI, etc.) have is compute capacity, since the models are easy to implement (and the most of the training data is out there). Once you have the weights, you can do with it as you please. This is probably why they hate/fight these leaks. Once the model weights are available, they lose much of their competitive advantage.
- visarga 4y agoIf you look at their paper they especially encouraged reproduction. That's why they only used publicly available data.
- gremlinsinc 4y agocould ai, just basically brute force better weights? say instead of doing training to get their own weights, maybe have 10 training inputs, test a new set of weights, have gpt4 rate the quality of the output from each set of weights and then try another and maybe infer a pattern to how weights could maybe potentially be created without all the work? I feel like soon, ai will be it's own researcher and go far beyond what we ever could do on our own.
- lovingCranberry 4y agoYou just reinvented reinforcement learning
- FormerBandmate 4y agoLLMs are very good imitations of what we expect from intelligent people. They are not intelligent themselves. Not only do they have no street smarts whatsoever, but their book smarts start to disappear when you deviate from the training data.
- mrguyorama 4y agoSpeak for yourself. What I've seen of all these language models so far is equal to a 10 year old with a good vocabulary and a tenuous grip on reality and no understanding of truth vs lie or right vs wrong.
- roflyear 4y agoI agree. GPT4 is impressive when asked to play chess, but the incapacity for understanding makes it really limiting
- thfuran 4y agoUnless you believe that you think with your soul or something, what else could you be other than your quantum state, or some close-enough compressed equivalent?
- nico 4y agoThat’s like saying that creating a song or a movie has no value because after they are created anyone can download a file with it. However it does open up a point which is: should we allow people to make huge amounts of money by infinitely copying and distributing their own work? Should we be protecting this model?
- Madmallard 4y agoUh the structure of the neural network matters just as much as the weights. Everyone has a different structure to their neurological arrangement.