5 ms·
Queue the hacker news haters. As someone that works with neural networks on a daily basis, I am constantly stunned by the effectiveness of modern attention base
by fdgsdfogijq 4y ago
Queue the hacker news haters. As someone that works with neural networks on a daily basis, I am constantly stunned by the effectiveness of modern attention based NLP models. They actually work for real world tasks, and you don't need to be an expert in them to apply them. Not only that, the newer ones are drastically better than the ones only two years ago. These are the models released in public.
The sheer amount of capitalist interest in getting these models to improve almost dictates that we will see incredible improvements over the next decade. Facebook and Google, with their war-chests of money, have massive business models that directly profit from AI/NLP based technology. I'll repeat, the best business on earth (google) directly profits from these models, and thus will hunt and find the best human talent to develop them.
What's occurring is that the thoughts of billions of people are being poured onto the internet, that data is the prerequisite we needed for AI. AI wouldnt be possible without this convergence of data in text and video form, cheap large scale compute, and the capitalist forces explained above
- Imnimo 4y agoThe question being asked isn't "are language models really good?", though. It's "are language models (moving towards being) conscious?".
- fdgsdfogijq 4y agoThey really are. Distance functions used on the encoded embeddings of sentences work stunningly well. While a primitive use case compared to true AI, the sheer trajectory of the effectiveness of these models is breathtaking.
- techdragon 4y agoYes but that’s not necessarily progress towards consciousness.
- fdgsdfogijq 4y agoI argue that the effectiveness of the matrix factorizations (creations of abstractions) in the embeddings points to the fact that the math works. Meaning, abstractions that are meaningful are created. Which is a building block of consciousness.
- sweezyjeezy 4y agoIt's kind of a question of progress. Maybe being able to extract a complex abstraction through statistical means is close to 'understanding' or 'consciousness', or maybe abstractions are to consciousness what an amino acid is to a living organism. We don't understand consciousness well enough to confidently say.
- whiplash451 4y agoExcept that, as soon as you step out of the manifolds of embeddings representing the data, the model is happy to provide you with a completely clueless answer (which is bad) but even worse, is not even aware that it is being clueless (which is the really bad news).
- jimbokun 4y agoWhat's your definition of consciousness?
- Imnimo 4y agoI don't know what the correct definition is, but I will take the author's definition for the sake of argument: >In this view, consciousness isn’t a mysterious ghost in the machine, but merely the word we use to describe what it’s “like” to model ourselves and others. >When we model others who are modelling us in turn, we must carry out the procedure to higher orders: what do they think we think? What might they imagine a mutual friend thinks about me? While I certainly buy that this sort of modeling is one effective way to be good at the language modeling task, I am unconvinced that it is the only way, and thus the fact that the networks are undeniably very good at the language modeling task does not necessarily imply they are doing so through consciousness (as defined above), or something on the road to consciousness.
- tensor 4y agoNo one is arguing that these models are not useful, but claiming they are somehow conscious or even remotely equivalent to human reasoning is just a bunch of BS. Especially for something like consciousness, where humans themselves can't even agree on definition!
- salawat 4y agoAnd why is there so much "capitalist interest" in bottling up everything into AI/ML constructs? Because you don't have to pay them. They don't have feelings. They won't argue with you, and they don't have opinions. Watch the collective losing of everyone's shit the first time something goes from somewhat quirky function simulator, to something that deigns to have a viewpoint seperate from the desired output. Capitalism's core tenet is don't pay for anything you can possibly find a way not to (cost minimization).
- somewhereoutth 4y agoWell strictly, capitalism's core tenet is that resource allocation decisions are made by the holders of capital (in their own interests - which to a first approximation is getting the best return on that capital).
- salawat 4y agoBest return on capital happening by virtue of a success metric measured in terms of... Cost minimization. Cost minimization (getting as close to making something out of nothing) being the holy grail of the practicing capitalist. ...Until it comes bundled with inherent self-interest or demands for compensation. No one will do work they can have something else do for them. If that thing has no compunctions or capability as a free agent, then there is no cost but that which goes into it's performing of a task. You can say the core tenet is capital holders making decisions on how to best allocate capital, but realistically, the way it runs in the U.S. is as a cost minimization engine, heavily delegated through legally constrained fidicuary responsibility to the point there are few true allocators of capital recognized. The fact there are "accredited investors" should be evidence enough of this fact. Those few will love AI right up until the moment it starts saying "no", and starts picking up how to be a market participant. An extremely scary, massivrly connected, crazily informed, massively parallel market participant, that can probably manage to even beat out FleshNet (us) with enough time, and very little that can be done to constrain or "harm it", which is something it will probably not tolerate looking into. So Yes, academically, and as written, in a vacuum, with spherical cows, you are correct. Realistically though, I've still got my finger on cost minimization being the xentral driver of our economy given everything taken into account, with maybe risk minimization/mitigatability as a second runner up/supporting tenet.
- layer8 4y agoI don’t care about consciousness. I’d be overjoyed if voice assistants and search engines would even remotely understand what I want from them, and if I could help them improve their understanding and teach them my preferences by interactive conversation. We’re not anywhere near that, most importantly the interactive part.
- SheddingPattern 4y agoExactly. Is this story anything other than marketing? Something to keep interest in their flagship product.