4 ms·
- There _was_ a problem with diminishing returns from increasing data size. Then they surpassed that by curating data. - Then the limits on the amount of curat
by ilaksh 2y ago
- There _was_ a problem with diminishing returns from increasing data size. Then they surpassed that by curating data.
- Then the limits on the amount of curatable data available made the performance gains level off. So they started generating data and that pushed the nose up again.
- Eventually, even with generated data, gains flattened out. So they started increasing inference time. They have now proven that this improves the performance quite a bit.
It's always been a series of S-curves and we have always (sooner or later) innovated to the next level.
Marcus has always been a mouth just trying to take down neural networks.
Someday we will move on from LLMs, large multimodal models, transformers, maybe even neural networks, in order to add new levels and types of intelligence.
But Marcus's mouth will never stop yapping about how it won't work.
I think we are now at the point where we can literally build a digital twin video avatar to handily win a debate with Marcus, and he will continue to deny that any of it really works.
- sheeshkebab 2y agoI’m still waiting for a neural net that can do my laundry. Until there is one I’m on Marcus’ side.
- Terretta 2y agoThat's less a GenAI problem, more robotics. And perhaps it's here, literally doing your laundry: https://x.com/physical_int/status/1852041726279794788 https://x.com/physical_int/status/1852041726279794788 https://www.physicalintelligence.company/blog/pi0 https://www.physicalintelligence.company/blog/pi0 https://www.entrepreneur.com/business-news/jeff-bezos-backed-physical-intelligence-robot-can-do-chores/482348 https://www.entrepreneur.com/business-news/jeff-bezos-backed...
- sheeshkebab 2y agoNo teleoperation, can haul the basket from 3 floor down and back up fully folded and put away in my closet without me doing a thing.
- ilaksh 2y agoThat video demo is not tele-operated. You are arguing a straw man. The discussion was about LLMs.
- sheeshkebab 2y agoIt it can’t do laundry I personally don’t care if it can add 2+2 correctly 99% the time
- edanm 2y agoI'm still waiting for a computer that can make my morning coffee. Until it's there I don't really believe in this whole "computer" or "internet" thing, it's all a giant scam that has no real-world benefit.
- bathtub365 2y agoWhat is lacking compared to current bean to cup coffee makers?
- GuB-42 2y agoAutomatic coffee machines are literally a computer making your morning coffee :) But my washing machine doesn't have a neural network... yet. I am sure that there is some startup somewhere planning to do it.
- EternalFury 2y agoNo one in their right mind will argue neural nets cannot outperform humans at resampling data they have previously been exposed to. So, digital twins and debate, they probably can do better than any human.
- ilaksh 2y agoMarcus would argue against digital Marcus on this point and lose.
- trott 2y ago> Marcus has always been a mouth just trying to take down neural networks. This isn't true. Marcus is against "pure NN" AI, especially in situations where reliability is desired, as would be the case with AGI/ASI. He advocates [1] neurosymbolic AI, i.e. hybridizing NNs with symbolic approaches, as a path to AGI. So he's in favor of NNs, but not "pure NNs". [1] https://arxiv.org/abs/2308.04445 https://arxiv.org/abs/2308.04445
- ilaksh 2y agoHe does not spend an appreciable amount of effort or time advocating for that though. He spends 95% of his energy trying to take down the merits of NN-based approaches. If he had something to show for it, like neurosymbolic wins over benchmarks for LLMs, that would be different. But he's not a researcher anymore. He's a mouth, and he is so inaccurate that it is actually dangerous, because some government officials listen to him. I actually think that neurosymbolic approaches could be incredible and bring huge gains in performance and interpretability. But I don't see Marcus spending a lot of effort and doing quality research in that area that achieves much. The quality of his arguments seems to be at the level of a used furniture salesman.
- xpe 2y ago> He spends 95% of his energy trying to take down the merits of NN-based approaches. The 95% figure comes from where? (I don't think the commenter above has a basis for it.) How often does Marcus-the-writer take aim at NN-based approaches? Does he get this specific? I often see Gary Marcus highlighting some examples where generative AI technologies are not as impressive as some people claim. I can't recall him doing the opposite. Neither can I recall a time when Marcus specifically explained why certain architectures are {inappropriate or irredeemable} either {in general or in particular}. Have I missed some writing where Marcus lays out a compelling multi-sided evaluation of AI systems or companies? I doubt it. But, please, if he has, let me know. Marcus knows how to cherry-pick failure. I'm not looking for a writer who has staked out one side of the arguments. Selection bias is on full display. It is really painful to read, because it seems like he would have the mental horsepower to not fall into these traps. Does he not have enough self-awareness nor intellectual honesty to write thoughtfully? Or is this purely a self-interested optimization -- he wants to build an audience, and the One-Sided Argument Pattern works well for him.