6 ms·
No.
by perone 3y ago
No.
- dontwearitout 3y agoMore correctly: we don't know how to do this efficiently. Biological neural networks don't use backpropagation and work great.
- sdenton4 3y agoConference takes years, though... It's entirely possible that back prop is unrealistic but far more efficient than biological learning.
- ShamelessC 3y agoOut of my depth so happy to be corrected. Don’t many/most state of the art models take many months to train on far more data than humans need for similar tasks? Also, while e.g. GPT4 is quite capable across many tasks - humans seem to average towards learning robust _learning techniques_ themselves. Learning a new subject becomes easier thanks to somehow tracking and encoding learning strategies that are robust to learning other unrelated topics.
- landryraccoon 3y ago> Don’t many/most state of the art models take many months to train on far more data than humans need for similar tasks? Humans generally need 18 years of pre training followed by 4-6 years of fine tuning before they can “one-shot” many difficult tasks. That’s way more training than any machine learning model I’m aware of. Even for tasks like reading the newspaper and summarizing what you read, you probably had to train for 10-12 years.
- tacheiordache 3y agoI see this stance of yours parroted over and over but a 3 year old can tell a dog from a cat doesn't need to be trained on millions of images. Also uses way less energy for that.
- landryraccoon 3y agoA 3 year old still takes 3 years to train. Even a state of the art image recognition model takes way less time than that.
- beepbooptheory 3y agoNot sure if you meant this as a joke, but it made me smile. That poor child!
- ChatGTP 3y agoThe moronic thing about this is, humans aren't just training, they're having a life. The "training part" is one part of it sure, but it's not the reason for existence.
- sebzim4500 3y agoWhat's the functional difference between training and having a life?
- naasking 3y ago> but a 3 year old can tell a dog from a cat doesn't need to be trained on millions of images A 3 year old has 3 years of multimodal training data and RLHF + a few billions of years of evolution that have primed and biased our visual and cognitive systems. That requires a lot more data than machine models that literally zero inherent bias. Assuming you want a true apples to apples comparison.
- AbrahamParangi 3y agoThe structure of that 3yr old’s visual cortex is itself the result of a ~500 million year optimization process.
- marcinzm 3y agoIt takes basically a week on a single GPU to train AlexNet which has human level ImageNet performance. Let's say it's 500 W for the GPU versus around 10 W for a human brain. So that's 84kwh for the model and 175kwh for the baby (over 3 years at 16h/day). That's without a half billion years of architecture and initialization tuning that the baby has. I think the model performs very favorably.
- deleted 3y ago[deleted]