5 ms·
As someone who is doing robotics and behavior cloning for a while, it’s incredibly funny to see the self-taught genius rediscover what’s been known in the field
by polygamous_bat 3y ago
As someone who is doing robotics and behavior cloning for a while, it’s incredibly funny to see the self-taught genius rediscover what’s been known in the field for 30 years and can be found by walking in to the first two lectures of any decision making/control theory class.
In practice, deep learning based behavior cloning works when there is enough data for the model to interpolate to the next “frame”, the proof of which is GPT and other LLMs whose primary training objective is practically behavior cloning. However, adding a neural network means there are some hallucinations, which can range for silly (for LLMs) to murderous (for self driving). However, this presents a huge opportunity as well, in that anyone who can solve hallucinations can make headway into not one but two billion dollar problems at once.
- dheuxysgg 3y ago[flagged]
- polygamous_bat 3y agoI am talking about the accumulated error part near the end of the article, so your (uncharitable) assumption is not correct.
- AndrewKemendo 3y agoI have a similar reaction when somebody rediscovers something that specialists all know by default However, the older I get the more I appreciate these folks because they are excited about the problem usually way more than people who know the systems rote In some cases you get a charlatan trying to recycle something, not open to feedback that they aren’t Kepler or whatever, but in my experience most of these folks are trying to figure the world out and are excited when something seems to fit Now, I try (not always successful) to “yes and” them into trying to figure out how their method compares to existing methods, with the idea that - hey maybe they figured out an edge case by accident that we can all learn from That said, this is geohot and he definitely isn’t new to BC/IL
- randomsolutions 3y agoYea, I don't see the problem with someone rediscovering knowledge. The question I would ask is, would geohot have been better to have gone to school to learn this and deferred all the work he has done? Who is farther ahead, the people who already knew this from school, or the guy who is building and learning this as he goes?
- AndrewKemendo 3y ago100% the person building and learning as they go A PhD in ML is worth less to me than day to day operational product engineering experience utilizing the fast changing ML tooling landscape
- polygamous_bat 3y ago> A PhD in ML is worth less to me than day to day operational product engineering experience utilizing the fast changing ML tooling landscape This is how you get a four-week transition from "I can fix twitter search in an internship!" to "twitter search is unfixable." Ideally, a PhD teaches you to bust your behind doing a deep dive on a seemingly unsolvable problem for three-five years. If only we could recalibrate ourselves to stop the search for easy money and remind ourselves that some problems are just genuinely hard.
- AndrewKemendo 3y agoI mean I’m talking generally not about him specifically The best senior engineers on the planet have a BS and 30 years experience
- deleted 3y ago[deleted]
- colesantiago 3y agoUnfortunately 'senior engineers' who have 30+ years of experience in the valley and other places are past their prime in SV by most top tech companies as engineers. They are expected to be in management roles.
- richrichie 3y agoThis is a problem in tech in general, where people don’t or haven’t studied math beyond discrete and counting stuff. Some people think that SGD was invented 10 years ago by ML “scientists”.
- j45 3y agoHe does seem open to asking. And not acting above anyone. Everyone is a beginner at something, even OP. Exploring curiousity, whether it's new to someone, or everyone is a good quality. I wonder how many of the students who have attended these classes have applied all that math that's existed for so long into a self driving solution