9 ms·
Yann LeCun on his start in AI, recent self-supervised learning research [video]
- IdiocyInAction 5y agoIs there a summary or transcript available?
- andreyk 5y agoThere is a rough (AI generated) transcript here: https://app.trint.com/public/7ad490c6-bf70-41ea-bb43-24c326464cd1 https://app.trint.com/public/7ad490c6-bf70-41ea-bb43-24c3264... We hope to produce polished transcripts in the future, but have yet to figure out the best way. A quick summary is: * First ~15 minutes is intro + discussion of Yann's early days of research in the 80s * Minutes 15-~45 cover several notable recent works in self-supervised learning for computer vision (including SimCLR, SWAV, SEER, simsiam, barlow twins). * Final ~15 minutes are discussion of empirical vs theoretical research in AI, how AI is used at Facebook, and whether there will be another AI winter.
- personjerry 5y agoThe best way is to have someone listen to the audio and type it out.
- andreyk 5y agoTrue, I should say the best way that does not involve us transcribing it by ourselves. In any case, we'll work on this!
- jazzyjackson 5y agoI wonder if you could post the transcript to a git repo and allow corrections via pull request. Auto-captioning is a great first step to get phrases set to time-codes, and then open it up to the community for corrections and translations.
- shoo 5y agoaccepting PRs would have the downside of generating additional work for maintainers of the repo to review PRs.
- jazzyjackson 5y agoStill, it beats paying by the minute to actually hire someone.
- somerandomness 5y agoASR works pretty well these days
- andreyk 5y agoNot as well as I would have expected tbh - see the trint link above, it's pretty good but there are lots of errors, so correcting it is quite time consuming.
- gnicholas 5y agoI've heard great things about Descript. It's not free (aside from a limited trial), but apparently it makes it really easy to get good transcripts, and also allows you to clean up the audio as well.
- scribu 5y agoThanks for that. It's hard to make out acronyms like "SWAV" or "SEER" from the audio, if you're not already familiar with them.
- andreyk 5y agoGood point! We should link to the papers discusses in the description at least, did not think of that.
- andreyk 5y agoHey, I am an editor at The Gradient and host of this interview! As this is only episode 6 the podcast domain is pretty new to us^, so would definitely welcome feedback on question choice or my style as an interviewer. We tried focusing much more on research than other interviews out there such as Lex Fridman's, would be curious to hear if you think that worked well. ^(we've existed as a digital publication focused on AI for way longer, see thegradient.pub if you're curious.)
- antimora 5y agoI couldn't see this episode in pocketcasts. Is it a technical delay or it usually becomes available on other platforms later.
- andreyk 5y agoYeah, it usually takes a little while to appear on other platforms, annoyingly.
- joe_the_user 5y agoYou don't seem to include a transcript here. Seems like a serious flaw (I personally prefer transcripts to audio but it's actually an accessibility issue for some people).
- andreyk 5y agoThanks! We'll work on that
- 908B64B197 5y agoA better format than random Twitter thread where a mob tries to cancel him [0]. You might recognize one name that got really famous not long ago! [0] https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter-amid-acrimonious-exchanges-on-ai-bias/ https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter...
- IfOnlyYouKnew 5y ago> You might recognize one name Yes, it’s terrible when people are subject to all sorts of personal attacks based on snark and innuendo.
- elefanten 5y agoThis is indecipherable to me. Who are you taking a shot at? Is this comment pro-snark or anti-snark? Sarcastic or straight? Who knows.
- horrified 5y agoHorrible story :-( And once again the grievance strategy was successful.
- mrtranscendence 5y agoI don't know how folks can be aware of how the exchange went down and say that it was a "successful" "grievance strategy". LeCunn wasn't necessarily in the right here, and it wasn't only Gebru's twitter followers going on the offensive.
- horrified 5y agoWell LeCunn quit Twitter, so it is "one down". That is what I meant by successful. And Gebru's "arguments" weren't even arguments, just "whatever you say is wrong because you are white and don't recognise our special grievances". I personally agree with what he said when he said it is a difference between a research project and a commercial product. No actual harm was done when the AI completed Obama's image into a white person. You could just laugh about it and move on.
- danmaz74 5y agoOne very interesting thing that was mentioned in the interview is how much Facebook relies on deep learning right now; specifically, how hate speech detection went from 75% manual to something like 2.5% manual, and how manual false negatives detection allowed this improvement. What I'm wondering is about false positive detection, which wasn't mentioned, and how much of this incredible decrease in false negatives came at the expense of an increase in false positives.
- joe_the_user 5y agoAnecdotally, FB's hate speech detector is pathetic. I have lots of friends run afoul of it for trivial things. It seems no more coherent than a bunch of regular expressions. I had a post in a group get flagged by the algorithm for something "your politics shouldn't involve saying '[bad word] about [protected group]'". I suspect the problem is there's nothing that would flag as wrong for a system that just defaults to such crudeness. So that's what happens.
- civilized 5y agoI was once asked to use machine learning to make a record linkage system for some crappy dataset. I got no requirements, of course, so I set it up to have a reasonable balance of precision and recall. After all, the point of asking for an ML system must be to allow fuzzy matches that a simple exact matching system would miss, right? But my boss apparently got complaints about bad matches, so he changed it to allow exact matches only. The machine learning system ended up being a Rube Goldberg machine for linking people based on exact name match.
- lumost 5y agoAnecdotally this is pretty typical of the evolution of mo systems. Some heuristic/standard algorithm works pretty well, but people see cases it didn't catch and think it could be better. An engineer/scientist takes a look and realizes the extra features required need a more complex/ml algorithm to add support for. Years later a model has made it to production, but there are now complaints about too lenient matching. In my experience, ML works best in scenarios where there is either such immense data volume that .2% improvement is a real benefit (these are rare) or the very notion of a heuristic method simply wouldn’t work.
- ok2938 5y agoI am no one. I have greatest respect for all Turing award winners. But one thing I am wary is that LeCun - while special and excellent - is just as many others, working at a place where "AI" is already used to "engage people up" - it is just the nature of the business if you are in the engagement business. And your "AI" will gladly help you in all kind of subtle ways. What is also nice is that it's uncharted territory now, so you can freely roam - and engage the heck out of your audience. And LeCun is just - as a "neutral" scientist - just doing his part. Why can't he not work at FB? Money? Data?
- rchaud 5y agoBecause Facebook makes the most money, and probably offered him the most. Same reason why back in the day, a lot of people got electrical engineering degrees but went into software development or finance. The skills were transferable, and the pay was a lot higher.
- throwawaygh 5y ago> Why can't he not work at FB? Money? IDK, I get it. Grad school sucks. Post-docs suck. Pre-tenure sucks. Post-tenure isn't any better. For that entire period of time you are working on de facto fixed term contracts. Which is extremely uncommon among salaried engineers, and those that take these sorts of contingent employment contracts are typically paid quite well. It's like 10-15 years of low pay, "will I have a job next year?" stress, and moving your family around all the time (or, more commonly, just not starting a family). And not even for good pay. These days, even after a decade or more of experience, you're making less than your undergrads. Half as much or even less in some cases. So, your undergrad once-peers start retiring -- or at least thinking about it -- around the time that you're finally transitioning from de facto fixed-term positions to something resembling a normal employment contract, but, again, for a third to a fifth of what you'd be making in industry at that point in your career. So, yeah, people say fuck it and cash in on influence/engagement/reputation where they can. The only real alternative is the public sector paying researchers better, but that's never going to happen.
- deleted 5y ago[deleted]
- mooseburger 5y agoLeCun is interesting. The way he reasons about AI X-risk makes him seem like a retard, let's not mince words. He's an actual example of a mind so specialized that it has lost the capacity for lateral thinking.
- TheDudeMan 5y agoDid he say he doesn't believe in the concept of AGI? What does that mean? He doesn't believe machine intelligence will ever surpass human intelligence? That's surprising.
- arminiusreturns 5y agoHaving been in the industry, I think a lot of the researchers at that level get tired of every layman jumping to discussion of AGI, so Ive heard some of them refuse to comment on it. (or have a prepped one liner) Who can blame them? AGI at some point will become more about the semantics of the term. Does it mean an intelligence that can self improve? etc. Generally I think game worlds are likely to have the right conditions for the first emergence of primitive agi-precursors.
- random314 5y agoThis is not surprising, because virtually every AI expert (except Kurzweil) don't think AGI is even a remote possibility in this century. The only "AI experts" who believe that AGI is something to worry about are self appointed AI experts - Elon Musk, Sam Harris, Sam Altman etc. Zuckerberg had actually set up a dinner with Musk and LeCun to explain to him that AGI is not a realistic possibility. This led to Musk stating that Zuckerberg had a very poor understanding of AI. Looks like Musk is the self appointed AI expert in that table of 3! Oh BTW, Musk, if you are reading this - how is the FSD project coming along? Note: Scientists working for Open AI will pretend that AGI is a real possibility because the folks funding Open AI have a predilection for AGI.
- salt-licker 5y agoDo you consider gwern not to be an AI expert? I think this is a reasonably technical explanation of how we may not be as far away from AGI as some choose to believe: https://www.gwern.net/Scaling-hypothesis https://www.gwern.net/Scaling-hypothesis
- random314 5y agoI am not sure who this person is. How much has he published in ML and what is his h-index. As examples, Le Cun has a h-index of 127 Hinton 164 Goodfellow 75 etc. None of them consider AGI to be a realistic possibility this century. Kurzweil, the AGI proponent, has an hindex of 22.