4 ms·
I expected this to be a smear / petty argument article. In fact, it's a concise, highly specific, quote by quote critique. I don't have enough context to take
by obblekk 4y ago
I expected this to be a smear / petty argument article. In fact, it's a concise, highly specific, quote by quote critique.
I don't have enough context to take a side, but this is not just a rant.
Beyond their interpersonal disagreements, I do wonder if LeCunn is seeing diminishing marginal returns to deep learning at FB...
- adamsmith143 4y agoDiminishing returns? Have you read the Gato, Palm, Stable Diffusion, etc. papers? Progress is racing ahead. Nothing is stalling... the only thing stopping progress from accelerating even faster is data.
- random314 4y agoHe is talking about Deep learning at FB
- mirker 4y agoMany of these scaling patterns are logarithmic with respect to data size. You can only double the dataset size so many times that it’s really not clear the scaling will continue.
- lostmsu 4y agoLow data modes are also progressing quite fast. There is Dreamer and more recent papers based on RL in learned world models.
- polotics 4y agoThe points are indeed very specific, but they are about opinions, mostly not-even-wrong statements, just reasonable unquantifiables. The elephant in the room is the use of the word "deep" in the field IMHO: it means something else than "many layered neural network" in common parlance...
- lostmsu 4y agoWhat does it mean? Techniques that avoid the vanishing gradient problem?