3 ms·

They spend that much time reiterating well-trodden established information because that's the easiest to redundantly speak to. All of the novel stuff is suspici

They spend that much time reiterating well-trodden established information because that's the easiest to redundantly speak to. All of the novel stuff is suspiciously vague and detail is left to the imagination. Because the author is fudging things.

summarity 3y ago
Indeed, this is also one of the worst abstracts to a paper ever. 90% of it is a generic introduction, and the rest goes out of its way to establish any quantitative results.
godelski 3y ago
This is actually something I really hate about ML research and why I get upset when people say "don't need math." I see a lot of unnecessary equations in papers (dear god, we don't have to write attention in every ViT paper or the density function in every diffusion paper) but then no math around where it matters. It feels like a strategy to get people to just turn on autopilot, since that's what you'll naturally do. It's also weird when you get reviewers asking for this. I've had a reviewer reject a work, saying that I should include the equation for attention... in 2023... There's a reason I don't value conferences and why I tell people they aren't good signals for quality. Hell, we're doing more peer review here than most papers get. I'm not unconvinced the work isn't novel (but novelty is a function of reader's background knowledge) but I think their claims are too strong. Were I the reviewer I would reject for sure based on the unsubstantiated claim of lossless alone, without explanation. Even if a work was great and useful (though conferences make this difficult to make such types of fixes). The method does seem interesting and there is evidence for its utility, but I do think it needs a lot of additional clarity and support of the claims. In fact, I'll go ahead and say what's happening here is how I'd love to see modern peer review work, because we should be updating works and collaborating like this. Which isn't just shitting on the paper. It's constructive. Edit: Sorry, got a bit ranty. CVPR reviews are out. I'll just say I'm so glad they made a big deal about LLMs to "protect the integrity of reviews" because... LLMs were the problems....

summarity 3y ago

Indeed, this is also one of the worst abstracts to a paper ever. 90% of it is a generic introduction, and the rest goes out of its way to establish any quantitative results.

godelski 3y ago

This is actually something I really hate about ML research and why I get upset when people say "don't need math." I see a lot of unnecessary equations in papers (dear god, we don't have to write attention in every ViT paper or the density function in every diffusion paper) but then no math around where it matters. It feels like a strategy to get people to just turn on autopilot, since that's what you'll naturally do. It's also weird when you get reviewers asking for this. I've had a reviewer reject a work, saying that I should include the equation for attention... in 2023... There's a reason I don't value conferences and why I tell people they aren't good signals for quality. Hell, we're doing more peer review here than most papers get. I'm not unconvinced the work isn't novel (but novelty is a function of reader's background knowledge) but I think their claims are too strong. Were I the reviewer I would reject for sure based on the unsubstantiated claim of lossless alone, without explanation. Even if a work was great and useful (though conferences make this difficult to make such types of fixes). The method does seem interesting and there is evidence for its utility, but I do think it needs a lot of additional clarity and support of the claims. In fact, I'll go ahead and say what's happening here is how I'd love to see modern peer review work, because we should be updating works and collaborating like this. Which isn't just shitting on the paper. It's constructive. Edit: Sorry, got a bit ranty. CVPR reviews are out. I'll just say I'm so glad they made a big deal about LLMs to "protect the integrity of reviews" because... LLMs were the problems....