2 ms·

I agree - I think scaling laws and scaling hypothesis are quite distinct personally. Scaling hypothesis is 'just go bigger with what we have and we'll get AGI',

I agree - I think scaling laws and scaling hypothesis are quite distinct personally. Scaling hypothesis is 'just go bigger with what we have and we'll get AGI', vs scaling laws are 'for these tasks and these models types, these are the empirical trends in performance we see'. I think scaling laws are still really valuable for vision research, but as you say we should not just abandon thinking about things beyond scaling even if we observe good scaling trends.

godelski 4y ago
Yeah I agree with this position. It is also what I see within my own research. But also in my own research I see the vast importance of architecture search. This may not be what the public sees, but I think it is well known to the research community or anyone with hands on experience with these types of models.