"Scaling things up" in and of itself is a very recent phenomenon. I would claim, in that sense that nothing before GPT-3 counts. However, for not-so-scaled mod

"Scaling things up" in and of itself is a very recent phenomenon. I would claim, in that sense that nothing before GPT-3 counts. However, for not-so-scaled models, Meta or rather FAIR, had good contributions with available code that were interoperable with other research in the field. Compare that with, say, Google research/brain/deepmind, which always put out lofty papers that never allowed reproducibility.