Google's transformer paper was largely just stripping out parts of an earlier (now apparently largely forgotten) paper/model and running it on Google scale comp

Google's transformer paper was largely just stripping out parts of an earlier (now apparently largely forgotten) paper/model and running it on Google scale compute and data (which the original researchers obviously couldn't afford).