Deep Learning training was always very expensive but models werent getting such a massive bump in size every year (for state of the art) and now they are just g

Deep Learning training was always very expensive but models werent getting such a massive bump in size every year (for state of the art) and now they are just getting 10x bigger every iteration but AI accelerators / GPUs are getting like 1.5x jump every 2 years so have fun for future AI academia / startups outside US.