I think we have seen there isn’t a huge incentive to be the firm exploring the cutting edge of this tech when you can save big by waiting for other firms to spe

I think we have seen there isn’t a huge incentive to be the firm exploring the cutting edge of this tech when you can save big by waiting for other firms to spend billions on dead ends and incrementally larger models which at any point can be distilled by a competitor for 1/10th the price of the final training run. I imagine we’ll start to see that model structure and weights are very easy targets of low grade corporate espionage. A company with Amazon’s resources should be able to catch up in under a year if they poach the right talent.