You mean of PaLM-1? No. PaLM-1 was extremely far off Chinchilla scaling, it's not something you can salvage with just a few more millions of TPU-pod time. Think

You mean of PaLM-1? No. PaLM-1 was extremely far off Chinchilla scaling, it's not something you can salvage with just a few more millions of TPU-pod time. Think more like, hundreds of millions... I think somewhere I estimated PaLM-1 was at something like 5% of Chinchilla-optimal, so they would have to train it 20x more. Obviously, they are not going to, not with methods improving so rapidly, with innovations like UL2. PaLM-1 is just a sunk cost, is all, and a useful historical datapoint (eg. studying inverse scaling). Equally obviously, PaLM-2 did not make the same mistake and so for that and other reasons, greatly outperforms PaLM-1. So, beating PaLM-1 at this point is an achievement, but you are still far from 'beating Google'.