Very valid point. The last papers I read on the subject conclusively show that all current networks do not hit a limit in scaling though. Bill Gates said that G

Very valid point. The last papers I read on the subject conclusively show that all current networks do not hit a limit in scaling though. Bill Gates said that GPT five will not be much better than GPT four so maybe things change