My gut instinct from reading about the phenomenon says that a “grokked” model of X parameters on Y tokens is not going to outperform an “ungrokked” model with 2

My gut instinct from reading about the phenomenon says that a “grokked” model of X parameters on Y tokens is not going to outperform an “ungrokked” model with 2X parameters on 2Y tokens - since “grokking” uses the same resources as parameter and token scaling, it’s simply not a competitive scaling mechanism at the moment. It might make sense in some applications where some other hard limit (e.g. memory capacity at inference time) occurs before your resource limit AND you would still see good returns on improvements in quality, but I suspect those are still fairly narrow and/or rare applications.