You do need to guess things like activation functions, number of attention heads, order of attention layers, etc. Often the parameter names reveal something abo

You do need to guess things like activation functions, number of attention heads, order of attention layers, etc. Often the parameter names reveal something about these.