I can't currently download the model, can people comment on the architecture. Is it a decoder only model? Is it using multi query attention? How many attention

I can't currently download the model, can people comment on the architecture. Is it a decoder only model? Is it using multi query attention? How many attention heads in all variants, how big is the context window? I wish they just published these details.