Demonstrating good performance from a non-transformer based architecture is cool. I agree though these particular models aren’t that useful given the current la

Demonstrating good performance from a non-transformer based architecture is cool. I agree though these particular models aren’t that useful given the current landscape. I think the intent here is probably to justify training a larger 400B model with their architecture.