Nobody cares about FLOPs anymore when it comes to transformer based architectures and considering the transformer architecture is applicable to almost any useca

Nobody cares about FLOPs anymore when it comes to transformer based architectures and considering the transformer architecture is applicable to almost any usecase beyond LLMs, memory bandwidth and memory capacity are the most important metrics. Grayskull has enough 16bit float performance to outrun its memory bandwidth even with 1 ternary bit quantization.