No, many schemes are acceptably fast. Eg isogenies, code based signatures, some lattice schemes, some hash based schemes. The key concern usually stems from the

No, many schemes are acceptably fast. Eg isogenies, code based signatures, some lattice schemes, some hash based schemes. The key concern usually stems from the size of the primitive (signature size or public key size)