There are a variety of tricks for making CPU inference competitive and start-ups who have made a business out of said software e.g. NeuralMagic. But yes the au

There are a variety of tricks for making CPU inference competitive and start-ups who have made a business out of said software e.g. NeuralMagic. But yes the author does not give a substantive position despite his expertise in the area (he’s worked on e.g. the usb TPU product Google used to sell).