There's also the Fabrice Bellard inference code: https://textsynth.com/technology.html https://textsynth.com/technology.html. He claims up to 41 tokens per seco

There's also the Fabrice Bellard inference code: https://textsynth.com/technology.html https://textsynth.com/technology.html. He claims up to 41 tokens per second on the GPT-Neox 20B model.