4 ms·Sortof a letdown that after 24 32Gb RAM sticks you only get 6-8 tokens per second.by plagiarist 2y agoSortof a letdown that after 24 32Gb RAM sticks you only get 6-8 tokens per second.Mekoloto 2y agoBut a token is not just a character. "hello how are you today?" - 7 tokens. And this is so much better than I could have imagined in a very short span of time.acchow 2y agoAnd only get to use 20k context length before it OOMs