2 ms·

10x slower is a deal breaker lol. I think max I could tolerate personally is 2x slower for same quality.

• damnitbuilds 5mo ago
  Working on it ! Well, other people are working on it, I'm just trying to get what they do running on my system. MTP looks good at the moment.
  • newaccountman2 5mo ago
    MTP?
    • damnitbuilds 5mo ago
      This explains it ( not sure about the diagram though ): https://www.hardware-corner.net/multi-token-prediction-llm-speed/ https://www.hardware-corner.net/multi-token-prediction-llm-s...
      • newaccountman2 5mo ago
        thnx