3 ms·

Runs nicely on my AMD 7900XTX too.

• kkzz99 2y ago
  How are AMD cards performing? I heard it was still very hit and miss in regards to support. Have you also tried things like F5-TTS?
  • zamalek 2y ago
    7900XT here, previously 6900XT. Support for 6000 series and beyond is great on Linux. You have to use an override envar (Ollama has info on this in their readme). ComfyUI has instructions for anything based on torch. TensorFlow is unusable.
• elorant 2y ago
  Care to say how many tokens per second you're getting?
  • delusional 2y ago
    For the "How many r's in the word strawberry" total duration: 15.278476756s load duration: 14.982999ms prompt eval count: 47 token(s) prompt eval duration: 5ms prompt eval rate: 9400.00 tokens/s eval count: 377 token(s) eval duration: 15.257s eval rate: 24.71 tokens/s
    • pton_xd 2y ago
      15 seconds to count the number of r's in a string using 120 TFlops... Really puts into perspective that we've got a long way yet to go.
      • sangnoir 2y ago
        This specific model is designed to second-guess itself/ chack it's work. So the slowness is part of the point.