7 ms·

I believe mlx will allow you to run the models marginally faster (per a recent blog post by @simonw)

• simonw 1y ago
  Yeah, you don't necessarily need it but it's optimized for Apple Silicon and in my experience feels like it gives slightly better performance than GGUFs. I really need to formally measure that so I'm not just running on vibes!
  • indigodaddy 1y ago
    I for one, am willing to just trust you bro ;)
    • turnsout 1y ago
      Yeah I’ll go with Simon’s vibes over most people’s measurements!