4 ms·

Huggingface has it built in.

• azinman2 7mo ago
  Where?
  • hhh 7mo ago
    In your preferences there is a local apps and hardware, I guess it's a little different because I just open the page of a model and it shows the hardware I've configured and shows me what quants fit.
    • Twirrim 7mo ago
      I haven't seen a page on HF that'll show me "what models will fit", it's always model by model. The shared tool gives a list of a whole bunch of models, their respective scores, and an estimated tok/s, so you can compare and contrast. I wish it didn't require to run on the machine though. Just let me define my spec on a web page and spit out the results.