3 ms·

How many tokens per second are you getting from Alpaca?

• selectodude 3y ago
  When llamaCPP came out, I was running 13B at 100ms/token on a base model MacBook Pro 14". Edit: apparently llama.cpp supports running on GPU, so I imagine it's gonna be a bit faster. Maybe a fun evening project for me to get going.
  • deleted 3y ago
    [deleted]