4 ms·

Is it faster than webgpu? If so, what makes it so?

• lxe 3y ago
  It is not. It's much slower. But it doesn't require you to use a canary browser with a command line flag, so that's that. EDIT: I guess I should give credit here... 100-200ms/token with a 1.7B model is not slow. Would love to do a benchmark with webgpu and see how it compares.
  • terrycody 3y ago
    Sorry for a layman like me, what does this differ with ChatGPT web interface on openAI website? I can't get it what it does.
    • 3np 3y ago
      It runs offline locally on your machine, as opposed to on remote servers.