3 ms·

The correct answer, as always, is the oogabooga text generation webUI, which supports all of the relevant backends: https://github.com/oobabooga/text-generation

The correct answer, as always, is the oogabooga text generation webUI, which supports all of the relevant backends: https://github.com/oobabooga/text-generation-webui https://github.com/oobabooga/text-generation-webui

cypress66 3y ago
Yep. Use ooba. And people who like to RP often use ooba as a backend, and sillitavern as a frontend.
- Roark66 3y ago
  Can it run onnx transformer models? I found optimised onnx models are at least twice the speed of vanilla pytorch on the CPU.