4 ms·
I am all local VoltaML for stable diffusion imagegen (as AITemplate is really fast, and controlnet and LORAs are super useful) Llama.cpp for textgen... I use
by brucethemoose2 3y ago
I am all local
VoltaML for stable diffusion imagegen (as AITemplate is really fast, and controlnet and LORAs are super useful)
Llama.cpp for textgen... I use the koboldcpp frontend (and sometimed AI Horde) for experimenting with models, though I am still figuring out the exact stack I want for productivity. I am watching this PR closely:
https://github.com/ggerganov/llama.cpp/pull/1773 https://github.com/ggerganov/llama.cpp/pull/1773
As well as Starcoder/Openllama finetunes for summarization, codegen and such.
- thepaulthomson 3y agoOh, nice flag on that PR Thanks for surfacing that! I've been hearing a lot about AITemplate on Twitter. Seems the consensus is it's hella quick.
- brucethemoose2 3y agoYeah, its insanely fast compared to PyTorch triton. And it supports dynamic input too. I think its overlooked in Stable Diffusion land because its (last I checked) linux/wsl only and HF diffusers based.
- jyu 3y agofor text, is there a standard to compare model results?
- brucethemoose2 3y agoThere are tons of metrics people have come up with, for example look at the huggingface leaderboard. There are more niche leaderboards/tests for chat models, chain of thought, summarization and such. But the best test is personal experimentation. Prompt engineering and subjective preference have a massive effect on finetune performance.