Simon's article didn't show local usage. Use one of the one-click installers linked in the README of https://github.com/oobabooga/text-generation-webui https:

Simon's article didn't show local usage. Use one of the one-click installers linked in the README of https://github.com/oobabooga/text-generation-webui https://github.com/oobabooga/text-generation-webui and you're set. Note that just in case you have the hardware necessary to run the biggest available model llama2-70b (for example two RTX 3090 with a total of 48GB of VRAM), there is currently a small bug (with a fix) documented at https://github.com/oobabooga/text-generation-webui/issues/3201 https://github.com/oobabooga/text-generation-webui/issues/32...