I have and I am running it locally. Mostly in 7b variant which runs pretty much at chatgpt speed (when streaming) on my ryzen 3700 cpu + 32gb ram + 2xnvme in a

I have and I am running it locally. Mostly in 7b variant which runs pretty much at chatgpt speed (when streaming) on my ryzen 3700 cpu + 32gb ram + 2xnvme in a mirror (it doesn't fit in the memory in its entirety, few gb go into the swap). Of course to run it like that I have to be running nothing else. No xorg, no chromium etc. Just a pure linux console. If you want to try falcon 40b instruct by yourself here I'd a public demo : https://huggingface.co/blog/falcon https://huggingface.co/blog/falcon Go to the bottom of the page.