3 ms·
Idk, has anyone tried falcon yet? The support for running it remains nonexistent except for one fork of llama.cpp that isn't integrated into anything. This tren
by moffkalast 3y ago
Idk, has anyone tried falcon yet? The support for running it remains nonexistent except for one fork of llama.cpp that isn't integrated into anything. This trend of every new model breaking compatibility really needs to stop.
- Roark66 3y agoI have and I am running it locally. Mostly in 7b variant which runs pretty much at chatgpt speed (when streaming) on my ryzen 3700 cpu + 32gb ram + 2xnvme in a mirror (it doesn't fit in the memory in its entirety, few gb go into the swap). Of course to run it like that I have to be running nothing else. No xorg, no chromium etc. Just a pure linux console. If you want to try falcon 40b instruct by yourself here I'd a public demo : https://huggingface.co/blog/falcon https://huggingface.co/blog/falcon Go to the bottom of the page.