Yeah, this is running in 8bit mode. The 30b 8bit version we released seems to do a lot better but it requires significantly more compute. https://huggingface.c

Yeah, this is running in 8bit mode. The 30b 8bit version we released seems to do a lot better but it requires significantly more compute. https://huggingface.co/baseten/alpaca-30b https://huggingface.co/baseten/alpaca-30b