3 ms·
Bloomz.cpp: Run multilingual BLOOM model with C++
- osanseviero 4y agobloomz.cpp allows running inference of BLOOM-like models in pure C/C++ (inspired by llama.cpp). It supports all models that can be loaded in transformers for BLOOM. As an example, you can run GPT-4 on your Mac or Pixel! On M1 Pro, you can achieve 16 tokens/sec.
- jerpint 4y agoThis is a big deal, up until now the only reasonable way of using BLOOM was with PETALS or lots of GPUs… great work!