4 ms·
Are you going to continue to train to a larger param size, say 13b or 30b?
by winddude 3y ago
Are you going to continue to train to a larger param size, say 13b or 30b?
- brucethemoose2 3y agoThere is definitely a demand for a 30B model (aka a model that will comfortably fit on 24GB GPUs (or 32GB of system RAM) and squeeze into 16GB).
- nullc 3y agoLlama.cpp inference on fast cpus is perfectly usable for 70B parameter models.
- deleted 3y ago[deleted]