3 ms·
have you considered cosmopolitan? e.g. like llamafile that works on everything up to and including toasters.
by exe34 1y ago
have you considered cosmopolitan? e.g. like llamafile that works on everything up to and including toasters.
- danielhanchen 1y agoOh llamafile is very cool! I might add it as an option actually :) For generic exports (ie to vLLM, llamafile etc), normally finetunes end with model.save_pretrained_merged and that auto merges to 16bit safetensors which allows for further processing downstream - but I'll investigate llamafile more! (good timing since llamafile is cross platform!)