3 ms·
It's not the best way. It's a really cool and technically interesting way. But embedding the model with the executable is terrible for anything beyond a demo. T
by superkuh 3y ago
It's not the best way. It's a really cool and technically interesting way. But embedding the model with the executable is terrible for anything beyond a demo. The best way would be just running llama.cpp (what llamafile uses) and loading external models. I get that compiling is too difficult for some people and llamafiles are great for them. Maybe even the best for them.
But it's far from best for people that actually want to explore LLM and play.
- breckenedge 3y agoPer the article: > You can also download a much smaller llamafile binary from their releases, which can then execute any model that has been compiled to GGUF format
- dragonwriter 3y agoOr you can download LM Studio or oobabooga and execute any model compiled to GGUF format, and also models in a wide variety of formats that aren't GGUF.
- rodrigobellusci 3y agoFrom what I've read llamafile can also load other models if you pass a flag with a path to them. There's also LM Studio for anyone who'd like to play with a ChatGPT-like GUI.
- brandall10 3y agoI'd argue LM Studio is the best way by far. - Download and launch the app - Use their interface to huggingface models, download the one you want - Go to the chat window and select load - Chat away w/ a ChatGPT like interface that includes markdown processing and easy copying of code blocks, etc Outside of the time to download a model, it's about 30 seconds of work to get up and running
- deleted 3y ago[deleted]