4 ms·
Not to be confused with llama.cpp and the GGML library, which is a seperate project (and almost immediately worked with Gemma).
by brucethemoose2 3y ago
Not to be confused with llama.cpp and the GGML library, which is a seperate project (and almost immediately worked with Gemma).
- throwaway19423 3y agoI am confused how all these things are able to interoperate. Are the creators of these models following the same IO for their models? Won't the tokenizer or token embedder be different? I am genuinely confused by how the same code works for so many different models.
- brucethemoose2 3y agoIt's complicated, but basically because most are llama architecture. Meta all but set the standard for open source llms when they released llama1, and anyone trying to deviate from it has run into trouble because the models don't work with the hyper optimized llama runtumes. Also, there's a lot of magic going on behind the scenes with configs stored in gguf/huggingface format models, and the libraries that use them. There are different tokenizers, but they mostly follow the same standards.
- null_point 3y agoI found the magic! https://github.com/search?q=repo%3Aggerganov%2Fggml%20magic&type=code https://github.com/search?q=repo%3Aggerganov%2Fggml%20magic&...
- null_point 3y agoHey, c'mon now. Just being playful about the "magic" string used in GGUF files to detect that it is in-fact a GGUF file.
- jebarker 3y agoI doubt there'd be confusion as the names are totally different