3 ms·
If you have to come up with a custom format anyway, why not just make it a draft extension to GGUF layout definitions (something like "coalesced expert fetch" o
by zozbot234 6mo ago
If you have to come up with a custom format anyway, why not just make it a draft extension to GGUF layout definitions (something like "coalesced expert fetch" or the like) and submit it for inclusion in the standard? Then future models could be autoconverted to such a format.
- adrian_b 6mo agoThis is a good suggestion. I will consider to do this after I gather enough experience to determine which is the best layout and when I will have enough benchmark data to support that.