3 ms·
Yeah, it's not like foundational models ever share compute kernels, or anything.
by throwawaymaths 2y ago
Yeah, it's not like foundational models ever share compute kernels, or anything.
- moffkalast 2y agoEh, there's so much shenanigans these days even in fine tuning, people adding empty layers and pruning and whatnot, it's unlikely that even models based on the same one will have the same architecture. For new foundation models it's even worse, because there's some fancy experiment every time and the llama.cpp team needs two weeks to figure out how to implement it so the model can even run.
- throwawaymaths 2y agoYeah so you might need to implement a weird activation or positional enciding in software or something but I suspect 90% will probably be the same... If it's just layer count or skipped matrices I assume it should be possible to write an orchestrator that could run most of those models... Unless we move to mamba or something