2 ms·
Cache sharing is not possible. The numbers in the cache are completely specific to the model.
by inigyou 4mo ago
Cache sharing is not possible. The numbers in the cache are completely specific to the model.
- xyzsparetimexyz 4mo agoCurrently. I'm sure that you could make a system where the cache values are a superset C of e.g. models A and B where C is probably bigger than max(A,B) but smaller than A+B
- inigyou 4mo agoIt's equal to A+B. There is literally no sharing possible.
- xyzsparetimexyz 4mo agoEven when training both models together in a novel way?
- wonnage 4mo agoReddit post so take with a grain of salt but this does seem possible. But unclear whether this is actually a viable architecture https://www.reddit.com/r/LocalLLaMA/comments/1t8s83r/nvidia_ai_releases_star_elastic_one_checkpoint/ https://www.reddit.com/r/LocalLLaMA/comments/1t8s83r/nvidia_...