3 ms·
Please, please, please call the final product Halfwit. Seriously though, this is a very interesting biologically inspired idea, since not all neuronal pathways
by queuebert 2y ago
Please, please, please call the final product Halfwit.
Seriously though, this is a very interesting biologically inspired idea, since not all neuronal pathways fire all the time.
It seems to follow that, if you can predict which weights you won't need, then you should be able to compress the model architecture permanently, at least for certain use cases.
- kolinko 2y agoHaha halfwit! I’m waiting for such a fork. As for predicting the weights - not necessarily so. It seems most weights are being used, just not all the time. Kind of like that saying that humans are using just 5% of their brain - perhaps they are, but it’s various parts of the 5%. Interestingly, Effort works just as well on MoE, if not better. I did most of the development on Mixtral and I think it go even to 15-20% effort before losing quality, but there is some sort of a bug right now that prevents the inference on Mixtral. It’s on a todo to fix, but I didn’t want to delay the release because of it.
- HPsquared 2y agoHalfweight. Half the weights, half the wait, half the wit.
- LorenDB 2y agoEffortless would be another great name (since you are literally reducing effort to get speed). OK, maybe not "great", but "an option if you're going for puns".
- deleted 2y ago[deleted]