3 ms·
I think frequency is not additive like the parameters. If we are looking for analogy in compute power, then FLOPs is a better analogy to parameters.
by WiSaGaN 3y ago
I think frequency is not additive like the parameters. If we are looking for analogy in compute power, then FLOPs is a better analogy to parameters.
- Deukhoofd 3y agoI'm fairly certain the point of the gp was that the number of parameters are also not additive.
- ddalex 3y agoif you have a basket with 4 apples and a basket with 3 pears, are you not having 7 fruits ? just because the first selection layer is very thin doesn't mean that the network cannot be considered composable
- deleted 3y ago[deleted]
- nirushiv 3y agoYou can’t claim it’s a 7-fruit Basket, unless you find a way to fit them in the same basket
- swyx 3y agobut you can claim its a 7 fruit grouping of baskets, we’re just in a territory where the fruit count matters less and less now which happens to be consistent with what openai is telling us
- dragonwriter 3y agoParameters are the number of weights needed to specify the model given the architecture; they are exactly additive.
- hgsgm 3y agoA tree is not a clique.
- wongarsu 3y agoIf you can add FLOPs, and you have a number of cores with the same fixed number of FLOPs/cycle, it follows that you can add their frequency. For the vast majority of real-life workloads there are vast differences between one core having X GHz or Y FLOPs vs lots of cores that sum up to that number. Which is the point GP tried to make.