8 ms·
GPT-4 is 1.7T params in the same way that an AMD Ryzen 9 7950X is 72 GHz.
by SheinhardtWigCo 3y ago
GPT-4 is 1.7T params in the same way that an AMD Ryzen 9 7950X is 72 GHz.
- eyegor 3y agoYou might be surprised to learn that dell "hpc engineers" (or maybe HPE?) have attempted to sell me hardware using that exact logic to compare servers.
- lillecarl 3y agoVMware still uses ghz for displaying load and compute capacity on your servers.
- DaiPlusPlus 3y agoWell, they're not wrong... Given a highly-parallelizable numerics job, and all other things being equal, a 20 core 2.0Ghz machine will run the job in the same time as a 40 core 1.0Ghz machine. ...though usually we'd just give it in FLOPS - though FLOPS doesn't describe non-float operation performance. Probably just a sales/marketing person who learned a new technical word and wants to sound knowledgable to the customer but it back-fires.
- zarzavat 3y agoAt the risk of torturing this analogy beyond recognition, it’s more like having an 8 core CPU but with a weak 150W PSU such that you can only run one core at a time.
- SecretDreams 3y agoAnalogy sufficiently tortured
- bee_rider 3y agoIs each core specialized to some particular type of task or something like that? Which would make this less silly of an idea.
- swyx 3y agoin my understanding with an 8 way MoE youre still running all 8 for every request. only if you’re making a router model then itd be the “weak PSU” analogy. and even then a sufficiently perfect router would essentially have to know enough about its routes to almost simulate them which idk if i can believe works
- Faaak 3y agoAmdahl's law though ...
- thfuran 3y ago...doesn't really bite you for embarrassingly parallel numerics work.
- mensetmanusman 3y agoAdd the hurts please!
- WiSaGaN 3y agoI think frequency is not additive like the parameters. If we are looking for analogy in compute power, then FLOPs is a better analogy to parameters.
- Deukhoofd 3y agoI'm fairly certain the point of the gp was that the number of parameters are also not additive.
- ddalex 3y agoif you have a basket with 4 apples and a basket with 3 pears, are you not having 7 fruits ? just because the first selection layer is very thin doesn't mean that the network cannot be considered composable
- deleted 3y ago[deleted]
- nirushiv 3y agoYou can’t claim it’s a 7-fruit Basket, unless you find a way to fit them in the same basket
- swyx 3y agobut you can claim its a 7 fruit grouping of baskets, we’re just in a territory where the fruit count matters less and less now which happens to be consistent with what openai is telling us
- dragonwriter 3y agoParameters are the number of weights needed to specify the model given the architecture; they are exactly additive.
- hgsgm 3y ago
- cochne 3y agoNo, GPT-4 is 1.7T params in the same way that an AMD Ryzen 9 7950X is (24 times X million transistors). It's not wrong to say GPT-4 is 1.7T parameters, your statement is wrong. Parameters can be configured in different ways, just as transistors on a chip can. In the same way that having 24x transistors does not imply 24x performance, having 24x parameters does not imply 24x performance.