4 ms·Note that it processes tokens at speed and cost of a 12B model though.by jug 3y agoNote that it processes tokens at speed and cost of a 12B model though.visarga 3y agoIf it uses 2 experts, they should parallelize so closer to 7B speed?knd775 3y agoMemory bandwidth is still a factor, right?