3 ms·
Wait a few month and they will have a distilled model with the same performance and 1% of the run cost.
by Kuinox 2y ago
Wait a few month and they will have a distilled model with the same performance and 1% of the run cost.
- peepeepoopoo98 2y ago100X efficiency improvement (doubtful) still means that costs grow 200X faster than benchmark performance.
- deleted 2y ago[deleted]
- achierius 2y agoEven assuming that past rates of inference cost scaling hold up, we would only expect a 2 OoM decrease after about a year or so. And 1% of 3.5b is still a very large number.
- popcorncowboy 2y agoAnd to your point "past performance is not indicative of future results". The extrapolate to infinity approach is the mindfever of this field.