4 ms·With 32k context the upper bound is more like 175B.by MacsHeadroom 3y agoWith 32k context the upper bound is more like 175B.Taek 3y agoIts probably only the 8k model that runs on 2