3 ms·
To me that's still a lot. A whole AA battery for a single response. Maybe I'm just built this way.
by 0xfedbee 3y ago
To me that's still a lot. A whole AA battery for a single response. Maybe I'm just built this way.
- thewataccount 3y agoThey do add up a bit. I'm concerned about a future where we do this for _everything_ because that'll truly be a massive scale unless we get smaller models. If we go off 10M queries a day that's 0.0033KW/H / 24 = 0.0001375/D 0.0001375KW/D * 10M = 1375KW/D or 1.375MW/D (if I did that right)