Y
HN Search
Hacker News Search
new
|
comments
|
top
|
jobs
ow5
searching PlanetScale…
1.
▲
2.
▲
3.
▲
4.
▲
5.
▲
6.
▲
3 ms
·
1.
▲
by
ow5
1y ago
Hi! one of the contributors to the paper — we have kernels not released yet that can shave down decoding latency by >20%. Also when we ran experiments for streaming with the current kernels, we were median ~1.3x slower at inference