Y
HN Search
Hacker News Search
new
|
comments
|
top
|
jobs
gongy
searching PlanetScale…
1.
▲
2.
▲
3.
▲
4.
▲
5.
▲
6.
▲
3 ms
·
1.
▲
by
gongy
1y ago
The improvement is real! And unlike a lot of research, the code actually runs well. I can reproduce the results using Modal GPUs, leaving the code here: https://github.com/mirage-project/mirage/pull/327/f
2.
▲
by
gongy
3y ago
You can have load balanced inference and also fine tune models on Modal (disclaimer: I wrote this guide) You can get a load balanced endpoint serving hundreds of tok/s per GPU using vLLM or Text Generation Inference in a few lines of c