Y
HN Search
Hacker News Search
new
|
comments
|
top
|
jobs
tgaddair
searching PlanetScale…
1.
▲
2.
▲
3.
▲
4.
▲
5.
▲
6.
▲
4 ms
·
1.
▲
by
tgaddair
3y ago
In LoRAX v0.8 we've added native integration with Outlines, allowing you to guarantee your output always comes back in the structure of your choosing. But while structured generation can guarantee the right format comes back, it cannot
2.
▲
LoRAX and Outlines: Better JSON Extraction with Structured Generation and LoRA
(predibase.com)
7 points
by
tgaddair
3y ago
|
2 comments
3.
▲
by
tgaddair
3y ago
Yes, we (LoRAX devs) saw that (we know the author pretty well). It's a useful addition, though quite a bit simpler than our level of support for multi-LoRA inference. We're planning on doing a more comprehensive comparison soon, n
4.
▲
by
tgaddair
3y ago
Thanks for the shoutout! We are adding the Outlines integration to LoRAX in the next two weeks, so stay tuned!
5.
▲
by
tgaddair
3y ago
Hey, LoRAX dev here. This was one thing we spent a lot of effort optimizing. The TL;DR is that in most cases latency will be with 80% of the baseline latency with 0 adapters with as many as 128 adapters at once under heavy request load. Che