3 ms·
So what? TGI also supports this.
by gvd 3y ago
So what? TGI also supports this.
- deleted 3y ago[deleted]
- george_123 3y ago1) when the blog was released, TGI didn’t support paged attention, 2) many people don’t even know about TGI to reduce inference costs.