3 ms·
I've been using this library a lot, it's amazing. However, I noticed a very considerable degradation (time taken + generation quality) with versions > 0.0.58 wh
by ilovefood 3y ago
I've been using this library a lot, it's amazing. However, I noticed a very considerable degradation (time taken + generation quality) with versions > 0.0.58 when used with local LLMs.
I haven't taken time to compare between the different releases but if anyone is having the same type of issues, I recommend downgrading even if it might mean less features.