Thanks for pointing that out, I didn't notice that. That makes sense. I still think a comparison with exllamav2 or other optimized inference library would make

Thanks for pointing that out, I didn't notice that. That makes sense. I still think a comparison with exllamav2 or other optimized inference library would make sense too.