3 ms·
Did similar with a 4000 series APU. It’s a bit weak for LLMs under near any circumstances but thinking it could probably handle two whisper cpp decodes over op
by Havoc 3y ago
Did similar with a 4000 series APU.
It’s a bit weak for LLMs under near any circumstances but thinking it could probably handle two whisper cpp decodes over opencl and maybe some embeddings api stuff.