This is pretty amazing. prefill: 16.9337 tokens/sec, decoding: 0.4631 tokens/sec on a Radeon 6600XT on the 7b default model. Does feel something isn't quite r

This is pretty amazing. prefill: 16.9337 tokens/sec, decoding: 0.4631 tokens/sec on a Radeon 6600XT on the 7b default model. Does feel something isn't quite right as it's only using a few % of GPU/CPU - though it is using the AMD GPU! Which I have never managed to get working in Windows or Linux with llama.cpp directly. I wonder if using WebGPU somehow would avoid all the horrendous problems of GPU support in LLMs, as it seems there is some sort of semi-working abstraction layer here which works across M1/M2, Nvidia and AMD?