Or Intel! They're not making any high-end cards currently, but the A770 16GB is pretty decent if you're on a budget. The software support isn't really amazing y

Or Intel! They're not making any high-end cards currently, but the A770 16GB is pretty decent if you're on a budget. The software support isn't really amazing yet, but their GPUs have some quite decent matmul acceleration. Unfortunately support for their GPUs is not upstream in Tensorflow or Pytorch, and I don't think they're well-supported by Llama.cpp either, but Intel does look quite promising. I also believe that Intel oneMKL works on Arc GPUs these days, which in CPU land is amongst the fastest BLAS libraries out there. Also their matrix hardware is accessible using OpenCL extensions, which means that rolling a custom kernel for things related to quantization should be quite possible. (For Tensorflow and PyTorch, you need to install a custom package called Intel Extension for $FRAMEWORK. The PyTorch one got updated to PyTorch 2.0 recently, which is promising.) Currently rumors seem to indicate that their second generation GPUs will go from 32 to 64 Xe cores for the top model, and keep the 256 bit bus. If Intel were to double the VRAM to 32 GB as well (at least as an option, just like the A770 comes in both 8GB and 16GB variants), it'd immediately make them the crown of consumer VRAM size, which I'd wager would drive a lot of interest from the ML community.