to me the most interesting part in this all is the quantization used. hardware doing LLMs is likely to be the new norm in few years anyway. with retrofitting ex

to me the most interesting part in this all is the quantization used. hardware doing LLMs is likely to be the new norm in few years anyway. with retrofitting existing hardware to use say USBs running LLM accelerators and alike...