As said in the article, all of the providers run different runtimes with proprietary, sometimes untested and weird optimisations There are multiple types of qu

As said in the article, all of the providers run different runtimes with proprietary, sometimes untested and weird optimisations There are multiple types of quantisation: weights and KV Cache. Quantizing kv cache can drastically hurt performance