>This assumes that the benchmarks are representative of real usage scenarios. I'm not saying that there is bad faith, but that benchmarking is really hard in th

>This assumes that the benchmarks are representative of real usage scenarios. I'm not saying that there is bad faith, but that benchmarking is really hard in the context of LLMs. It's a fair point, but the conclusion is 'i dont know' I could assume that it gets better because it'll keep to simpler code.