4 ms·

This is great food for thought, thank you

• onlyrealcuzzo 5mo ago
  Additionally, on the context front -> all the labs are aware that for many tasks you can get 10x+ increases in output quality by feeding better context. See https://arxiv.org/abs/2604.04364 https://arxiv.org/abs/2604.04364. This won't really show up in benchmarks, but it will impact real world usage on the most common use cases. I'm doing a study right now on the impacts of better context for small models to fix bugs. A very dumb algorithm can make small models perform at 10x+ model sizes. I'll be surprised if it can't get to 20x+