There's been a lot of thought at Weights & Biases about the tradeoff between competition and collaboration. Competition certainly fosters activity, but it encou

There's been a lot of thought at Weights & Biases about the tradeoff between competition and collaboration. Competition certainly fosters activity, but it encourages behaviors like ensembling everything to eek out a few more hundredths of a percent. This benchmark isn't incentivized in any way other than "let's drive the field forward", so we may see less of that behavior. We've considered benchmarks that proceed in phases: a closed competitive phase for 3 months, then award a prize to the top result, and another prize for best user writeup. Follow that by a collaborative phase where it's more about sharing, teamwork etc. Rinse and repeat. The question of attribution is really interesting. Who made the largest contribution to the development of a model? It could have actually happened in a hallway conversation, or something equally as untrackable. We'd love to hear other peoples' thoughts on this. Stacey on our team has put a lot of thought into these topics and may have more to say here!