I don’t know about “zero benefit”—certainly their access to customer queries gives them a gods eye view into how people are using their LM implementation and to

I don’t know about “zero benefit”—certainly their access to customer queries gives them a gods eye view into how people are using their LM implementation and to some degree could probably be used to fine-tune their model—but I doubt it gives them a leg up that couldn’t be matched through the sheer scale of academics and volunteers who contribute to open solutions & research. Maybe we invent a method for directly fine-tuning based on user interactions, or we discover that there’s some shelf where further fine-tuning only confuses the model, or discover some magical architectural modification or dataset that enables high-quality interactions. We’re still at the stage where basic modifications to the model breaks seemingly insurmountable problems. ALiBi for example swaps in a new attention mechanism, literally just a matrix addition operation, and found that it enables inference far past the context limit. I’m ultimately optimistic, their data is valuable but I don’t think it’s insurmountable.