That simply isn't true. LLM weights are an output of a model training process. They are also an input of a model inference process. Providing weights for people

That simply isn't true. LLM weights are an output of a model training process. They are also an input of a model inference process. Providing weights for people to use does not change the source code in any way, shape, or form. While a model does require weights in order to function, there is nothing about the model that requires you to use any weights provided by anybody, regardless of how they were trained. The model is open source. You can train a Llama 3.1 model from scratch on your proprietary collection of alien tentacle erotica, and you can do so precisely because the model is open source. You can claim that the weights themselves are not open source, but that says absolutely nothing about the model being open source. The weights distributed are simply not required. But more importantly, under your definition, there will never exist in any form a useful open source set of weights. Because almost all data is proprietary. Anybody can train on large quantities of proprietary data without permission using fair use protections, but no matter what you can't redistribute it without permission. Any weights derived from training a model on data that can be redistributed by a single entity would inherently be so tiny that it would be almost useless. You could create a model with a few billion parameters that could memorize it all verbatim. Open weights can be useful, and they can be a huge boon to users that don't have the resources to train large models, but they aren't required for any meaningful definition of open source.