3 ms·

This viewpoint doesn't make any sense to me. The weights + inference code are the "source code" for AI. I literally don't know what else you are demanding for t

This viewpoint doesn't make any sense to me. The weights + inference code are the "source code" for AI. I literally don't know what else you are demanding for the "open source" label.

keketi 2mo ago
> I literally don't know what else you are demanding for the "open source" label Training data and code.
kingstnap 2mo ago
If you think of LLMs as programs. The weights and inference code are very much a binary. While the training code and data are the true source. Since if you want to robustly modify the LLM that's actually what you need. But since "compilation" (training) is extremely compute intensive this isn't something accessible to anyone without an entire datacenter. Anyway semantics aside having the binary is still infinitely better than dealing with an api as far as privacy and control go.
- kzrdude 2mo ago
  I agree. But models in difference to compiled binaries, are useful as just weights and can be further refined and post-trained, at least. I don't know LLM theory well enough to say if there's some secret sauce they can hold back that makes training ineffective. Less effective I'm sure, we don't have access to their smart training schemes, but post-training should always be possible IIUC.
  - MeetingsBrowser 2mo ago
    At the risk of taking the analogy too far, I would treat refining like modifying a dynamic library. You can technically modify behavior, but only in a very coarse way. post-training is like writing a wrapper around the binary. It is closer to building on top of than truly modifying, in that you can tailor things to your needs slightly but cannot make fundamental changes to the underlying thing.
    - adastra22 2mo ago
      Are you assuming single-layer LORA fine-tuning on top? Because with open-weights you can do full back-propagation training to mold the model into whatever shape you want. For a stretched analogy, I think it is more like LEGO sets. Someone hands you a 10,000 piece masterpiece, and a box of unused LEGO parts. Hackers on HN object that the LEGO part manufacturing process is not included, you can't make your own parts, etc. But it's LEGO. You can pull apart the model, see how it is constructed, add your own refinements and features, or even redo it from the ground up. In a practical sense having knowledge about the factory making the parts doesn't really matter here.
- adastra22 2mo ago
  That’s not how it works though. Two training runs on the same data don’t produce the same weights. And if you want to modify the AI, you do so by fine tuning the weights not rerunning training. In every respect that matters, the weights are both the binary and the source code together.
  - stavros 2mo ago
    If I want to remove censorship from your open-weights model, how do I do that?
    - Tryk 2mo ago
      The process is called Ablation, there are many ablated models available to download https://en.wikipedia.org/wiki/Ablation_(artificial_intelligence) https://en.wikipedia.org/wiki/Ablation_(artificial_intellige...
    - overfeed 2mo ago
      > ...how do I do that? Put in the work. This is akin to asking how to remove Rust from a Rust project; just because something is legally available to you doesn't mean you wont need to apply dome elbow grease, depending how deep the changes you want are, ablation, fine-tuning, or distillation are tools you can use to remove "censorship"
  - raincole 2mo ago
    By this standard there is no closed source software on PC. You can always modify the binary directly or hook your functions in.
    - riversflow 2mo ago
      Wouldn’t any software that is a client and relies on a proprietary server to function (like most modern AAA games) be “closed source” under this definition? Not that I agree with this definition of open source btw, just saying that there is precedent.
  - 2mo ago
- Vetch 2mo ago
  The weights + the architecture are already 100% of the code, the transformer is just a mathematical expression + helper programs whose sources are provided. The transformer itself is not even a stateful program, so a it is no more a binary than Piet or Tromp's BLC are. It's merely incomprehensible. Training isn't compilation either, since training a model is closer to program induction and the data are samples defining the solution space.
chasd00 2mo ago
> I literally don't know what else you are demanding for the "open source" label. you need to "literally" go read the definition of open source software or even ask an LLM to define it for you. Weights + inference code are not the source code they're more like the compiled binary. Making modifications to the behavior of a model with additional training is like writing a mod for minecraft. Sure, you can change things but it doesn't make it open source. Calling these models "open source" is an old trap that software companies use to use. Free to download but then, once you're fully comitted, the trap snaps shut and you must pay up to continue.
- jmiskovic 2mo ago
  The vendor lock-in trap does not apply here. Users don't actually even run the original weights, they mostly use open source inference engines with open source agent harness. If AI provider decided to start charging, a) released models would not be affected, and b) people would drop it instantly and move to other models. The actual problem is that we know nothing about the training set of any open-weights model. They could be intentionally biased to influence users, from political censorship to brand advertising, or general shaping of cultural norms. You run the model on own hardware not knowing if it is designed to act against you. Having whole chain open source would allow audit and reproducing the results.
satvikpendem 2mo ago
Everything required to recreate the model, ie training code and data, namely.

This viewpoint doesn't make any sense to me. The weights + inference code are the "source code" for AI. I literally don't know what else you are demanding for the "open source" label.

keketi 2mo ago

> I literally don't know what else you are demanding for the "open source" label Training data and code.

kingstnap 2mo ago

If you think of LLMs as programs. The weights and inference code are very much a binary. While the training code and data are the true source. Since if you want to robustly modify the LLM that's actually what you need. But since "compilation" (training) is extremely compute intensive this isn't something accessible to anyone without an entire datacenter. Anyway semantics aside having the binary is still infinitely better than dealing with an api as far as privacy and control go.

kzrdude 2mo ago
I agree. But models in difference to compiled binaries, are useful as just weights and can be further refined and post-trained, at least. I don't know LLM theory well enough to say if there's some secret sauce they can hold back that makes training ineffective. Less effective I'm sure, we don't have access to their smart training schemes, but post-training should always be possible IIUC.
- MeetingsBrowser 2mo ago
  At the risk of taking the analogy too far, I would treat refining like modifying a dynamic library. You can technically modify behavior, but only in a very coarse way. post-training is like writing a wrapper around the binary. It is closer to building on top of than truly modifying, in that you can tailor things to your needs slightly but cannot make fundamental changes to the underlying thing.
  - adastra22 2mo ago
    Are you assuming single-layer LORA fine-tuning on top? Because with open-weights you can do full back-propagation training to mold the model into whatever shape you want. For a stretched analogy, I think it is more like LEGO sets. Someone hands you a 10,000 piece masterpiece, and a box of unused LEGO parts. Hackers on HN object that the LEGO part manufacturing process is not included, you can't make your own parts, etc. But it's LEGO. You can pull apart the model, see how it is constructed, add your own refinements and features, or even redo it from the ground up. In a practical sense having knowledge about the factory making the parts doesn't really matter here.
adastra22 2mo ago
That’s not how it works though. Two training runs on the same data don’t produce the same weights. And if you want to modify the AI, you do so by fine tuning the weights not rerunning training. In every respect that matters, the weights are both the binary and the source code together.
- stavros 2mo ago
  If I want to remove censorship from your open-weights model, how do I do that?
  - Tryk 2mo ago
    The process is called Ablation, there are many ablated models available to download https://en.wikipedia.org/wiki/Ablation_(artificial_intelligence) https://en.wikipedia.org/wiki/Ablation_(artificial_intellige...
  - overfeed 2mo ago
    > ...how do I do that? Put in the work. This is akin to asking how to remove Rust from a Rust project; just because something is legally available to you doesn't mean you wont need to apply dome elbow grease, depending how deep the changes you want are, ablation, fine-tuning, or distillation are tools you can use to remove "censorship"
- raincole 2mo ago
  By this standard there is no closed source software on PC. You can always modify the binary directly or hook your functions in.
  - riversflow 2mo ago
    Wouldn’t any software that is a client and relies on a proprietary server to function (like most modern AAA games) be “closed source” under this definition? Not that I agree with this definition of open source btw, just saying that there is precedent.
- 2mo ago
Vetch 2mo ago
The weights + the architecture are already 100% of the code, the transformer is just a mathematical expression + helper programs whose sources are provided. The transformer itself is not even a stateful program, so a it is no more a binary than Piet or Tromp's BLC are. It's merely incomprehensible. Training isn't compilation either, since training a model is closer to program induction and the data are samples defining the solution space.

chasd00 2mo ago

> I literally don't know what else you are demanding for the "open source" label. you need to "literally" go read the definition of open source software or even ask an LLM to define it for you. Weights + inference code are not the source code they're more like the compiled binary. Making modifications to the behavior of a model with additional training is like writing a mod for minecraft. Sure, you can change things but it doesn't make it open source. Calling these models "open source" is an old trap that software companies use to use. Free to download but then, once you're fully comitted, the trap snaps shut and you must pay up to continue.

jmiskovic 2mo ago
The vendor lock-in trap does not apply here. Users don't actually even run the original weights, they mostly use open source inference engines with open source agent harness. If AI provider decided to start charging, a) released models would not be affected, and b) people would drop it instantly and move to other models. The actual problem is that we know nothing about the training set of any open-weights model. They could be intentionally biased to influence users, from political censorship to brand advertising, or general shaping of cultural norms. You run the model on own hardware not knowing if it is designed to act against you. Having whole chain open source would allow audit and reproducing the results.

satvikpendem 2mo ago

Everything required to recreate the model, ie training code and data, namely.