4 ms·

I’m sure this will be controversial for some reason, but I think we should mostly reject the OSI’s definitions of “open” anything and leave that to the engineer

I’m sure this will be controversial for some reason, but I think we should mostly reject the OSI’s definitions of “open” anything and leave that to the engineering public. I don’t need a board to tell me what’s open. And in the case of AI, if I can’t train the model from source materials with public source code and end up with the same weights, then it’s not open. I don’t need people to tell me that. OSI approved this and that has turned into a Ministry of Magic approved thinking situation that feels gross to me.

JumpCrisscross 2y ago
> if I can’t train the model from source materials with public source code and end up with the same weights, then it’s not open This is the new cracker/hacker, GIF pronunciation, crypto(currency)/crypto(graphy) mole hill. Like sure, nobody forces you to recognise any word. But the common usage already precludes open training data—that will only get more ensconced as more contracts and jurisdictions embrace it.
strangecasts 2y ago
> And in the case of AI, if I can’t train the model from source materials with public source code and end up with the same weights, then it’s not open. Making training exactly reproducible locks off a lot of optimizations, you are practically not going to get bit-for-bit reproducibility for nontrivial models
- didibus 2y ago
  That's kind of true for normal programs as well, depending on the compiler you use, and if it has non-deterministic processes in it's compilation. But still, it's about being able to reproduce the same build process, and get a true realization of the program, even if not bit-for-bit, it's the same intended program.
- samj 2y ago
  Nobody's asking for exact reproducibility — if the source code produces the software and it's appropriately licensed then it's Open Source. Similarly, if you run the scripts and it produces the model then it's Open Source that happens to be AI. To quote Bruce Perens (definition author): the training data IS the source code. Not a perfect analogy but better than a recipe calling for unicorn horns (e.g., FB/IG social graphs) and other toxic candy (e.g., NYT articles that will get users sued).
didibus 2y ago
I agree. If it's open source, surely I can at least "compile" it myself. If the data is missing, I can't do that. We'll end up with like 5 versions of the same "open source" model, all performing differently because they're all built with their own dataset. And yet, none of those will be considered a fork lol? I don't know what the obsession is either. If you don't want to give others permission to use and modify everything that was used to build the program, why are you wanting to trick me in thinking you are, and still calling it open source?
- rettichschnidi 2y ago
  > If you don't want to give others permission to use and modify everything that was used to build the program, why are you wanting to trick me in thinking you are, and still calling it open source? Because there is an excemption clause in the EU AI Act for free and open source AI.
  - seba_dos1 2y ago
    ...which doesn't rely on any OSI decisions.
rockskon 2y ago
To be fair, OSI approval also deters marketing teams from watering down the definition of open source into worthless feelgood slop.
- tourmalinetaco 2y ago
  That’s already what’s happened though, even with MLMs. Without training data we’re back to modifying a binary file without the original source.
mistrial9 2y ago
in historical warfare, Roman soldiers easily and brutally defeated brave, individualist and social opponents on the battlefield, arguably in markets afterwards. It is a sad and essential lesson that applies to modern situations. In marketing terms, a simple market communication, consistently and diligently applied, in varied contexts and over time, can and usually will take hold despite untold number of individuals who shake their fists at the sky or cut with clever and cruel words that few hear IMHO OSI branding and market communications seem very likely to me to be effective in the future, even if the content is exactly what is being objected to here so vehemently.

JumpCrisscross 2y ago

> if I can’t train the model from source materials with public source code and end up with the same weights, then it’s not open This is the new cracker/hacker, GIF pronunciation, crypto(currency)/crypto(graphy) mole hill. Like sure, nobody forces you to recognise any word. But the common usage already precludes open training data—that will only get more ensconced as more contracts and jurisdictions embrace it.

strangecasts 2y ago

> And in the case of AI, if I can’t train the model from source materials with public source code and end up with the same weights, then it’s not open. Making training exactly reproducible locks off a lot of optimizations, you are practically not going to get bit-for-bit reproducibility for nontrivial models

didibus 2y ago
That's kind of true for normal programs as well, depending on the compiler you use, and if it has non-deterministic processes in it's compilation. But still, it's about being able to reproduce the same build process, and get a true realization of the program, even if not bit-for-bit, it's the same intended program.
samj 2y ago
Nobody's asking for exact reproducibility — if the source code produces the software and it's appropriately licensed then it's Open Source. Similarly, if you run the scripts and it produces the model then it's Open Source that happens to be AI. To quote Bruce Perens (definition author): the training data IS the source code. Not a perfect analogy but better than a recipe calling for unicorn horns (e.g., FB/IG social graphs) and other toxic candy (e.g., NYT articles that will get users sued).

didibus 2y ago

I agree. If it's open source, surely I can at least "compile" it myself. If the data is missing, I can't do that. We'll end up with like 5 versions of the same "open source" model, all performing differently because they're all built with their own dataset. And yet, none of those will be considered a fork lol? I don't know what the obsession is either. If you don't want to give others permission to use and modify everything that was used to build the program, why are you wanting to trick me in thinking you are, and still calling it open source?

rettichschnidi 2y ago
> If you don't want to give others permission to use and modify everything that was used to build the program, why are you wanting to trick me in thinking you are, and still calling it open source? Because there is an excemption clause in the EU AI Act for free and open source AI.
- seba_dos1 2y ago
  ...which doesn't rely on any OSI decisions.

rockskon 2y ago

To be fair, OSI approval also deters marketing teams from watering down the definition of open source into worthless feelgood slop.

tourmalinetaco 2y ago
That’s already what’s happened though, even with MLMs. Without training data we’re back to modifying a binary file without the original source.

mistrial9 2y ago

in historical warfare, Roman soldiers easily and brutally defeated brave, individualist and social opponents on the battlefield, arguably in markets afterwards. It is a sad and essential lesson that applies to modern situations. In marketing terms, a simple market communication, consistently and diligently applied, in varied contexts and over time, can and usually will take hold despite untold number of individuals who shake their fists at the sky or cut with clever and cruel words that few hear IMHO OSI branding and market communications seem very likely to me to be effective in the future, even if the content is exactly what is being objected to here so vehemently.