3 ms·

> The problem (right now) is that Open Weight models depend right now on huge companies to spend billion of dollars to train and develop them, all backed up by

by ericmay 3mo ago

> The problem (right now) is that Open Weight models depend right now on huge companies to spend billion of dollars to train and develop them, all backed up by their incentives and their state to support this, while essentially giving away their monetization path. Right... and there are two problems with this: 1. Eventually the capabilities of closed-weight models will just vastly outstrip open-weight models if the underlying assumptions about compute and scale needed are mostly on the mark. So you can release open-weight models and they will have great use cases and applications, but ultimately similar to how you don't use an open-source phone or a budget Android phone from Wal-Mart and you buy an iPhone instead, you will see that although they "do the same thing" one product is clearly superior and you just have to pay for it. For this to not be true... 2. then it incentivizes most (all?) companies, American, Chinese, or European to halt development of models because if you spend all the CAPEX and it can just be copied and turned open-source nobody will invest in that. Given that China is not halting development of proprietary models I believe the current strategy and the subsequent approach to release open-weight models is at best a stall tactic, and at worse a sign of desperation. Open source and the support and development models around it have been great. But folks are a little too dogmatic about it. Open-source software isn't a moral good, and closed-source software isn't a moral wrong either.

cyanydeez 3mo ago
Did you just assume the singularity?
- ericmay 3mo ago
  No
jbm 3mo ago
> Open-source software isn't a moral good Yes it is.
- ericmay 3mo ago
  Prove it
  - Razengan 3mo ago
    No u
    - ericmay 3mo ago
      That's just like, your opinion.... man....
  - Isamu 3mo ago
    I’ll take a swing at it. Open Source is a form of sharing with the wider community. Closed Source is not sharing. Moral good is based on doing good outside of your own benefit (the opposite of selfishness.) Ergo it’s a kind of moral good. And I’m not even an advocate for open source.
    - ericmay 3mo ago
      This argument boils down to X is good, therefore more of X is good. But you can see how that breaks down with even trivial examples. Not a great argument. The second piece of this "a moral good is based on doing good outside of your own benefit" - says who? Why? This logic is also faulty. You're also cargo-cutting self-interest in here as a moral failure when many good things depend on humans acting in their own self interest. For example I completely and selfishly installed a new tree at my house. But the community benefits from carbon capture, shade, &c. I understand the sentiment you have here and I think for everyday use and having some guiding principles it is probably fine, but don't confuse this for a principle that is actually examined. You can find contradictions rather easily, never mind solid arguments which expose cases where what you think is true is not really true and so forth.
      - Isamu 3mo ago
        Thanks for the discussion. >This argument boils down to X is good, therefore more of X is good. No, I only argued that it was a moral good, the kind of good. I actually may disagree with others about whether you should pursue a good just because it’s good. >says who? Why? Good question, it’s just a common framing that I see in classical discussions. I didn’t intend for it to be exclusive, I think there’s moral good outside of that. >don't confuse this for a principle that is actually examined I hear you, I think this is a simplified version suitable for an online comment. In particular I’m not saying that if you do something other than a moral good then you are doing something wrong. There are many actions that are morally neutral. Also it is possible to construct artificial situations where you may violate some moral good in pursuit of another.
- Natfan 3mo ago
  is open-source malware a moral good?
Danox 3mo ago
Open source is in the tradition of humans sharing past knowledge, long-term we just can’t keep a secret it’s a time, honored tradition…
chasd00 3mo ago
It's important also that open-weight isn't open source. If you can't download the training data (fully labeled), source code of the NN, and follow the README to build and train it yourself assuming oyu had the hardware then it's not open source. tldr there's no "source" in open weight models therefore they are not open source.
- msdz 3mo ago
  Exactly. AFAIK none of the popular "open Chinese" models have published the full pre- and post-training pipeline, so the models are only partially open, if at all (plus the openly shared final weights, of course).
rurp 3mo ago
Software being open source has many strong positive externalities. It advances human knowledge and freedom. If you don't think that counts as a moral good then I'm baffled by what you think a moral good is.
- ericmay 3mo ago
  It depends on how it is applied. You can release open source software that advances knowledge and freedom that results in economic destruction or the loss of life, for example.
derangedHorse 3mo ago
Imagine there’s a school where all the kids there are being tutored by the best. Also imagine a bunch of neighboring schools drastically falling behind that would need insane amounts of money to keep up. This becomes a problem because all the kids from the rich school will dominate the order schools. They’ll get even more money as time goes on from their kids paying it forward to the point where all other kids are bound to work for them. Now let’s say one other school does have the money for best tutors, BUT they know they’ll run out pretty quickly. Instead of trying to compete in a losing game, they decide to give every school in the world access to their elite lesson plan. Now, for a time, everyone will be on close to a level playing field. If the other schools improve upon their own lesson plans and keep sharing them with others, one day the elite school will wake up to find they are no longer on top. The parents have started to move their kids to other schools because the rich school is no longer attractive at the high cost they charge students
- ericmay 3mo ago
  Sure and to complete your analogy here, the rich schools realize that the curriculum they develop and put a lot of time and money into creating is just used by the cheap schools, so they stop developing it because nobody loses money for long and so neither the rich or cheap schools develop any new curriculum. Now what? The fundamental problem here is incentives and tactics. Either the models are actually better (which I think the iPhone to cheap Android phone really speaks to, i.e. they do the same thing but one is 50x better at 5x-10x the cost) and thus they can be gate kept and like the iPhone the vast majority of profits go to a select few with high end implementations. OR the models aren't actually that much better, companies lose a fortune and then nobody can create any better commercial models or build out scale needed for open source models because it's not profitable. We could wind up with only open-source models or something along those lines, but if the compute and scale is needed to train the models, nobody will be able to do that profitably and so AI research is either gate kept and silo'd for something like military applications or it just doesn't really happen because there's no funding for this scale of build out.
  - derangedHorse 3mo ago

by ericmay 3mo ago

cyanydeez 3mo ago

Did you just assume the singularity?

ericmay 3mo ago
No

jbm 3mo ago

> Open-source software isn't a moral good Yes it is.

ericmay 3mo ago
Prove it
- Razengan 3mo ago
  No u
  - ericmay 3mo ago
    That's just like, your opinion.... man....
- Isamu 3mo ago
  I’ll take a swing at it. Open Source is a form of sharing with the wider community. Closed Source is not sharing. Moral good is based on doing good outside of your own benefit (the opposite of selfishness.) Ergo it’s a kind of moral good. And I’m not even an advocate for open source.
  - ericmay 3mo ago
    This argument boils down to X is good, therefore more of X is good. But you can see how that breaks down with even trivial examples. Not a great argument. The second piece of this "a moral good is based on doing good outside of your own benefit" - says who? Why? This logic is also faulty. You're also cargo-cutting self-interest in here as a moral failure when many good things depend on humans acting in their own self interest. For example I completely and selfishly installed a new tree at my house. But the community benefits from carbon capture, shade, &c. I understand the sentiment you have here and I think for everyday use and having some guiding principles it is probably fine, but don't confuse this for a principle that is actually examined. You can find contradictions rather easily, never mind solid arguments which expose cases where what you think is true is not really true and so forth.
    - Isamu 3mo ago
      Thanks for the discussion. >This argument boils down to X is good, therefore more of X is good. No, I only argued that it was a moral good, the kind of good. I actually may disagree with others about whether you should pursue a good just because it’s good. >says who? Why? Good question, it’s just a common framing that I see in classical discussions. I didn’t intend for it to be exclusive, I think there’s moral good outside of that. >don't confuse this for a principle that is actually examined I hear you, I think this is a simplified version suitable for an online comment. In particular I’m not saying that if you do something other than a moral good then you are doing something wrong. There are many actions that are morally neutral. Also it is possible to construct artificial situations where you may violate some moral good in pursuit of another.
Natfan 3mo ago
is open-source malware a moral good?

Danox 3mo ago

Open source is in the tradition of humans sharing past knowledge, long-term we just can’t keep a secret it’s a time, honored tradition…

chasd00 3mo ago

It's important also that open-weight isn't open source. If you can't download the training data (fully labeled), source code of the NN, and follow the README to build and train it yourself assuming oyu had the hardware then it's not open source. tldr there's no "source" in open weight models therefore they are not open source.

msdz 3mo ago
Exactly. AFAIK none of the popular "open Chinese" models have published the full pre- and post-training pipeline, so the models are only partially open, if at all (plus the openly shared final weights, of course).

rurp 3mo ago

Software being open source has many strong positive externalities. It advances human knowledge and freedom. If you don't think that counts as a moral good then I'm baffled by what you think a moral good is.

ericmay 3mo ago
It depends on how it is applied. You can release open source software that advances knowledge and freedom that results in economic destruction or the loss of life, for example.

derangedHorse 3mo ago

Imagine there’s a school where all the kids there are being tutored by the best. Also imagine a bunch of neighboring schools drastically falling behind that would need insane amounts of money to keep up. This becomes a problem because all the kids from the rich school will dominate the order schools. They’ll get even more money as time goes on from their kids paying it forward to the point where all other kids are bound to work for them. Now let’s say one other school does have the money for best tutors, BUT they know they’ll run out pretty quickly. Instead of trying to compete in a losing game, they decide to give every school in the world access to their elite lesson plan. Now, for a time, everyone will be on close to a level playing field. If the other schools improve upon their own lesson plans and keep sharing them with others, one day the elite school will wake up to find they are no longer on top. The parents have started to move their kids to other schools because the rich school is no longer attractive at the high cost they charge students

ericmay 3mo ago
Sure and to complete your analogy here, the rich schools realize that the curriculum they develop and put a lot of time and money into creating is just used by the cheap schools, so they stop developing it because nobody loses money for long and so neither the rich or cheap schools develop any new curriculum. Now what? The fundamental problem here is incentives and tactics. Either the models are actually better (which I think the iPhone to cheap Android phone really speaks to, i.e. they do the same thing but one is 50x better at 5x-10x the cost) and thus they can be gate kept and like the iPhone the vast majority of profits go to a select few with high end implementations. OR the models aren't actually that much better, companies lose a fortune and then nobody can create any better commercial models or build out scale needed for open source models because it's not profitable. We could wind up with only open-source models or something along those lines, but if the compute and scale is needed to train the models, nobody will be able to do that profitably and so AI research is either gate kept and silo'd for something like military applications or it just doesn't really happen because there's no funding for this scale of build out.
- derangedHorse 3mo ago