3 ms·

You forgot to mention that if things continue as they are, a very small group of people will have complete control over this giant library.

hu3 1y ago
It's a concern. But there are open source models.
- zdwolfe 1y ago
  I find it odd that any LLM could be considered open source. Sure the weights are available to download and use, but you can't reasonably reconstruct the output model as it's impractical for an individual to gather a useful dataset or spend $5,000,000+ of GPU time training.
  - _ea1k 1y ago
    Distillation can extract the knowledge from an existing model into a newly trained one. That doesn't solve the cost problem, but costs are steadily coming down.
    - goku12 1y ago
      That's still a crude repurposement of an inscrutable artifact. Open source requires you to share the source data from which that artifact (the model parameters) was created.
- alganet 1y ago
  No, there aren't. There is open source training and inference software. And there are open weights. Those things are not enough to reproduce the training. Even if you had the hardware, you would not be able to recreate llama (for example) because you don't know what data went into the training. That's a very weird library. You can get their summaries, but you don't have access to the original works used when creating it. Sounds terrible, open source or not.
- vineyardmike 1y ago
  Open source model, created at great expense… by a still small cohort of people. There are like a dozen organizations globally creating anything close to state of the art models. The fact that you can use some for free on your own hardware doesn’t change that those weights were trained by a small cohort of people, with training data selected by those people, and fine-tuning and “alignment” created by those people. Sure you can fine-tune the smaller ones yourself, but that still leaves you at the will of original creator.
- ares623 1y ago
  Are you able to build these models from source?

hu3 1y ago

It's a concern. But there are open source models.

zdwolfe 1y ago
I find it odd that any LLM could be considered open source. Sure the weights are available to download and use, but you can't reasonably reconstruct the output model as it's impractical for an individual to gather a useful dataset or spend $5,000,000+ of GPU time training.
- _ea1k 1y ago
  Distillation can extract the knowledge from an existing model into a newly trained one. That doesn't solve the cost problem, but costs are steadily coming down.
  - goku12 1y ago
    That's still a crude repurposement of an inscrutable artifact. Open source requires you to share the source data from which that artifact (the model parameters) was created.
alganet 1y ago
No, there aren't. There is open source training and inference software. And there are open weights. Those things are not enough to reproduce the training. Even if you had the hardware, you would not be able to recreate llama (for example) because you don't know what data went into the training. That's a very weird library. You can get their summaries, but you don't have access to the original works used when creating it. Sounds terrible, open source or not.
vineyardmike 1y ago
Open source model, created at great expense… by a still small cohort of people. There are like a dozen organizations globally creating anything close to state of the art models. The fact that you can use some for free on your own hardware doesn’t change that those weights were trained by a small cohort of people, with training data selected by those people, and fine-tuning and “alignment” created by those people. Sure you can fine-tune the smaller ones yourself, but that still leaves you at the will of original creator.
ares623 1y ago
Are you able to build these models from source?