3 ms·

> Open-Source AI There are no open source AI models, at least not useful ones (yet [1]). Open weight is not the same as open source. "Open weight" models are s

by petcat 19d ago

> Open-Source AI There are no open source AI models, at least not useful ones (yet [1]). Open weight is not the same as open source. "Open weight" models are still just inscrutable binary blobs that you can (theoretically) run on your own computer instead of through a SAAS web app. The open weight model labs don't even provide a high level catalog or any description whatsoever about what went into the training data. This is not open source and we should stop conflating the two things. [1] https://allenai.org/ https://allenai.org/

markasoftware 19d ago
I thought Nemotron tried to be pretty open?
- nessex 19d ago
  Many of the Nemotron datasets are gated behind approval, and a license agreement. The preamble on these datasets is: "This repository is publicly accessible, but you have to accept the conditions to access its files and contents." I don't know what others have experienced, but I requested access to multiple Nemotron datasets and those requests were ignored for months before all but one request was rejected. There's no explanation for why, nor anything I can see which would lead to a rejection. So it's purely anecdotal and YMMV, but I don't see these as being particularly open.
Gracana 19d ago
I think you would be pleasantly surprised by the content of the linked article.
vlyan 19d ago
>The open weight model labs don't even provide a high level catalog or any description whatsoever about what went into the training data. because the training data is full of copyrighted works. the answer to "what went into the training data" is "everything we could get our hands on".
- fc417fc802 18d ago
  For text based just assume "the entirety of anna's archive" and you'll be most of the way there. For multimodal pull up a torrent tracker and sort by size descending if you want to see some of the aggregated datasets.
foopod 19d ago
Completely agree with this, I'm sick of people conflating the two. Open-weight models should be treated no more favourably than proprietary freeware. Sure you can run tests and benchmarks on open-weight models, but that is the extent - there is no scrutiny, no auditing for bias or copyright contamination - just a black box that you rely on for "intelligence". I'm still shocked the way people can hand over not just huge swathes of data, but also decisions of all shapes and sizes - to AI companies with no way of being able to assess how the sausage is made.
- kennywinker 19d ago
  You can create derivative works from open weight models
kennywinker 19d ago
K2 horizon is also fully open source i believe - https://ifm.ai/k2/ https://ifm.ai/k2/ I understand your quibble with terminology, but i think the “inscrutable binary blob” thing is a bit off base. You can create finetunes and post train models using only their open weights. You can’t create derivative works like that from an inscrutable binary blob
alansaber 19d ago
It's not just copyrighted training data. Truly open source e2e model training would include scrapers, data cleaning, all pretraining scripts, posttraining scripts, exact hardware info, etc. Open weights labs will release a sanitized version to make themselves look good / not give too much away.
scoopr 18d ago
This irks me as well. There have been perfectly good words to describe these. The models are freeware, and some have additional restrictions in the license that make them shareware, imho!
Razengan 18d ago
Can "frontier" AI models help with making open-source AI models?
- tancop 17d ago
  Definitely, but make sure you're doing it with Astra. Fable will silently sabotage your work or just refuse to do it. They can help you design challenge generators for RLVR and maybe write training code but you'll be running into safeguards if you want them to make a data scraper.
azath92 18d ago
I think there needs to be some nuance to this. Without the training data and open sourcing the training code and protocol you cant reproduce the model from scratch, thats true, but because the weights and structure are free to download you can modify, inspect and make derivative works off it. Calling it an inscrutable binary blob is misleading. I do agree that its not reproducible and its often talked about like it is. In some ways its like most scientific research with a "methodology section", which in todays academic climate often falls quite a bit short of a detailed step-by-step to reproduce results with all the ingredients, but in principle could be used by other experts to do this again. Even without publishing the exact data, a catalogue or description of what data and how it was cleaned/processed would go a long way towards a reproducible methodology i agree.
- foopod 18d ago
  > you can modify, inspect and make derivative works Even with a team of researchers inspecting the weights and understanding what is going on internally is a tedious operation. Having the training data doesn't really solve any issues here. But what it it does do is allow for auditability and public scrutiny - which imo is pretty important if you rely on these models.

by petcat 19d ago

markasoftware 19d ago

I thought Nemotron tried to be pretty open?

nessex 19d ago
Many of the Nemotron datasets are gated behind approval, and a license agreement. The preamble on these datasets is: "This repository is publicly accessible, but you have to accept the conditions to access its files and contents." I don't know what others have experienced, but I requested access to multiple Nemotron datasets and those requests were ignored for months before all but one request was rejected. There's no explanation for why, nor anything I can see which would lead to a rejection. So it's purely anecdotal and YMMV, but I don't see these as being particularly open.

Gracana 19d ago

I think you would be pleasantly surprised by the content of the linked article.

vlyan 19d ago

>The open weight model labs don't even provide a high level catalog or any description whatsoever about what went into the training data. because the training data is full of copyrighted works. the answer to "what went into the training data" is "everything we could get our hands on".

fc417fc802 18d ago
For text based just assume "the entirety of anna's archive" and you'll be most of the way there. For multimodal pull up a torrent tracker and sort by size descending if you want to see some of the aggregated datasets.

foopod 19d ago

Completely agree with this, I'm sick of people conflating the two. Open-weight models should be treated no more favourably than proprietary freeware. Sure you can run tests and benchmarks on open-weight models, but that is the extent - there is no scrutiny, no auditing for bias or copyright contamination - just a black box that you rely on for "intelligence". I'm still shocked the way people can hand over not just huge swathes of data, but also decisions of all shapes and sizes - to AI companies with no way of being able to assess how the sausage is made.

kennywinker 19d ago
You can create derivative works from open weight models

kennywinker 19d ago

K2 horizon is also fully open source i believe - https://ifm.ai/k2/ https://ifm.ai/k2/ I understand your quibble with terminology, but i think the “inscrutable binary blob” thing is a bit off base. You can create finetunes and post train models using only their open weights. You can’t create derivative works like that from an inscrutable binary blob

alansaber 19d ago

It's not just copyrighted training data. Truly open source e2e model training would include scrapers, data cleaning, all pretraining scripts, posttraining scripts, exact hardware info, etc. Open weights labs will release a sanitized version to make themselves look good / not give too much away.

scoopr 18d ago

This irks me as well. There have been perfectly good words to describe these. The models are freeware, and some have additional restrictions in the license that make them shareware, imho!

Razengan 18d ago

Can "frontier" AI models help with making open-source AI models?

tancop 17d ago
Definitely, but make sure you're doing it with Astra. Fable will silently sabotage your work or just refuse to do it. They can help you design challenge generators for RLVR and maybe write training code but you'll be running into safeguards if you want them to make a data scraper.

azath92 18d ago

I think there needs to be some nuance to this. Without the training data and open sourcing the training code and protocol you cant reproduce the model from scratch, thats true, but because the weights and structure are free to download you can modify, inspect and make derivative works off it. Calling it an inscrutable binary blob is misleading. I do agree that its not reproducible and its often talked about like it is. In some ways its like most scientific research with a "methodology section", which in todays academic climate often falls quite a bit short of a detailed step-by-step to reproduce results with all the ingredients, but in principle could be used by other experts to do this again. Even without publishing the exact data, a catalogue or description of what data and how it was cleaned/processed would go a long way towards a reproducible methodology i agree.

foopod 18d ago
> you can modify, inspect and make derivative works Even with a team of researchers inspecting the weights and understanding what is going on internally is a tedious operation. Having the training data doesn't really solve any issues here. But what it it does do is allow for auditability and public scrutiny - which imo is pretty important if you rely on these models.