5 ms·

EDIT - the title formerly referred to "Qualcomm open sources ...." Calling freely downloadable weights "Open Source" quite diminishes the term. It's laudable,

EDIT - the title formerly referred to "Qualcomm open sources ...." Calling freely downloadable weights "Open Source" quite diminishes the term. It's laudable, so I'd hate to discourage it. But it's not Open Source. What constitutes Open Source is not vague, btw. it's well defined by the OSI [1]. Let's call it open weights or freely downloadable weights or something. EDIT: I was mistaken - they are down sampled/right-sized w half precision: BTW props to Qualcomm but these are "just" quantized versions of existing models? Useful, yes, but maybe not that novel. [1] https://opensource.org/osd https://opensource.org/osd

tobyjsullivan 3y ago
OSI seem like a privately organized special interest group. What authority do they have to define the term “open source”?
- karolist 3y ago
  Valid question. Why is this down voted?
- monocasa 3y ago
  Their original board created the term in the 90s as an alternative to GNU's free software.
  - tobyjsullivan 3y ago
    Oh, thanks. OSI has been around much longer than I expected. This post[0] dives into who coined the term (spoiler: it predates OSI by a long, long time), but it’s reasonable that OSI popularized it alongside their specific definition. [0] https://lunduke.substack.com/p/who-really-coined-the-term-open-source https://lunduke.substack.com/p/who-really-coined-the-term-op... IMHO, to the extent that their goal was to find a less confusing term than “free”, I’d say they’ve failed.
thomastjeffery 3y ago
"open weights" would be just fine. There's no need to overcomplicate this.
ben_w 3y ago
1. What does "source code" even mean, in this context? I don't buy the requirement that all training data needs to also be open sourced for any of the things that people normally discuss with regard to open source topics (though that kind of openness would be good for different reasons). Network weights and biases plus architecture models are things that can be directly build upon, just like graphics are, and I would count a photo licensed under, say, MIT, as "open source" even if the JPEG codec on the camera which took the photo was not. 2. How are these two licenses compatible with https://opensource.org/osd https://opensource.org/osd part 2. Source Code? a. https://opensource.org/license/unicode-license-v3 https://opensource.org/license/unicode-license-v3 - separately lists data files b. https://opensource.org/license/nasa1-3-php https://opensource.org/license/nasa1-3-php - "Notwithstanding any provisions contained herein, Recipient is hereby put on notice that export of any goods or technical data from the United States may require some form of export license from the U.S. Government. Failure to obtain necessary export licenses may result in criminal liability under U.S. laws. Government Agency neither represents that a license shall not be required nor that, if required, it shall be issued. Nothing granted herein provides any such export license."
- szundi 3y ago
  Whatever, but I think the word “source” has a meaning that is pretty sure does not equals “end result binary blob” after a lengthy process
- mvkel 3y ago
  Agreed. It's like saying it's not true "open source" if you don't include then node_modules folder in your git repo
  - Nullabillity 3y ago
    This situation is more like they didn't include the package.json.
- HPsquared 3y ago
  Simply put, it should be reproducible with the published material. An ML model is definitely not reproducible without the training data (or a recipe to reproduce the training data).
bick_nyers 3y ago
Does anyone know of any truly open source LLM model, that's better than GPT2? How do researchers reproduce results in this space?
beeboobaa 3y ago
> What constitutes Open Source is not vague, btw. it's well defined by the OSI [1]. Who is this organisation that they get to mandate the definition of the english language?
- bee_rider 3y ago
  It also has an obvious english meaning. The source of the model isn’t fully open, it is not possible to inspect and modify the input used to build the models.
- mynjin 3y ago
  It's not about that. Open Source is more like a designation. It is an agreed upon set of requirements that, if you change a requirement, it is something else. This is important. Some things have legally protected designations such as 'ice cream'. Ice Cream has specific meaning in industry and even a grading system. If someone wants to make a cheaper product than the lowest grade of ice cream, they can't call it ice cream, they have to call it something like: frozen dairy dessert. This makes it easy for people understand what they are actually getting and paying for. I wouldn't get indignant about mandating english language definitions. I would be indignant that ai companies are not fulfilling the requirements to call it open source and are providing a cheaper product than the abilities that an actual open source model would provide.
wyldfire 3y ago
> What constitutes Open Source is not vague, btw. it's well defined by the OSI [1]. I'm pretty surprised to see objections to my claims here, I kinda thought I was just stating the obvious. > Who is this organisation that they get to mandate the definition of the english language? > What authority do they have to define the term “open source”? This term has meaning and while its meaning started with an group of people forming an organization saying "this is what this means" its meaning doesn't derive from OSI (nor some farcical aquatic ceremony). Its meaning comes from its popular use in language. For example, Wikipedia does describe this term and how it came to be used [1]. IMO it would be unclear and confusing to use the same term Open Source to describe both what it has historically described and how model weights like these are distributed. The term "Open Source" itself was coined to disambiguate merely "open" source from "free-as-in-freedom" source. [1] https://en.wikipedia.org/wiki/Open_source https://en.wikipedia.org/wiki/Open_source
cosmojg 3y ago
The point is moot. Like anything else created by an automated computer process, model weights are not protected under US copyright law. The exclusion of machine-generated works from copyright protection has been pretty well established by the US Copyright Office. In fact, going off letters and rulings recently published by the US Copyright Office[1], even the outputs of generative models are excluded from copyright protection, regardless of the amount of human skill (e.g., prompt composition, parameter selection) involved in their production. IANAL, but at most, publishing weights with a license may amount to little more than a ToS agreement, allowing distributors a bit more leeway in managing their legal/commercial relationship with recipients of said models. In other words, breaking the terms laid out in a text file entitled "LICENSE.txt" and distributed alongside a set of model weights may constitute a breach of contract, but it is in no way a copyright violation. [1] https://fingfx.thomsonreuters.com/gfx/legaldocs/klpygnkyrpg/AI%20COPYRIGHT%20decision.pdf https://fingfx.thomsonreuters.com/gfx/legaldocs/klpygnkyrpg/...
7moritz7 3y ago
Each time I see a comment like this I wonder who declared the open source initiative the single authority on this. I agree with the comment but this really rubs me the wrong way.
GaggiX 3y ago
They almost all seem to run at FP16 precision, so they are not quantized, just optimized to run on Qualcomm NPU.
spywaregorilla 3y ago
downloadable weights are the preferred form to modify the program.
- nwsm 3y ago
  And command line arguments are the preferred form to modify CLI programs. But their existence doesn't make the application open source.
  - spywaregorilla 3y ago
    ... no, command line arguments are the preferred form to interact with CLI programs, not to modify them. edit: I'm confused how this very basic claim got downvoted.
notpushkin 3y ago
Probably bad editorializing on the submitter's part – I don't see open source being mentioned anywhere on Qualcomm's Hugging Face (although at least some models are distributed under an open source license it seems – not unlike blobs in Linux kernel).
imjonse 3y ago
The term open source fell prey to hype, ignorance and VC money in the case of AI models. The most incorrect uses are for literally binary blobs without a recipe to reproduce (no training data in the majority of cases) and for products that are basically a thin layer over OpenAI calls which is not privacy-preserving, local or free.
Brajeshwar 3y ago
Really sorry. I should have double checked the content more properly before copy-pasting the title from the source.
abetusk 3y ago
Sorry, what's not open source about them? I've only checked a few models but they look to be under a BSD-3-clause license. The first few I looked at all have the same BSD-3 license [0] [1] [2]. Are you saying they've just repacked other existing models under their own banner but haven't opened sourced some other component? [0] https://huggingface.co/qualcomm/FFNet-40S https://huggingface.co/qualcomm/FFNet-40S [1] https://huggingface.co/qualcomm/ResNet50 https://huggingface.co/qualcomm/ResNet50 [2] https://huggingface.co/qualcomm/VIT https://huggingface.co/qualcomm/VIT
- wyldfire 3y ago
  > The program must include source code, and must allow distribution in source code as well as compiled form. Can you reproduce these models? if not then it's probably not open source. With a model the closest analog seems to be the training data. Is that all published?
  - visarga 3y ago
    ML training runs are not reproducible, GPUs are non-deterministic when doing large sums, the order operands are added changes the result, thread execution times also depend on caching, which is hard to predict. If you want to force deterministic mode be prepared for a huge slowdown.
    - monocasa 3y ago
      A lot of work at larger orgs is put into reproducible training runs. It's about the only way to debug 'did the small parameter tweak tank performance because of a hardware failure or is there something special about that parameter?'
- sgu999 3y ago
  From the link the the person you're reply to posted: > The program must include source code, and must allow distribution in source code as well as compiled form. The compute graph with trained weight is very much a compiled form of the model. The source code would include everything needed to train that model and reproduce it. EDIT: wasn't fast enough

tobyjsullivan 3y ago

OSI seem like a privately organized special interest group. What authority do they have to define the term “open source”?

karolist 3y ago
Valid question. Why is this down voted?
monocasa 3y ago
Their original board created the term in the 90s as an alternative to GNU's free software.
- tobyjsullivan 3y ago
  Oh, thanks. OSI has been around much longer than I expected. This post[0] dives into who coined the term (spoiler: it predates OSI by a long, long time), but it’s reasonable that OSI popularized it alongside their specific definition. [0] https://lunduke.substack.com/p/who-really-coined-the-term-open-source https://lunduke.substack.com/p/who-really-coined-the-term-op... IMHO, to the extent that their goal was to find a less confusing term than “free”, I’d say they’ve failed.

thomastjeffery 3y ago

"open weights" would be just fine. There's no need to overcomplicate this.

ben_w 3y ago

1. What does "source code" even mean, in this context? I don't buy the requirement that all training data needs to also be open sourced for any of the things that people normally discuss with regard to open source topics (though that kind of openness would be good for different reasons). Network weights and biases plus architecture models are things that can be directly build upon, just like graphics are, and I would count a photo licensed under, say, MIT, as "open source" even if the JPEG codec on the camera which took the photo was not. 2. How are these two licenses compatible with https://opensource.org/osd https://opensource.org/osd part 2. Source Code? a. https://opensource.org/license/unicode-license-v3 https://opensource.org/license/unicode-license-v3 - separately lists data files b. https://opensource.org/license/nasa1-3-php https://opensource.org/license/nasa1-3-php - "Notwithstanding any provisions contained herein, Recipient is hereby put on notice that export of any goods or technical data from the United States may require some form of export license from the U.S. Government. Failure to obtain necessary export licenses may result in criminal liability under U.S. laws. Government Agency neither represents that a license shall not be required nor that, if required, it shall be issued. Nothing granted herein provides any such export license."

szundi 3y ago
Whatever, but I think the word “source” has a meaning that is pretty sure does not equals “end result binary blob” after a lengthy process
mvkel 3y ago
Agreed. It's like saying it's not true "open source" if you don't include then node_modules folder in your git repo
- Nullabillity 3y ago
  This situation is more like they didn't include the package.json.
HPsquared 3y ago
Simply put, it should be reproducible with the published material. An ML model is definitely not reproducible without the training data (or a recipe to reproduce the training data).

bick_nyers 3y ago

Does anyone know of any truly open source LLM model, that's better than GPT2? How do researchers reproduce results in this space?

beeboobaa 3y ago

> What constitutes Open Source is not vague, btw. it's well defined by the OSI [1]. Who is this organisation that they get to mandate the definition of the english language?

bee_rider 3y ago
It also has an obvious english meaning. The source of the model isn’t fully open, it is not possible to inspect and modify the input used to build the models.
mynjin 3y ago
It's not about that. Open Source is more like a designation. It is an agreed upon set of requirements that, if you change a requirement, it is something else. This is important. Some things have legally protected designations such as 'ice cream'. Ice Cream has specific meaning in industry and even a grading system. If someone wants to make a cheaper product than the lowest grade of ice cream, they can't call it ice cream, they have to call it something like: frozen dairy dessert. This makes it easy for people understand what they are actually getting and paying for. I wouldn't get indignant about mandating english language definitions. I would be indignant that ai companies are not fulfilling the requirements to call it open source and are providing a cheaper product than the abilities that an actual open source model would provide.

wyldfire 3y ago

> What constitutes Open Source is not vague, btw. it's well defined by the OSI [1]. I'm pretty surprised to see objections to my claims here, I kinda thought I was just stating the obvious. > Who is this organisation that they get to mandate the definition of the english language? > What authority do they have to define the term “open source”? This term has meaning and while its meaning started with an group of people forming an organization saying "this is what this means" its meaning doesn't derive from OSI (nor some farcical aquatic ceremony). Its meaning comes from its popular use in language. For example, Wikipedia does describe this term and how it came to be used [1]. IMO it would be unclear and confusing to use the same term Open Source to describe both what it has historically described and how model weights like these are distributed. The term "Open Source" itself was coined to disambiguate merely "open" source from "free-as-in-freedom" source. [1] https://en.wikipedia.org/wiki/Open_source https://en.wikipedia.org/wiki/Open_source

cosmojg 3y ago

The point is moot. Like anything else created by an automated computer process, model weights are not protected under US copyright law. The exclusion of machine-generated works from copyright protection has been pretty well established by the US Copyright Office. In fact, going off letters and rulings recently published by the US Copyright Office[1], even the outputs of generative models are excluded from copyright protection, regardless of the amount of human skill (e.g., prompt composition, parameter selection) involved in their production. IANAL, but at most, publishing weights with a license may amount to little more than a ToS agreement, allowing distributors a bit more leeway in managing their legal/commercial relationship with recipients of said models. In other words, breaking the terms laid out in a text file entitled "LICENSE.txt" and distributed alongside a set of model weights may constitute a breach of contract, but it is in no way a copyright violation. [1] https://fingfx.thomsonreuters.com/gfx/legaldocs/klpygnkyrpg/AI%20COPYRIGHT%20decision.pdf https://fingfx.thomsonreuters.com/gfx/legaldocs/klpygnkyrpg/...

7moritz7 3y ago

Each time I see a comment like this I wonder who declared the open source initiative the single authority on this. I agree with the comment but this really rubs me the wrong way.

GaggiX 3y ago

They almost all seem to run at FP16 precision, so they are not quantized, just optimized to run on Qualcomm NPU.

spywaregorilla 3y ago

downloadable weights are the preferred form to modify the program.

nwsm 3y ago
And command line arguments are the preferred form to modify CLI programs. But their existence doesn't make the application open source.
- spywaregorilla 3y ago
  ... no, command line arguments are the preferred form to interact with CLI programs, not to modify them. edit: I'm confused how this very basic claim got downvoted.

notpushkin 3y ago

Probably bad editorializing on the submitter's part – I don't see open source being mentioned anywhere on Qualcomm's Hugging Face (although at least some models are distributed under an open source license it seems – not unlike blobs in Linux kernel).

imjonse 3y ago

The term open source fell prey to hype, ignorance and VC money in the case of AI models. The most incorrect uses are for literally binary blobs without a recipe to reproduce (no training data in the majority of cases) and for products that are basically a thin layer over OpenAI calls which is not privacy-preserving, local or free.

Brajeshwar 3y ago

Really sorry. I should have double checked the content more properly before copy-pasting the title from the source.

abetusk 3y ago

Sorry, what's not open source about them? I've only checked a few models but they look to be under a BSD-3-clause license. The first few I looked at all have the same BSD-3 license [0] [1] [2]. Are you saying they've just repacked other existing models under their own banner but haven't opened sourced some other component? [0] https://huggingface.co/qualcomm/FFNet-40S https://huggingface.co/qualcomm/FFNet-40S [1] https://huggingface.co/qualcomm/ResNet50 https://huggingface.co/qualcomm/ResNet50 [2] https://huggingface.co/qualcomm/VIT https://huggingface.co/qualcomm/VIT

wyldfire 3y ago
> The program must include source code, and must allow distribution in source code as well as compiled form. Can you reproduce these models? if not then it's probably not open source. With a model the closest analog seems to be the training data. Is that all published?
- visarga 3y ago
  ML training runs are not reproducible, GPUs are non-deterministic when doing large sums, the order operands are added changes the result, thread execution times also depend on caching, which is hard to predict. If you want to force deterministic mode be prepared for a huge slowdown.
  - monocasa 3y ago
    A lot of work at larger orgs is put into reproducible training runs. It's about the only way to debug 'did the small parameter tweak tank performance because of a hardware failure or is there something special about that parameter?'
sgu999 3y ago
From the link the the person you're reply to posted: > The program must include source code, and must allow distribution in source code as well as compiled form. The compute graph with trained weight is very much a compiled form of the model. The source code would include everything needed to train that model and reproduce it. EDIT: wasn't fast enough