5 ms·

Which is more "democratized": a large language model which can be downloaded, and accessed as a library, originally based from the work of the FAANG giants (e.g

by abeppu 5y ago

Which is more "democratized": a large language model which can be downloaded, and accessed as a library, originally based from the work of the FAANG giants (e.g. huggingface transformers) or an API, where every invocation is a call that flows through Cohere's servers?

vgeek 5y ago
In America, money is speech-- especially in politics, so unfortunately maybe this is a "fitting" use of the word.
vkou 5y ago
"Democratized" in the context of a startup usually doesn't mean letting the public access something, it usually means letting a different group of investors (the funders of the startup) access a market (formerly controlled by a monopoly or oligopoly of established mega-firms).
- samhw 5y ago
  > it usually means letting a different group of investors (the funders of the startup) access a market I guess 'democratize' rolls off the tongue more easily than 'heteroligocratize'.
- BoxOfRain 5y ago
  This reminds me of the whole privatisation vs nationalisation debates in the UK. Labour claimed when it nationalised the railways and other parts of the economy it was giving them "back to the people" in a democratic fashion because they were now owned by the state which it conceives of as an expression of the body politic, rather than corporations which are seen as entirely seperate from the people. The Tories when they privatised these things also claimed they were giving them "back to the people" in a democratic fashion as private individuals could now freely invest if they chose in these companies rather than them being controlled by the state, which it conceives of as something entirely seperate from the people*. "Democracy" and especially "the people" can mean lots of different and completely contradictory things to lots of different people. I'm always very cautious of anyone who invokes "democracy" and "the people" directly, I don't automatically consider them untrustworthy but I prefer a direct argument for a particular idea. If someone's truly a democrat, they'd be unafraid to make their point without such potentially dishonest tactics as people would democratically choose that idea anyway. * I'm massively oversimplifying here - there's such things as Blairism and One-Nation Conservatism which blur these lines enormously.
1vuio0pswjnm7 5y ago
The former. "Data dumps" are privacy-friendly. For example, a user can download Wikipedias dumps and search through them to her hearts content, and never use the network. Zero data collection by third parties. All those observing the network can see is that she downloaded some data dumps. "Web APIs" are "tech" company and surveillance-friendly. Network access is required and all activity is observed and recorded. Web APIs are also used as a means of controlling access to what is often publicly available data/info. The company does not own the data/info, its a middleman. "Too many" requests, API user gets cut off. Too often its publicly available data/info that is being served by APIs. Hard to sell data dumps of public data as a "product". I sometimes see entities that provide "data dumps", e.g., a corpus, for free who then try to restrict usage of it through a license, yet they do not themsleves own the data. Whether they even have the rights to "license" it is debatable. They are never legally challenged so we cannot say for sure. The more interesting issue is whether they had the rights to collect the data thats in it. The so-called "web scraping" issue.
- IIAOPSW 5y ago
  Reminds me of a joke. "They say New Yorkers are selfish and unfriendly, but its all untrue. When I visited, a guy overheard I was a tourist and came right up and offered me a great deal on Staten Island Ferry tickets. Just $7.50 for a round trip!" (In case you don't know, the Staten Island Ferry is free).
  - tsjq 5y ago
    Information arbitrage
yobbo 5y ago
Not only that, democratize would imply that the general public has influence on the project through voting, for example.
- BoxOfRain 5y ago
  I think there's degrees of democracy. There's the directly democratic model at one end and "not hidden away in a black box at $mega_corp" on the other. You can be in favour of democracy without advocating mob rule for example.
tgsovlerkhgsel 5y ago
If the former has equivalent performance (as in output quality) etc., then the former obviously. If the former doesn't exist (at a comparable quality), then the latter is better than the nothing.
beaconstudios 5y ago
When a startup says anything, you should hear "we plan to use this to get hella rich". There is no such thing as a startup with altruistic intent.
- moonchrome 5y ago
  Altruistic startup = a non profit ?
  - beaconstudios 5y ago
    Non profits and charities can certainly be altruistic.
- WanderPanda 5y ago
  But it is also not zero sum. Long term viable companies provide a higher value to their customers than the price they charge.
- pinky1417 5y ago
  To reductio ad absurdum that - every time you see any marketing, eg “Drink Coca-Cola because it’s refreshing”, you should hear “Drink Coca-Cola because it’ll make us money.” To your point though: “democratize x” makes my eyes roll. It’s overused hip marketing speak.
  - beaconstudios 5y ago
    That's accurate though; coca cola don't care about whether you're refreshed unless it's profitable to them. The refreshment is the means towards the end of profit.
- deleted 5y ago
  [deleted]
jasode 5y ago
>Which is more "democratized": a large language model which can be downloaded, and accessed as a library, [...] or an API, where every invocation is a call that flows through Cohere's servers? I do understand the point you're trying to make: local autonomy is superior to cloud access mediated by a private commercial entity. However at this time, we may have a counterintuitive situation where API access is more "democratic" than downloading a huge model. Based on various reports[1], the GPT-3 model was trained on ~45 terabytes of text corpus (Wikipedia + Web Common Crawl + book texts, etc) and the final runtime model (175 billion parameters) requires ~350 gigabytes of RAM. In that case, the model size is ~1% of the training set. So "democractize" depends on how ambitious the user is. If you want to use a very large 350GB RAM model, the cloud model with API will be more accessible by the masses than running on local hardware. Last time I looked, an Intel Xeon motherboard has max ram of 128GB so scaling up to 350GB RAM is not going to be cheap or trivial to build. Let's further extrapolate to a future hypothetical GPT-4 using ~10x multiplier: train on 450 terabytes of text with a model requiring 3.5 terabytes of RAM. How do we make that future huge model accessible to the masses? Probably via a cloud API. Unfortunately, there's an unavoidable hardware capital expense barrier there. [1] https://www.google.com/search?q=gpt-3+350gb+ram https://www.google.com/search?q=gpt-3+350gb+ram
- occamrazor 5y ago
  If OpenAI made the weights and the NN configuration freely available,I am sure that several organizations would offer cheap (or even free with strict rate limits) API access. And users would also be able to run the model locally, if they can afford the hardware.
- WanderPanda 5y ago
  This is a strawman argument. Publishing the code/weights is not mutually exclusive to providing an API
  - jasode 5y ago
    >This is a strawman argument. Publishing the code/weights is not mutually exclusive to providing an API You're misinterpreting my comment. I'm directly addressing this fragment by the gp: >which can be downloaded, and accessed as a library, I'm not making any moral ideology statements about the model's "openness", "transparency", or "intellectual property". As a person very interested in playing with something like GPT-3, I'm talking about practical concerns of even running the model. Some type of cloud API access let's me run experiments today. Hopefully the API cost is reasonable or free with limits. I believe that's true of most researchers because they can't afford the hardware in the near future to run a GPT-3-size model as local library.

by abeppu 5y ago

vgeek 5y ago

In America, money is speech-- especially in politics, so unfortunately maybe this is a "fitting" use of the word.

vkou 5y ago

"Democratized" in the context of a startup usually doesn't mean letting the public access something, it usually means letting a different group of investors (the funders of the startup) access a market (formerly controlled by a monopoly or oligopoly of established mega-firms).

samhw 5y ago
> it usually means letting a different group of investors (the funders of the startup) access a market I guess 'democratize' rolls off the tongue more easily than 'heteroligocratize'.
BoxOfRain 5y ago
This reminds me of the whole privatisation vs nationalisation debates in the UK. Labour claimed when it nationalised the railways and other parts of the economy it was giving them "back to the people" in a democratic fashion because they were now owned by the state which it conceives of as an expression of the body politic, rather than corporations which are seen as entirely seperate from the people. The Tories when they privatised these things also claimed they were giving them "back to the people" in a democratic fashion as private individuals could now freely invest if they chose in these companies rather than them being controlled by the state, which it conceives of as something entirely seperate from the people*. "Democracy" and especially "the people" can mean lots of different and completely contradictory things to lots of different people. I'm always very cautious of anyone who invokes "democracy" and "the people" directly, I don't automatically consider them untrustworthy but I prefer a direct argument for a particular idea. If someone's truly a democrat, they'd be unafraid to make their point without such potentially dishonest tactics as people would democratically choose that idea anyway. * I'm massively oversimplifying here - there's such things as Blairism and One-Nation Conservatism which blur these lines enormously.

1vuio0pswjnm7 5y ago

The former. "Data dumps" are privacy-friendly. For example, a user can download Wikipedias dumps and search through them to her hearts content, and never use the network. Zero data collection by third parties. All those observing the network can see is that she downloaded some data dumps. "Web APIs" are "tech" company and surveillance-friendly. Network access is required and all activity is observed and recorded. Web APIs are also used as a means of controlling access to what is often publicly available data/info. The company does not own the data/info, its a middleman. "Too many" requests, API user gets cut off. Too often its publicly available data/info that is being served by APIs. Hard to sell data dumps of public data as a "product". I sometimes see entities that provide "data dumps", e.g., a corpus, for free who then try to restrict usage of it through a license, yet they do not themsleves own the data. Whether they even have the rights to "license" it is debatable. They are never legally challenged so we cannot say for sure. The more interesting issue is whether they had the rights to collect the data thats in it. The so-called "web scraping" issue.

IIAOPSW 5y ago
Reminds me of a joke. "They say New Yorkers are selfish and unfriendly, but its all untrue. When I visited, a guy overheard I was a tourist and came right up and offered me a great deal on Staten Island Ferry tickets. Just $7.50 for a round trip!" (In case you don't know, the Staten Island Ferry is free).
- tsjq 5y ago
  Information arbitrage

yobbo 5y ago

Not only that, democratize would imply that the general public has influence on the project through voting, for example.

BoxOfRain 5y ago
I think there's degrees of democracy. There's the directly democratic model at one end and "not hidden away in a black box at $mega_corp" on the other. You can be in favour of democracy without advocating mob rule for example.

tgsovlerkhgsel 5y ago

If the former has equivalent performance (as in output quality) etc., then the former obviously. If the former doesn't exist (at a comparable quality), then the latter is better than the nothing.

beaconstudios 5y ago

When a startup says anything, you should hear "we plan to use this to get hella rich". There is no such thing as a startup with altruistic intent.

moonchrome 5y ago
Altruistic startup = a non profit ?
- beaconstudios 5y ago
  Non profits and charities can certainly be altruistic.
WanderPanda 5y ago
But it is also not zero sum. Long term viable companies provide a higher value to their customers than the price they charge.
pinky1417 5y ago
To reductio ad absurdum that - every time you see any marketing, eg “Drink Coca-Cola because it’s refreshing”, you should hear “Drink Coca-Cola because it’ll make us money.” To your point though: “democratize x” makes my eyes roll. It’s overused hip marketing speak.
- beaconstudios 5y ago
  That's accurate though; coca cola don't care about whether you're refreshed unless it's profitable to them. The refreshment is the means towards the end of profit.
deleted 5y ago
[deleted]

jasode 5y ago

>Which is more "democratized": a large language model which can be downloaded, and accessed as a library, [...] or an API, where every invocation is a call that flows through Cohere's servers? I do understand the point you're trying to make: local autonomy is superior to cloud access mediated by a private commercial entity. However at this time, we may have a counterintuitive situation where API access is more "democratic" than downloading a huge model. Based on various reports[1], the GPT-3 model was trained on ~45 terabytes of text corpus (Wikipedia + Web Common Crawl + book texts, etc) and the final runtime model (175 billion parameters) requires ~350 gigabytes of RAM. In that case, the model size is ~1% of the training set. So "democractize" depends on how ambitious the user is. If you want to use a very large 350GB RAM model, the cloud model with API will be more accessible by the masses than running on local hardware. Last time I looked, an Intel Xeon motherboard has max ram of 128GB so scaling up to 350GB RAM is not going to be cheap or trivial to build. Let's further extrapolate to a future hypothetical GPT-4 using ~10x multiplier: train on 450 terabytes of text with a model requiring 3.5 terabytes of RAM. How do we make that future huge model accessible to the masses? Probably via a cloud API. Unfortunately, there's an unavoidable hardware capital expense barrier there. [1] https://www.google.com/search?q=gpt-3+350gb+ram https://www.google.com/search?q=gpt-3+350gb+ram

occamrazor 5y ago
If OpenAI made the weights and the NN configuration freely available,I am sure that several organizations would offer cheap (or even free with strict rate limits) API access. And users would also be able to run the model locally, if they can afford the hardware.
WanderPanda 5y ago
This is a strawman argument. Publishing the code/weights is not mutually exclusive to providing an API
- jasode 5y ago
  >This is a strawman argument. Publishing the code/weights is not mutually exclusive to providing an API You're misinterpreting my comment. I'm directly addressing this fragment by the gp: >which can be downloaded, and accessed as a library, I'm not making any moral ideology statements about the model's "openness", "transparency", or "intellectual property". As a person very interested in playing with something like GPT-3, I'm talking about practical concerns of even running the model. Some type of cloud API access let's me run experiments today. Hopefully the API cost is reasonable or free with limits. I believe that's true of most researchers because they can't afford the hardware in the near future to run a GPT-3-size model as local library.