4 ms·

Do they really? As far as I can tell, diffusers/transformer are open sourced wrappers around torch implementations, there are a bunch of companies that are offe

by htrp 3y ago

Do they really? As far as I can tell, diffusers/transformer are open sourced wrappers around torch implementations, there are a bunch of companies that are offering inference (just like they are), and there is some value in offering S3 storage to allow for model search. If they reach a point where they actively become community-hostile, someone will just fork their codebase and release a web app called "FaceHugger"

jeroenhd 3y ago
Their Git LFS based hosting is used by a lot of AI tools. Huggingface level data storage and transfer would also be VERY costly to accomplish for a small party. Some of these models are several gigabytes in size and downloaded hundreds or thousands of time per day. What they do isn't too hard to replicate, but the price for which they do it is impossible to compete with.
- htrp 3y ago
  If Amazon/Google/Microsoft wanted to own that space, they could do it in a heartbeat. Rumor has it Github is actively working on something similar to HF Hub already
  - mistrial9 3y ago
    agile ; community-minded ; efficient pick one if you are lucky. Those Fortune-12 giants can't do anything "in a heartbeat" .. listen to what that means.. nonsense
  - forgingahead 3y ago
    I've seen some repos on Github host the models on "Releases". Not sure how viable that is long term, but definitely easier to download and access than some random Google Drive link which is typical for a lot of other repos.
- lsmeducation 3y ago
  Lol, I’m sorry what? Has anyone heard of BitTorrent or piracy in general? You’re telling me AI models/data are too big? Someone utilize p2p for the love of god. Pretty sure there’s gigabytes of porn moving (freely) through the internet millions of times per day.
  - jeroenhd 3y ago
    Some models are distributed P2P. They download slower, in my experience, and require special software rather than the usual git commands. P2P model distribution isn't impossible, but it's a lot less interesting to many people, or they would've already spread their models through torrents in the first place.
    - lostmsu 3y ago
      There is just no convenient torrent-based model hub yet. But if HF's server code is open source it should be trivial to make one.
  - esafak 3y ago
    GitTorrent came and went.
- brucethemoose2 3y ago
  CivitAI has somehow managed this over in Stable Diffusion land. In fact, I think the whole community is HF averse. The two most popular frameworks are based around the Stability implemenetaion and file format, not HF diffusers.
  - version_five 3y ago
    Any sense of why they are HF averse?
    - brucethemoose2 3y ago
      I'm not even sure its deliberate. Automatic1111 and ComfyUI implemented the SAI backend/format in the early days because thats all there was, and now they are stuck with it. The intertia is tremendous.
      - Auracle 3y ago
        The popular fork of Automatic1111 SD.Next recently implemented diffusers for its SDXL support. It seems like an odd move to me as what makes Auto1111 so appealing are the extensions and I’m assuming that breaks an awful lot of them, but what do I know?
    - dragonwriter 3y ago
      > Any sense of why they are HF averse? Technology-wise, path dependency from stuff that was built out before HF diffusers were available, and on which more continues to be built in the ecosystem. If you are doing a greenfield project that is mostly standalone, its not an issue, but for existing popular projects and the communities of satellite projects around them, the switching cost is high. In terms of content hosting, well, a fair amount is hosted on HF, but there is a difference of content focus between CivitAI and HF, and a lot of what CivitAI hosts HF probably wouldn’t want to. Also, CivitAI has a UI focused on the narrow space of imagegen, whereas HF is more general.
    - IKantRead 3y ago
      I think they're HF averse because HF is fairly anti-NSFW content and it's pretty clear that civit.ai has a fairly large NFSW focused audience. On top of that HF is just hard to use for your average user. Civit.ai is just "click to download" while HF is "look here's a broken model card... you can figure it out from here". Despite the cute logo, I think most people find that HF comes across as fairly anti-user. Despite having years doing ML related work, I still find HF a bit byzantine to navigate.
- dharmab 3y ago
  Is that so different than some free to play video games? Those are sometimes hundreds of GB and downloaded by millions.
  - coffeebeqn 3y ago
    They come with a finely tuned monetization system built in and are very profitable
version_five 3y ago
They have this "accelerate" framework they push too: https://huggingface.co/docs/accelerate/index https://huggingface.co/docs/accelerate/index I don't know much about it, and I think (but couldn't quickly confirm) it's open source, so your point about a fork still stands, although I don't think that solves everything. If it did, people wouldn't care that e.g. Hashicorp changed their license away from open source.

by htrp 3y ago

jeroenhd 3y ago

Their Git LFS based hosting is used by a lot of AI tools. Huggingface level data storage and transfer would also be VERY costly to accomplish for a small party. Some of these models are several gigabytes in size and downloaded hundreds or thousands of time per day. What they do isn't too hard to replicate, but the price for which they do it is impossible to compete with.

htrp 3y ago
If Amazon/Google/Microsoft wanted to own that space, they could do it in a heartbeat. Rumor has it Github is actively working on something similar to HF Hub already
- mistrial9 3y ago
  agile ; community-minded ; efficient pick one if you are lucky. Those Fortune-12 giants can't do anything "in a heartbeat" .. listen to what that means.. nonsense
- forgingahead 3y ago
  I've seen some repos on Github host the models on "Releases". Not sure how viable that is long term, but definitely easier to download and access than some random Google Drive link which is typical for a lot of other repos.
lsmeducation 3y ago
Lol, I’m sorry what? Has anyone heard of BitTorrent or piracy in general? You’re telling me AI models/data are too big? Someone utilize p2p for the love of god. Pretty sure there’s gigabytes of porn moving (freely) through the internet millions of times per day.
- jeroenhd 3y ago
  Some models are distributed P2P. They download slower, in my experience, and require special software rather than the usual git commands. P2P model distribution isn't impossible, but it's a lot less interesting to many people, or they would've already spread their models through torrents in the first place.
  - lostmsu 3y ago
    There is just no convenient torrent-based model hub yet. But if HF's server code is open source it should be trivial to make one.
- esafak 3y ago
  GitTorrent came and went.
brucethemoose2 3y ago
CivitAI has somehow managed this over in Stable Diffusion land. In fact, I think the whole community is HF averse. The two most popular frameworks are based around the Stability implemenetaion and file format, not HF diffusers.
- version_five 3y ago
  Any sense of why they are HF averse?
  - brucethemoose2 3y ago
    I'm not even sure its deliberate. Automatic1111 and ComfyUI implemented the SAI backend/format in the early days because thats all there was, and now they are stuck with it. The intertia is tremendous.
    - Auracle 3y ago
      The popular fork of Automatic1111 SD.Next recently implemented diffusers for its SDXL support. It seems like an odd move to me as what makes Auto1111 so appealing are the extensions and I’m assuming that breaks an awful lot of them, but what do I know?
  - dragonwriter 3y ago
    > Any sense of why they are HF averse? Technology-wise, path dependency from stuff that was built out before HF diffusers were available, and on which more continues to be built in the ecosystem. If you are doing a greenfield project that is mostly standalone, its not an issue, but for existing popular projects and the communities of satellite projects around them, the switching cost is high. In terms of content hosting, well, a fair amount is hosted on HF, but there is a difference of content focus between CivitAI and HF, and a lot of what CivitAI hosts HF probably wouldn’t want to. Also, CivitAI has a UI focused on the narrow space of imagegen, whereas HF is more general.
  - IKantRead 3y ago
    I think they're HF averse because HF is fairly anti-NSFW content and it's pretty clear that civit.ai has a fairly large NFSW focused audience. On top of that HF is just hard to use for your average user. Civit.ai is just "click to download" while HF is "look here's a broken model card... you can figure it out from here". Despite the cute logo, I think most people find that HF comes across as fairly anti-user. Despite having years doing ML related work, I still find HF a bit byzantine to navigate.
dharmab 3y ago
Is that so different than some free to play video games? Those are sometimes hundreds of GB and downloaded by millions.
- coffeebeqn 3y ago
  They come with a finely tuned monetization system built in and are very profitable

version_five 3y ago

They have this "accelerate" framework they push too: https://huggingface.co/docs/accelerate/index https://huggingface.co/docs/accelerate/index I don't know much about it, and I think (but couldn't quickly confirm) it's open source, so your point about a fork still stands, although I don't think that solves everything. If it did, people wouldn't care that e.g. Hashicorp changed their license away from open source.