5 ms·

We raised $100M for open and collaborative machine learning

175 points by minimaxir 5y ago

mistrial9 5y ago
great - somewhere in your game plan, please recognize that not everyone is going to use opaque, trained-elsewhere binary blobs as base models; nor does everyone require CNN/DeepLearning to do useful, real research or invention. Things like python skLearn, an internally built supervised model or a few hundred of them; analysis runs that are done without logging into a cloud with AUTH linked to some records behind a closed glass door somewhere.. If you are FAIR, then you let people try things without being attached to you. agree?
- adamsmith143 5y ago
  >great - somewhere in your game plan, please recognize that not everyone is going to use opaque, trained-elsewhere binary blobs as base models; nor does everyone require CNN/DeepLearning to do useful, real research or invention. Shocking that a company focused on Deep Learning, NLP and CV might focus their efforts on people who do Deep Learning, NLP and CV??
- k8si 5y ago
  You can turn off the part where it logs into the cloud and insert your own logging backend fairly easily. Also if your research/biz needs are satisfied by sklearn, then why not just use sklearn? But for a lot of NLP systems, BERT is actually really really useful. And if you don't want to use a pretrained BERT, you can easily initialize their BERT implementation randomly and train it on your own data.
- visarga 5y ago
  Can't use sklearn so much for NLP. Sure, it's great for small ML tasks, but you need big honking transformer models to compete in the field. On the other hand prompting GPT-3 or fine-tuning text transformers is quite accessible now. About as easy as using sklearn.
m_ke 5y ago
Wish them luck but having gone through something similar with another deep learning company that raised a ton of money, things will probably get a lot harder for them after the C round when investors start looking at revenue and not growth, and with a large valuation that limits the pool of potential acquirers.
- hackernewds 5y ago
  Interesting you mentioned revenue and not net income / profits. How are revenue and growth detached?
  - m_ke 5y ago
    A lot of companies use other metrics to show growth, like github stars and package downloads
- lumost 5y ago
  I recently searched through the ML companies listed on https://topstartups.io/ https://topstartups.io/. One repeating pattern that surprised me is that there are few successful ML startups, the ones that are there don't seem to be remotely close to self-sustaining - or even product market fit in many cases. Why is ML such a struggle? are we overstating the impact relative to "old fashioned" data collection and analytics? Is the tech to expensive for companies to adopt in terms of man hours?
  - joshcryer 5y ago
    A lot of people come up with an idea and then have no product. And investors are just happy to throw money at them. It's that whole idea that if you want to start an AI company, hire a bunch of people in India to do your work, then once the funding round is done, actually build the system. Hugging Face actually has a product and I think they'll be fine. I actually think $100 million is undervalued, because when I think "open models" or "training datasets" I think "Hugging Face." If you really want free money as an AI startup just say you're going to solve safe / friendly AI. People throw money at that without even showing anything. Hugging Face actually recently put out a job for someone who can work in "bias mitigation": https://nitter.net/mmitchell_ai/status/1520483233132990464 https://nitter.net/mmitchell_ai/status/1520483233132990464
ushakov 5y ago
how will you earn that money back to the investors?
- orangepurple 5y ago
  Looks like they are selling a chat bot to Monzo and allegedly Bing uses them for search?!
- w1nk 5y ago
  So this question can only come from a place where you have no idea what they do in their field. For every news article or arxiv post that you see talking about how this amazing new GPT-N model has broken all sorts of language benchmark scores, you'll notice that basically nobody can reproduce those results. That's mostly due to the barrier of entry with respect to hardware for training the models. Huggingface is releasing APIs and model checkpoints that allow any random internet user to execute (almost) SOTA language models in production. FYI - that's an amazing leap forward and a strong piece of kit for MLEs to have access to. So let me rephrase your question: Is general access to SOTA language models worth 100mm to the software market? I suspect the answer is a resounding yes.
  - karpierz 5y ago
    > So let me rephrase your question: Is general access to SOTA language models worth 100mm to the software market? That doesn't answer the original question. The question was: "How will you extract $100M+ from the software market?" Your answer was: "I think that Huggingface will produce $100M worth of value." Which may or may not be true, but just because something produces X amount of value doesn't mean the project will be able to extract that value. See any open source project.
    - w1nk 5y ago
      So if you want someone to answer precisely how they'll extract hundreds of millions of dollars from an emerging market, I have to imagine this isn't the correct forum to expect such answers. Enabling the general software community access to SOTA language models will absolutely unlock an order of magnitude more money (than 100mm) over time. At least for now their obvious strategy for capturing this value is providing these APIs to enable it, and I suspect they'll gladly host such versions for the orgs that don't have the capacity to fine tune / host their own LLMs.
orangepurple 5y ago
What is this? Question: _________ is just an attention mechanism Answer: Huggingface https://www.microsoft.com/en-us/research/uploads/prod/2020/06/Alchemy-with-Friends-ML-Expansion-Pack.pdf https://www.microsoft.com/en-us/research/uploads/prod/2020/0...
orangepurple 5y ago
The brand name only makes me think of Half Life head crabs (they hug the face). Maybe that's the joke on us.
- UnpossibleJim 5y ago
  Oh, I went straight to Alien.
  - spywaregorilla 5y ago
    Same. They're literally face huggers
- wodenokoto 5y ago
  It comes from the emoji[1], but I always think of Boaty McBoatface[2] [1] https://emojipedia.org/hugging-face/ https://emojipedia.org/hugging-face/ [2] https://en.wikipedia.org/wiki/RRS_Sir_David_Attenborough#Naming_poll https://en.wikipedia.org/wiki/RRS_Sir_David_Attenborough#Nam...
minimaxir 5y ago
Hugging Face is one of the few companies that recognizes that creating good OSS (e.g. the transformers Python package) and supporting that with managed services is better than focusing on managed services primarily and having open source as a crippled afterthought designed to drive people to said managed services. An example is HF's push toward ONNX export support for their major AI models in the transformers package, which allows faster model inference and could theoretically compete with their managed inference service.
axg11 5y ago
Congrats to the Huggingface team! They closed this round at a great time. I except there will be much stronger pressure over the next 18-24 months for them to show revenue growth.
throwaway83242 5y ago
We tried huggingface.co with great hope recently. Unfortunately, though their system was well orchestrated, we could not make progress on baby steps. We uploaded our model to github and then downloaded to hugging face. Why the package installed correctly, it failed because underlying Glibc headers were compiled with a version that is different from Hugging face's. So, while the platform works for some situations, it still has a long way to go.
kolbe 5y ago
Getting that kind of raise in the current environment is a huge accomplishment. Congratulations.
lvl102 5y ago
Huggingface + WandB are some of my favorite tools doing ML.
bjourne 5y ago
Congrats. I'm so jealous. Personally, I found Huggingface's models clunky to work with and for me it was easier to build and train my own. I really didn't like how they tried to make a "unified api" and treat PyTorch, TensorFlow 2, etc as backends. Though I realize that for those who are not adept in ml, or don't have the need for much customization, using Huggingface's models makes a lot of sense. No doubt they'll make themselves rich and their investors even richer.
fxtentacle 5y ago
I don't get how they can earn those $100M back without becoming significantly less open. The gist of huggingface is that they host pre-trained open source models for free. Anyone can download them and then use them offline without HF. And their paid offerings aren't even close to being cost competitive with buying a few 3090 workstations. Also, HF doesn't really have any technical moat. Other people research and train those models, they just provide the hosting. In my opinion, their biggest value is the community. But how do you monetize a group of motivated volunteers? What stops GitHub from offering free LFS hosting for AI models, thereby copying the foundation of HFs community? And lastly, is there really much of a market in making SOTA AI beginner-friendly? You still need to buy/rent that A100 GPU server. Who's going to be greedy on salary for people operating a $20k/month machinery?
- skdotdan 5y ago
  The moat is network effects, not the hosting service. GitHub is indeed a potential competitor but, again, you are just thinking about the hosting, while Huggingface provides an API compatible with all the uploaded models and datasets. GitHub would have to create their own API and then hope that users would switch to their service.
  - fxtentacle 5y ago
    Thanks for explaining :)
johntiger1 5y ago
Wonder how this news will impact cohere

175 points by minimaxir 5y ago

great - somewhere in your game plan, please recognize that not everyone is going to use opaque, trained-elsewhere binary blobs as base models; nor does everyone require CNN/DeepLearning to do useful, real research or invention. Things like python skLearn, an internally built supervised model or a few hundred of them; analysis runs that are done without logging into a cloud with AUTH linked to some records behind a closed glass door somewhere.. If you are FAIR, then you let people try things without being attached to you. agree?

adamsmith143 5y ago
>great - somewhere in your game plan, please recognize that not everyone is going to use opaque, trained-elsewhere binary blobs as base models; nor does everyone require CNN/DeepLearning to do useful, real research or invention. Shocking that a company focused on Deep Learning, NLP and CV might focus their efforts on people who do Deep Learning, NLP and CV??
k8si 5y ago
You can turn off the part where it logs into the cloud and insert your own logging backend fairly easily. Also if your research/biz needs are satisfied by sklearn, then why not just use sklearn? But for a lot of NLP systems, BERT is actually really really useful. And if you don't want to use a pretrained BERT, you can easily initialize their BERT implementation randomly and train it on your own data.
visarga 5y ago
Can't use sklearn so much for NLP. Sure, it's great for small ML tasks, but you need big honking transformer models to compete in the field. On the other hand prompting GPT-3 or fine-tuning text transformers is quite accessible now. About as easy as using sklearn.

m_ke 5y ago

Wish them luck but having gone through something similar with another deep learning company that raised a ton of money, things will probably get a lot harder for them after the C round when investors start looking at revenue and not growth, and with a large valuation that limits the pool of potential acquirers.

hackernewds 5y ago
Interesting you mentioned revenue and not net income / profits. How are revenue and growth detached?
- m_ke 5y ago
  A lot of companies use other metrics to show growth, like github stars and package downloads
lumost 5y ago
I recently searched through the ML companies listed on https://topstartups.io/ https://topstartups.io/. One repeating pattern that surprised me is that there are few successful ML startups, the ones that are there don't seem to be remotely close to self-sustaining - or even product market fit in many cases. Why is ML such a struggle? are we overstating the impact relative to "old fashioned" data collection and analytics? Is the tech to expensive for companies to adopt in terms of man hours?
- joshcryer 5y ago
  A lot of people come up with an idea and then have no product. And investors are just happy to throw money at them. It's that whole idea that if you want to start an AI company, hire a bunch of people in India to do your work, then once the funding round is done, actually build the system. Hugging Face actually has a product and I think they'll be fine. I actually think $100 million is undervalued, because when I think "open models" or "training datasets" I think "Hugging Face." If you really want free money as an AI startup just say you're going to solve safe / friendly AI. People throw money at that without even showing anything. Hugging Face actually recently put out a job for someone who can work in "bias mitigation": https://nitter.net/mmitchell_ai/status/1520483233132990464 https://nitter.net/mmitchell_ai/status/1520483233132990464

ushakov 5y ago

how will you earn that money back to the investors?

orangepurple 5y ago
Looks like they are selling a chat bot to Monzo and allegedly Bing uses them for search?!
w1nk 5y ago
So this question can only come from a place where you have no idea what they do in their field. For every news article or arxiv post that you see talking about how this amazing new GPT-N model has broken all sorts of language benchmark scores, you'll notice that basically nobody can reproduce those results. That's mostly due to the barrier of entry with respect to hardware for training the models. Huggingface is releasing APIs and model checkpoints that allow any random internet user to execute (almost) SOTA language models in production. FYI - that's an amazing leap forward and a strong piece of kit for MLEs to have access to. So let me rephrase your question: Is general access to SOTA language models worth 100mm to the software market? I suspect the answer is a resounding yes.
- karpierz 5y ago
  > So let me rephrase your question: Is general access to SOTA language models worth 100mm to the software market? That doesn't answer the original question. The question was: "How will you extract $100M+ from the software market?" Your answer was: "I think that Huggingface will produce $100M worth of value." Which may or may not be true, but just because something produces X amount of value doesn't mean the project will be able to extract that value. See any open source project.
  - w1nk 5y ago
    So if you want someone to answer precisely how they'll extract hundreds of millions of dollars from an emerging market, I have to imagine this isn't the correct forum to expect such answers. Enabling the general software community access to SOTA language models will absolutely unlock an order of magnitude more money (than 100mm) over time. At least for now their obvious strategy for capturing this value is providing these APIs to enable it, and I suspect they'll gladly host such versions for the orgs that don't have the capacity to fine tune / host their own LLMs.

orangepurple 5y ago

What is this? Question: _________ is just an attention mechanism Answer: Huggingface https://www.microsoft.com/en-us/research/uploads/prod/2020/06/Alchemy-with-Friends-ML-Expansion-Pack.pdf https://www.microsoft.com/en-us/research/uploads/prod/2020/0...

orangepurple 5y ago

The brand name only makes me think of Half Life head crabs (they hug the face). Maybe that's the joke on us.

UnpossibleJim 5y ago
Oh, I went straight to Alien.
- spywaregorilla 5y ago
  Same. They're literally face huggers
wodenokoto 5y ago
It comes from the emoji[1], but I always think of Boaty McBoatface[2] [1] https://emojipedia.org/hugging-face/ https://emojipedia.org/hugging-face/ [2] https://en.wikipedia.org/wiki/RRS_Sir_David_Attenborough#Naming_poll https://en.wikipedia.org/wiki/RRS_Sir_David_Attenborough#Nam...

minimaxir 5y ago

Hugging Face is one of the few companies that recognizes that creating good OSS (e.g. the transformers Python package) and supporting that with managed services is better than focusing on managed services primarily and having open source as a crippled afterthought designed to drive people to said managed services. An example is HF's push toward ONNX export support for their major AI models in the transformers package, which allows faster model inference and could theoretically compete with their managed inference service.

axg11 5y ago

Congrats to the Huggingface team! They closed this round at a great time. I except there will be much stronger pressure over the next 18-24 months for them to show revenue growth.

throwaway83242 5y ago

We tried huggingface.co with great hope recently. Unfortunately, though their system was well orchestrated, we could not make progress on baby steps. We uploaded our model to github and then downloaded to hugging face. Why the package installed correctly, it failed because underlying Glibc headers were compiled with a version that is different from Hugging face's. So, while the platform works for some situations, it still has a long way to go.

kolbe 5y ago

Getting that kind of raise in the current environment is a huge accomplishment. Congratulations.

lvl102 5y ago

Huggingface + WandB are some of my favorite tools doing ML.

bjourne 5y ago

Congrats. I'm so jealous. Personally, I found Huggingface's models clunky to work with and for me it was easier to build and train my own. I really didn't like how they tried to make a "unified api" and treat PyTorch, TensorFlow 2, etc as backends. Though I realize that for those who are not adept in ml, or don't have the need for much customization, using Huggingface's models makes a lot of sense. No doubt they'll make themselves rich and their investors even richer.

fxtentacle 5y ago

I don't get how they can earn those $100M back without becoming significantly less open. The gist of huggingface is that they host pre-trained open source models for free. Anyone can download them and then use them offline without HF. And their paid offerings aren't even close to being cost competitive with buying a few 3090 workstations. Also, HF doesn't really have any technical moat. Other people research and train those models, they just provide the hosting. In my opinion, their biggest value is the community. But how do you monetize a group of motivated volunteers? What stops GitHub from offering free LFS hosting for AI models, thereby copying the foundation of HFs community? And lastly, is there really much of a market in making SOTA AI beginner-friendly? You still need to buy/rent that A100 GPU server. Who's going to be greedy on salary for people operating a $20k/month machinery?

skdotdan 5y ago
The moat is network effects, not the hosting service. GitHub is indeed a potential competitor but, again, you are just thinking about the hosting, while Huggingface provides an API compatible with all the uploaded models and datasets. GitHub would have to create their own API and then hope that users would switch to their service.
- fxtentacle 5y ago
  Thanks for explaining :)

johntiger1 5y ago

Wonder how this news will impact cohere