4 ms·

I guess Sama plans on manufacturing growth metrics by forcing YC companies to pretend that they're using this. Generic machine learning APIs are a shitty busin

by m_ke 6y ago

I guess Sama plans on manufacturing growth metrics by forcing YC companies to pretend that they're using this. Generic machine learning APIs are a shitty business to get into unless you plan on hiring a huge sales team and selling to dinosaurs or doing a ton of custom consulting work, which doesn't scale the way VCs like it to. Anybody who will have enough know how to use their API properly can jus grab an open source model and tune it on their own data. If they plan on commercializing things they should focus on building real products.

wildermuthn 6y ago
I imagine they’re considering offering GPT-3, which would be cost prohibitive to fine-tune for most people. I also I heard inference was too slow to be practical. Perhaps they have some FPGA magic up their Microsoft sleeves.
- m_ke 6y ago
  Nobody is putting these huge models in production, even the smaller transformer models are still too expensive to run for most use cases. With the way the field is moving, GPT-3 will be old news in a month, when more advances are made and open sourced.
  - wildermuthn 6y ago
    Precisely my point. If they could put a model as large as GPT-3 into production (at a reasonable price to the consumer), wouldn’t that be a 10x improvement?
    - krallistic 6y ago
      GPT-3 isn't a 10X improvement. (At least from everything we know so far.)
      - wildermuthn 6y ago
        If the OP is right that nobody is putting the largest models into production (which I think is in inaccurate statement), then GPT-3 in production would be a 10x (ok, 5x?) improvement over the small GPT-2s and BERTS in production? So 10x in practice, if the hypothesis is correct? Which like I said, I don’t believe to be the case.
  - fongitosous 6y ago
    i don't understand. if they run it for you and you apply transfer learning and fine tuning on your specific use case that would reduce drastically the costs hence why their offer make sense
    - deleted 6y ago
      [deleted]
deleted 6y ago
[deleted]
antris 6y ago
Not everyone wants to be an admin to their infrastructure. Real existing services like Heroku and Squarespace exist as useful services because even though you might know how to design and build a website from scratch, sometimes you just need something done quickly without too much worrying about details of the system that do not matter for your project at this point. I really don't see how this wouldn't apply to AI projects as well. I could make a much better site coding my own website from scratch and setting up servers myself, but for some projects I wouldn't even think about it that way, because using Heroku or Squarespace I can save a LOT of time and get the results I need much quicker.
- m_ke 6y ago
  That's true, but machine learning models are not twilio or sendgrid, you have to tune them for your use case, monitor their performance and handle the uncertainty of their outputs. Doing that well requires a data scientist and if you have one they will be much more productive iterating on their own models instead of depending on a 3rd party black box.
  - Grimm1 6y ago
    Except the point of these larger transformer models is they generalize well over a wide range of domains or only require a small amount of transfer learning for really specific domains. I'd say they're perfect candidates for the API as a service model.
  - zoopdewoop 6y ago
    I'm pretty sure people said the exact same thing about Algolia when it was getting started (you have to tune search for your use case! How could you possibly use a search provider?!?) Truth about the situation: - Transformers generalize well and don't need much fine tuning - OpenAI can probably fine tune for your use case better than you can - Getting new models into production takes 6 months to a year at companies of this size, if you did have Data Scientists in house, it might just be better to go with a solution like this for velocity - Not every company has the talent to make an in house ML program successful.
  - 6y ago
dang 6y ago
> I guess Sama plans on manufacturing growth metrics by forcing YC companies to pretend that they're using this. That's wrong in almost too many ways to list. Sam left YC over a year ago, nor would he do such a thing. Nor does YC have that kind of power over companies, nor would it use it that way if it did. That would be wrong and also dumb.
- m_ke 6y ago
  Sorry, that was supposed to be sarcastic. What I meant to say is that Sam has a huge network and is a phone call away from pitching any CEO in the valley. One of the biggest benefits of YC these days is the huge network of companies in your portfolio, which makes getting intros and pilots a lot easier, leading to "traction" and more VC dollars.

by m_ke 6y ago

wildermuthn 6y ago

I imagine they’re considering offering GPT-3, which would be cost prohibitive to fine-tune for most people. I also I heard inference was too slow to be practical. Perhaps they have some FPGA magic up their Microsoft sleeves.

m_ke 6y ago
Nobody is putting these huge models in production, even the smaller transformer models are still too expensive to run for most use cases. With the way the field is moving, GPT-3 will be old news in a month, when more advances are made and open sourced.
- wildermuthn 6y ago
  Precisely my point. If they could put a model as large as GPT-3 into production (at a reasonable price to the consumer), wouldn’t that be a 10x improvement?
  - krallistic 6y ago
    GPT-3 isn't a 10X improvement. (At least from everything we know so far.)
    - wildermuthn 6y ago
      If the OP is right that nobody is putting the largest models into production (which I think is in inaccurate statement), then GPT-3 in production would be a 10x (ok, 5x?) improvement over the small GPT-2s and BERTS in production? So 10x in practice, if the hypothesis is correct? Which like I said, I don’t believe to be the case.
- fongitosous 6y ago
  i don't understand. if they run it for you and you apply transfer learning and fine tuning on your specific use case that would reduce drastically the costs hence why their offer make sense
  - deleted 6y ago
    [deleted]

deleted 6y ago

[deleted]

antris 6y ago

Not everyone wants to be an admin to their infrastructure. Real existing services like Heroku and Squarespace exist as useful services because even though you might know how to design and build a website from scratch, sometimes you just need something done quickly without too much worrying about details of the system that do not matter for your project at this point. I really don't see how this wouldn't apply to AI projects as well. I could make a much better site coding my own website from scratch and setting up servers myself, but for some projects I wouldn't even think about it that way, because using Heroku or Squarespace I can save a LOT of time and get the results I need much quicker.

m_ke 6y ago
That's true, but machine learning models are not twilio or sendgrid, you have to tune them for your use case, monitor their performance and handle the uncertainty of their outputs. Doing that well requires a data scientist and if you have one they will be much more productive iterating on their own models instead of depending on a 3rd party black box.
- Grimm1 6y ago
  Except the point of these larger transformer models is they generalize well over a wide range of domains or only require a small amount of transfer learning for really specific domains. I'd say they're perfect candidates for the API as a service model.
- zoopdewoop 6y ago
  I'm pretty sure people said the exact same thing about Algolia when it was getting started (you have to tune search for your use case! How could you possibly use a search provider?!?) Truth about the situation: - Transformers generalize well and don't need much fine tuning - OpenAI can probably fine tune for your use case better than you can - Getting new models into production takes 6 months to a year at companies of this size, if you did have Data Scientists in house, it might just be better to go with a solution like this for velocity - Not every company has the talent to make an in house ML program successful.
- 6y ago

dang 6y ago

> I guess Sama plans on manufacturing growth metrics by forcing YC companies to pretend that they're using this. That's wrong in almost too many ways to list. Sam left YC over a year ago, nor would he do such a thing. Nor does YC have that kind of power over companies, nor would it use it that way if it did. That would be wrong and also dumb.

m_ke 6y ago
Sorry, that was supposed to be sarcastic. What I meant to say is that Sam has a huge network and is a phone call away from pitching any CEO in the valley. One of the biggest benefits of YC these days is the huge network of companies in your portfolio, which makes getting intros and pilots a lot easier, leading to "traction" and more VC dollars.