7 ms·

I don't think most of the large customers will care about OSS AI. Over the last decade they've learned (trained themselves?) where to put their money towards

by ugh123 3y ago

I don't think most of the large customers will care about OSS AI. Over the last decade they've learned (trained themselves?) where to put their money towards (cloud vs. in-house infra for all manner of things, for better or worse) and I think AI tools will follow similar trends. Businesses will certainly care about cost, but just as important will be: - Customization and fine-tuning capabilities (also 'white labeling' where appropriate) - Integrations (with 3rd party and in-house services & data stores) - SLA & performance concerns - Safety features Open Source AI will have a place, but may be more towards personal-use and academic work. And it will certainly drive competition with the major players (OpenAI, Google, etc) and push them to innovate more which is starting to play out now.

nullc 3y ago
Companies that aren't mindful of vendor lock in aren't long for the world. Though those cloud platforms all have their own proprietary components most users are savvy enough to constrain and compartmentalize their use of them lest they find themselves having all their profits taken by a platform that knows it can set its prices arbitrarily. The cloud vs in-house adoption is what it is in large part because the cloud offerings are a commodity and a big part of them being a commodity is that much of the underlying software is free software.
- deltree7 3y ago
  History is littered with companies that went dead because they focused on things that don't matter (open source, anti-microsoft, pro-linux). There will be a time when those things matter when it hurts the bottom-line (Dropbox), but to prematurely optimize for that while you are finding product-market-fit is crazy and all companies are finding product-market-fit in the new AI era
- jacobr1 3y ago
  On the other-hand, companies that fall behind their competitors because they are spending time on adjacent activities rather than leveraging new capabilities on the market will also lose out. It isn't clear at this time that LLMs fall into that categories ... and smart applications of in-house systems can be as much as an advantage as badly done NIH projects can be an albatross.
ibains 3y ago
A lot of B2B startups can technically the cloud API to provide value added applications to Enterprises, but often the banks and healthcare companies will not want their data running through startups pipes to OpenAI pipes. We provide a low code data transformation product (prophecy.io), and we’ll never close sales at any volume, if we have a to get an MSA that approves this. Might get easier if we become large :)
hdjjhhvvhga 3y ago
> I don't think most of the large customers will care about OSS AI. One would think the same in the 90s but yet, for some reason, Open Source prevailed and took over the world. I don't believe it was about cost, at least not only. In my career I had to evaluate many technical solutions and products and OSS was often objectively superior at several levels without taking account the cost. The first really successful alternative to "Open"AI will: * gather many talented developers * will quickly become a de facto standard solution * people will rapidly start developing a wide range of integrations for it * everybody will be using it, including large orgs, because, well, it's open source
- ugh123 3y ago
  True, but the difference here is that running a performant and capable AI solution will be infrastructure-dependent, which has real costs.
  - hdjjhhvvhga 3y ago
    Yes, and I believe it will develop in two ways over decades. Just like all major legacy hosting companies such as DigitalOcean, OVH or Hetzner have been offering a kind of public cloud services (at various levels, and after much feet dragging), they - and new AI hosting providers focusing exclusively on this use case - will provide the necessary services. The other trend is the one we are already seeing right now: more and more mature solutions that you can use even on your laptop with a relatively new GPU. I'm sure we'll see some interesting results in this area, too.
- deleted 3y ago
  [deleted]
- jacobr1 3y ago
  Open Source hasn't really taken over the world, if you look at end-solutions. As a software developer I might use an open source database, but as end-user I'm probably not going to use open-source accounting package - but I will use an accounting SaaS system that happens to be implemented with that OSS DB. As a software developer I might use an OSS operating system, but as end-user I use a software that has been packaged and maintained by corporation like OSX, or even if OSS in license, has been fully packaged like Android.
ebiester 3y ago
Yes, but I think for most companies this has more to do with cost. They're not going to pay for the OSS model, and if they can use an OSS model + fine tuning, they'll choose to save the money.
simonw 3y ago
Here's a really important reason to care about open source models: prompt engineering is fiddly enough without the risk of your model provider "upgrading" the model you are using in a way that breaks your existing prompts. OpenAI already upset a lot of (admittedly non-paying academic) users when they shut off access to the old Ada code model with only a few week's notice.
- spacebanana7 3y ago
  I’m curious about how enterprises will manage model upgrades. On one hand, as you mention, upgrades could break or degrade prompts in ways that are hard to fix. However, these models will need constant streams of updates for bugs and security fixes just like any other piece of software. Plus the temptation to get better performance. The decisions around how and whether to upgrade LLMs will be much more complicated than upgrading Postgres versions.
  - Vecr 3y ago
    Why would the models themselves need security fixes? The software running the models, sure, but you should be able to upgrade that without changing anything observable about the actual model.
    - jonplackett 3y ago
      Not necessarily. The insecurity from LLMs comes from the fact they’re a black box - what if it turns out that particular version can be easily tricked into giving out terrorism ideas. You could try to add safeguards on top, but they’ve already been bypassed if it has been used for something like that. You might just have to retrain it somehow to make it safe
    - spacebanana7 3y ago
      LLMs (at least the ones with read/write memory) can exactly simulate the execution of a universal Turing machine [1]. AFAIK running such models will therefore entails the same fundamental security risks as ordinary software. [1] https://arxiv.org/pdf/2301.04589.pdf https://arxiv.org/pdf/2301.04589.pdf
lannisterstark 3y ago
>I don't think most of the large customers will care about OSS AI Problem again, is centralization of LLMs by either the governments (and they always act in your best interest, amirite?) and corporation, which Non-FOSS LLMs prevent. Democratization of the models is the only way to actually prevent bad actors from doing bad things. "But they'll then have access to it too" you say. Yes, they will, but given how many more people who will also have access to open LLMs we'd have tools to prevent actually malicious acts.
- warkdarrior 3y ago
  A good guy with an LLM stops a bad guy with an LLM. - This message brought to you by the National LLM Association
  - lannisterstark 3y ago
    Ah yes, the only other alternative is to give companies and the State sole access to LLMs, and guns, and drones, and militarized police. Because they'll surely only think of YOUR benefits, right?
dragonwriter 3y ago
> I don't think most of the large customers will care about OSS AI. OSS AI will open up more diverse and useful services than the first-party offerings from relatively risk averse major vendors, which customers *will" care about.
taneq 3y ago
The thing to remember when selling to businesses is that a business is just a stack of people in a trench coat. This might sound a tad evil but you don’t have to offer something that benefits the business as a whole, just something that benefits the person who holds the purse strings. This is why cloud services are so popular. They’re easy and they don’t cost the decision makers personally.

by ugh123 3y ago

nullc 3y ago

Companies that aren't mindful of vendor lock in aren't long for the world. Though those cloud platforms all have their own proprietary components most users are savvy enough to constrain and compartmentalize their use of them lest they find themselves having all their profits taken by a platform that knows it can set its prices arbitrarily. The cloud vs in-house adoption is what it is in large part because the cloud offerings are a commodity and a big part of them being a commodity is that much of the underlying software is free software.

deltree7 3y ago
History is littered with companies that went dead because they focused on things that don't matter (open source, anti-microsoft, pro-linux). There will be a time when those things matter when it hurts the bottom-line (Dropbox), but to prematurely optimize for that while you are finding product-market-fit is crazy and all companies are finding product-market-fit in the new AI era
jacobr1 3y ago
On the other-hand, companies that fall behind their competitors because they are spending time on adjacent activities rather than leveraging new capabilities on the market will also lose out. It isn't clear at this time that LLMs fall into that categories ... and smart applications of in-house systems can be as much as an advantage as badly done NIH projects can be an albatross.

ibains 3y ago

A lot of B2B startups can technically the cloud API to provide value added applications to Enterprises, but often the banks and healthcare companies will not want their data running through startups pipes to OpenAI pipes. We provide a low code data transformation product (prophecy.io), and we’ll never close sales at any volume, if we have a to get an MSA that approves this. Might get easier if we become large :)

hdjjhhvvhga 3y ago

> I don't think most of the large customers will care about OSS AI. One would think the same in the 90s but yet, for some reason, Open Source prevailed and took over the world. I don't believe it was about cost, at least not only. In my career I had to evaluate many technical solutions and products and OSS was often objectively superior at several levels without taking account the cost. The first really successful alternative to "Open"AI will: * gather many talented developers * will quickly become a de facto standard solution * people will rapidly start developing a wide range of integrations for it * everybody will be using it, including large orgs, because, well, it's open source

ugh123 3y ago
True, but the difference here is that running a performant and capable AI solution will be infrastructure-dependent, which has real costs.
- hdjjhhvvhga 3y ago
  Yes, and I believe it will develop in two ways over decades. Just like all major legacy hosting companies such as DigitalOcean, OVH or Hetzner have been offering a kind of public cloud services (at various levels, and after much feet dragging), they - and new AI hosting providers focusing exclusively on this use case - will provide the necessary services. The other trend is the one we are already seeing right now: more and more mature solutions that you can use even on your laptop with a relatively new GPU. I'm sure we'll see some interesting results in this area, too.
deleted 3y ago
[deleted]
jacobr1 3y ago
Open Source hasn't really taken over the world, if you look at end-solutions. As a software developer I might use an open source database, but as end-user I'm probably not going to use open-source accounting package - but I will use an accounting SaaS system that happens to be implemented with that OSS DB. As a software developer I might use an OSS operating system, but as end-user I use a software that has been packaged and maintained by corporation like OSX, or even if OSS in license, has been fully packaged like Android.

ebiester 3y ago

Yes, but I think for most companies this has more to do with cost. They're not going to pay for the OSS model, and if they can use an OSS model + fine tuning, they'll choose to save the money.

simonw 3y ago

Here's a really important reason to care about open source models: prompt engineering is fiddly enough without the risk of your model provider "upgrading" the model you are using in a way that breaks your existing prompts. OpenAI already upset a lot of (admittedly non-paying academic) users when they shut off access to the old Ada code model with only a few week's notice.

spacebanana7 3y ago
I’m curious about how enterprises will manage model upgrades. On one hand, as you mention, upgrades could break or degrade prompts in ways that are hard to fix. However, these models will need constant streams of updates for bugs and security fixes just like any other piece of software. Plus the temptation to get better performance. The decisions around how and whether to upgrade LLMs will be much more complicated than upgrading Postgres versions.
- Vecr 3y ago
  Why would the models themselves need security fixes? The software running the models, sure, but you should be able to upgrade that without changing anything observable about the actual model.
  - jonplackett 3y ago
    Not necessarily. The insecurity from LLMs comes from the fact they’re a black box - what if it turns out that particular version can be easily tricked into giving out terrorism ideas. You could try to add safeguards on top, but they’ve already been bypassed if it has been used for something like that. You might just have to retrain it somehow to make it safe
  - spacebanana7 3y ago
    LLMs (at least the ones with read/write memory) can exactly simulate the execution of a universal Turing machine [1]. AFAIK running such models will therefore entails the same fundamental security risks as ordinary software. [1] https://arxiv.org/pdf/2301.04589.pdf https://arxiv.org/pdf/2301.04589.pdf

lannisterstark 3y ago

>I don't think most of the large customers will care about OSS AI Problem again, is centralization of LLMs by either the governments (and they always act in your best interest, amirite?) and corporation, which Non-FOSS LLMs prevent. Democratization of the models is the only way to actually prevent bad actors from doing bad things. "But they'll then have access to it too" you say. Yes, they will, but given how many more people who will also have access to open LLMs we'd have tools to prevent actually malicious acts.

warkdarrior 3y ago
A good guy with an LLM stops a bad guy with an LLM. - This message brought to you by the National LLM Association
- lannisterstark 3y ago
  Ah yes, the only other alternative is to give companies and the State sole access to LLMs, and guns, and drones, and militarized police. Because they'll surely only think of YOUR benefits, right?

dragonwriter 3y ago

> I don't think most of the large customers will care about OSS AI. OSS AI will open up more diverse and useful services than the first-party offerings from relatively risk averse major vendors, which customers *will" care about.

taneq 3y ago

The thing to remember when selling to businesses is that a business is just a stack of people in a trench coat. This might sound a tad evil but you don’t have to offer something that benefits the business as a whole, just something that benefits the person who holds the purse strings. This is why cloud services are so popular. They’re easy and they don’t cost the decision makers personally.