4 ms·

> but it's hard to see situations where the local approach beats out the cloud approach. I think the most glaring situation where this is true is simply one of

> but it's hard to see situations where the local approach beats out the cloud approach. I think the most glaring situation where this is true is simply one of trust and privacy. Cloud solutions involve trusting 3rd parties with data. Sometimes that fine, sometimes it's really not. Personally - LLMs start to feel more like they're sitting in the confidant/peer space in many ways. I behave differently when I know I'm hitting a remote resource for LLMs in the same way that I behave differently when I know I'm on camera in person: Less genuinely. And beyond merely trusting that a company won't abuse or leak my data, there are other trust issues as well. If I use an LLM as a digital assistant - I need to know that it's looking out for me (or at least acting neutrally) and not being influenced by a 3rd party to give me responses that are weighted to benefit that 3rd party. I don't think it'll be too long before we see someone try to create an LLM that has advertising baked into it, and we have very little insight into how weights are generated and used. If I'm hitting a remote resource - the model I'm actually running can change out from underneath me at any time, jarring at best and utterly unacceptable at worst. From my end - I'd rather pay and run it locally, even if it's slower or more expensive.

sebzim4500 4y ago
Almost everyone is willing to trust 3rd parties with data, including enterprise and government customers. I find it hard to believe that there are enough people willing to pay a large premium to run these locally to make it worth the R&D cost.
- horsawlarway 4y ago
  Having done a lot of Bank/Gov related work... I can tell you this > Almost everyone is willing to trust 3rd parties with data, including enterprise and government customers. Is absolutely not true. In it's most basic sense - sure... some data is trusted to some 3rd parties. Usually it's not the data that would be most useful for these models to work with. We're already getting tons of "don't put our code into chatGPT/Copilot" warnings across tech companies - I can't imagine not getting fired if I throw private financial docs for my company in there, or ask it for summaries of our high level product strategy documents.
  - allturtles 4y ago
    Yes, just like you might get fired for transacting sensitive company business on a personal gmail account, even if that company uses enterprise gmail. Saying that cloud models will win over local models is not the same as saying it will be a free-for-all where workers can just use whatever cloud offering they want. It will take time to enterprisify cloud LLM offerings to satisfy business/government data security needs, but I'm sure it will happen.
mike_hearn 4y ago
People have trusted search engines with their most intimate questions for nearly 30 years and there has been what ... one? ... leak of query data during this time, and that was from AOL back when people didn't realize that you could sometimes de-anonymize anonymized datasets. It hasn't happened since. LLMs will require more than privacy to move locally. Latency, flexibility and cost seem more likely drivers.
- horsawlarway 4y ago
  You're still focused on trusting that my data is safe. And while I think that matters - I don't really think that's the trust I care most about. I care more about the trust I have to place in the response from the model. Hell - since you mentioned search... Just look at the backlash right now happening to google. They've sold out search (a while back, really) and people hate it. Ads used to be clearly delimited from search results, and the top results used to be organic instead of paid promos. At some point, that stopped being true. At least with google search I could still tell that it was showing me ads. You won't have any fucking clue that OpenAI has entered into a partnering agreement with "company [whatever]" and has retrained the model that users on plans x/y/z interact with to make it more likely to push them towards their new partner [whatever]'s products when prompted with certain relevant contexts.
  - mike_hearn 4y ago
    You wouldn't know that even if the model ran locally.
  - Karrot_Kream 4y ago
    > Hell - since you mentioned search... Just look at the backlash right now happening to google. They've sold out search (a while back, really) and people hate it. Ads used to be clearly delimited from search results, and the top results used to be organic instead of paid promos. At some point, that stopped being true. Only people in HN-like communities care about this stuff. Most people find the SEO spam in their results more annoying. > At least with google search I could still tell that it was showing me ads. You won't have any fucking clue that OpenAI has entered into a partnering agreement with "company [whatever]" and has retrained the model that users on plans x/y/z interact with to make it more likely to push them towards their new partner [whatever]'s products when prompted with certain relevant contexts. You won't know this for any local models either.

sebzim4500 4y ago

Almost everyone is willing to trust 3rd parties with data, including enterprise and government customers. I find it hard to believe that there are enough people willing to pay a large premium to run these locally to make it worth the R&D cost.

horsawlarway 4y ago
Having done a lot of Bank/Gov related work... I can tell you this > Almost everyone is willing to trust 3rd parties with data, including enterprise and government customers. Is absolutely not true. In it's most basic sense - sure... some data is trusted to some 3rd parties. Usually it's not the data that would be most useful for these models to work with. We're already getting tons of "don't put our code into chatGPT/Copilot" warnings across tech companies - I can't imagine not getting fired if I throw private financial docs for my company in there, or ask it for summaries of our high level product strategy documents.
- allturtles 4y ago
  Yes, just like you might get fired for transacting sensitive company business on a personal gmail account, even if that company uses enterprise gmail. Saying that cloud models will win over local models is not the same as saying it will be a free-for-all where workers can just use whatever cloud offering they want. It will take time to enterprisify cloud LLM offerings to satisfy business/government data security needs, but I'm sure it will happen.

mike_hearn 4y ago

People have trusted search engines with their most intimate questions for nearly 30 years and there has been what ... one? ... leak of query data during this time, and that was from AOL back when people didn't realize that you could sometimes de-anonymize anonymized datasets. It hasn't happened since. LLMs will require more than privacy to move locally. Latency, flexibility and cost seem more likely drivers.

horsawlarway 4y ago
You're still focused on trusting that my data is safe. And while I think that matters - I don't really think that's the trust I care most about. I care more about the trust I have to place in the response from the model. Hell - since you mentioned search... Just look at the backlash right now happening to google. They've sold out search (a while back, really) and people hate it. Ads used to be clearly delimited from search results, and the top results used to be organic instead of paid promos. At some point, that stopped being true. At least with google search I could still tell that it was showing me ads. You won't have any fucking clue that OpenAI has entered into a partnering agreement with "company [whatever]" and has retrained the model that users on plans x/y/z interact with to make it more likely to push them towards their new partner [whatever]'s products when prompted with certain relevant contexts.
- mike_hearn 4y ago
  You wouldn't know that even if the model ran locally.
- Karrot_Kream 4y ago
  > Hell - since you mentioned search... Just look at the backlash right now happening to google. They've sold out search (a while back, really) and people hate it. Ads used to be clearly delimited from search results, and the top results used to be organic instead of paid promos. At some point, that stopped being true. Only people in HN-like communities care about this stuff. Most people find the SEO spam in their results more annoying. > At least with google search I could still tell that it was showing me ads. You won't have any fucking clue that OpenAI has entered into a partnering agreement with "company [whatever]" and has retrained the model that users on plans x/y/z interact with to make it more likely to push them towards their new partner [whatever]'s products when prompted with certain relevant contexts. You won't know this for any local models either.