3 ms·

The idea isn't terrible, but I don't understand the market. People do 90% of their work "on the web" or "in the cloud", but they're going to be concerned about

The idea isn't terrible, but I don't understand the market. People do 90% of their work "on the web" or "in the cloud", but they're going to be concerned about running an LLM locally (and have no ability to audit security)?

daveguy 3y ago
Are you implying that running an LLM locally means having no ability to audit security vs running one "in the cloud"? Seems like the opposite to me.
- belltaco 3y ago
  I think they meant people trust the web enough to use it for most of their activities so the ability to audit security in a local LLM may not be a big selling point.
- UncleMeat 3y ago
  No the point is that I'm already doing basically all of my computing (both work and non-work) via web applications. All of the AI integrations into products can happily run on backends rather than locally on my machine. There are exceptions, of course, but this is the state of the bulk of modern computing.
- itsoktocry 3y ago
  >having no ability to audit security vs running one "in the cloud"? Seems like the opposite to me. By "ability" I mean, literally, ability. If the customer is someone who does most of their computing in the cloud today, what value is a "personal secure LLM"? Can this person even tell if their local LLM is secure? Doubt it. For the average person, that is best left to "the cloud".
wongarsu 3y ago
Individuals aren't concerned about that, but plenty of companies and government agencies are. Most wouldn't even want to audit the LLM, they just want to audit that the information doesn't leave their network. Running the LLM locally on some "AI PC" would satisfy that perfectly, and the types of organizations that care the most about this are also the ones with the most generous budgets for these kinds of problems.
- itsoktocry 3y ago
  >Running the LLM locally on some "AI PC" would satisfy that perfectly, and the types of organizations that care the most about this are also the ones with the most generous budgets for these kinds of problems. There are far better solutions, today, for intranet LLMs, then having an AI PC on each worker's desk.
indymike 3y ago
> People do 90% of their work "on the web" or "in the cloud", I'm much more excited about embedded applications where cloud connected might not be the answer for running an LLM.

daveguy 3y ago

Are you implying that running an LLM locally means having no ability to audit security vs running one "in the cloud"? Seems like the opposite to me.

belltaco 3y ago
I think they meant people trust the web enough to use it for most of their activities so the ability to audit security in a local LLM may not be a big selling point.
UncleMeat 3y ago
No the point is that I'm already doing basically all of my computing (both work and non-work) via web applications. All of the AI integrations into products can happily run on backends rather than locally on my machine. There are exceptions, of course, but this is the state of the bulk of modern computing.
itsoktocry 3y ago
>having no ability to audit security vs running one "in the cloud"? Seems like the opposite to me. By "ability" I mean, literally, ability. If the customer is someone who does most of their computing in the cloud today, what value is a "personal secure LLM"? Can this person even tell if their local LLM is secure? Doubt it. For the average person, that is best left to "the cloud".

wongarsu 3y ago

Individuals aren't concerned about that, but plenty of companies and government agencies are. Most wouldn't even want to audit the LLM, they just want to audit that the information doesn't leave their network. Running the LLM locally on some "AI PC" would satisfy that perfectly, and the types of organizations that care the most about this are also the ones with the most generous budgets for these kinds of problems.

itsoktocry 3y ago
>Running the LLM locally on some "AI PC" would satisfy that perfectly, and the types of organizations that care the most about this are also the ones with the most generous budgets for these kinds of problems. There are far better solutions, today, for intranet LLMs, then having an AI PC on each worker's desk.

indymike 3y ago

> People do 90% of their work "on the web" or "in the cloud", I'm much more excited about embedded applications where cloud connected might not be the answer for running an LLM.