3 ms·

>Another issue is deployment. Imagine this: You are a software developer using an LLM in your commercial product. You have two/three choices. >Call a cloud API

>Another issue is deployment. Imagine this: You are a software developer using an LLM in your commercial product. You have two/three choices. >Call a cloud API using a simple cross-platform HTTP call. This will reliably work. While at it, you may gain useful data on your customers In the not too distant future, I expect most applications to be replaced by LLM assistants. So the above scenario won't really happen. You will interact with one LLM, maybe it's a local flavor or maybe it's a cloud flavor. The LLM will then call other services for you. So I don't expect services to call LLMs but I do expect LLMs to call other services. Yes, we are getting a little ahead of ourselves.

yyyk 3y ago
>>Call a cloud API using a simple cross-platform HTTP call. This will reliably work. While at it, you may gain useful data on your customers. >In the not too distant future, I expect most applications to be replaced by LLM assistants. So the above scenario won't really happen. Regardless we have the question of how to deploy the local LLM. If the frontend will be the OS, than we can be reliably sure that Microsoft and Google will have motivation to use their cloud hosted AI and you may have to fight the OS to get it to install your model. We may need regulation to ensure local LLMs are possible.
- usrbinbash 3y ago
  > Regardless we have the question of how to deploy the local LLM. The same as we do any other software? There is nothing inherently more difficult in deploying an ML model than there is, say, in setting up a redudant high traffic RDBMS. > If the frontend will be the OS Why would the frontend be the OS? The OS isn't the frontend to my database. The OS isn't my IDE. The OS isn't the games I play. The OS isn't even my browser. All these are Userspace programs.
  - yyyk 3y ago
    >Why would the frontend be the OS? I answered this above. https://news.ycombinator.com/item?id=36194304 https://news.ycombinator.com/item?id=36194304

yyyk 3y ago

>>Call a cloud API using a simple cross-platform HTTP call. This will reliably work. While at it, you may gain useful data on your customers. >In the not too distant future, I expect most applications to be replaced by LLM assistants. So the above scenario won't really happen. Regardless we have the question of how to deploy the local LLM. If the frontend will be the OS, than we can be reliably sure that Microsoft and Google will have motivation to use their cloud hosted AI and you may have to fight the OS to get it to install your model. We may need regulation to ensure local LLMs are possible.

usrbinbash 3y ago
> Regardless we have the question of how to deploy the local LLM. The same as we do any other software? There is nothing inherently more difficult in deploying an ML model than there is, say, in setting up a redudant high traffic RDBMS. > If the frontend will be the OS Why would the frontend be the OS? The OS isn't the frontend to my database. The OS isn't my IDE. The OS isn't the games I play. The OS isn't even my browser. All these are Userspace programs.
- yyyk 3y ago
  >Why would the frontend be the OS? I answered this above. https://news.ycombinator.com/item?id=36194304 https://news.ycombinator.com/item?id=36194304