2 ms·
I don't think that is too unrealistic. • Small models have performed really well in recent years. • Newer phones will have more RAM. • Private LLMs do not nece
by maltelandwehr 1y ago
I don't think that is too unrealistic.
• Small models have performed really well in recent years.
• Newer phones will have more RAM.
• Private LLMs do not necessarily need to run locally.
- const_cast 1y agoIf it's not running locally, then by nature of what an LLM is it cannot be private. LLMs cannot act on encrypted information - the server must have access to all your plaintext prompts. And you simply are forced to trust they don't keep them forever, but they probably do.