3 ms·

can we have tiny LLM that can run on smartphone now

by tonyhart7 2y ago

rubslopes 2y ago
Is a tiny large language model equivalent to a normal sized one?
- tonyhart7 2y ago
  Yes is called MLM (Medium Language Model)
winter_blue 2y ago
Apple Intelligence has an LLM that runs locally on the iPhone (15 Pro and up). But the quality of Apple Intelligence shows us what happens when you use a tiny ultra-low-wattage LLM. There’s a whole subreddit dedicated to its notable fails: https://www.reddit.com/r/AppleIntelligenceFail/top/?t=all https://www.reddit.com/r/AppleIntelligenceFail/top/?t=all One example of this is “Sorry I was very drunk and went home and crashed straight into bed” being summarized by Apple Intelligence as ”Drunk and crashed”.
- deleted 2y ago
  [deleted]
- Spooky23 2y ago
  I think the real problem with LLMs is we have deterministic expectations of non-deterministic tools. We’ve been trained to expect that the computer is correct. Personally, I think the summaries of alerts is incredibly useful. But my expectation of accuracy for a 20 word summary of multiple 20-30 word summaries is tempered by the reality that’s there’s gonna be issues given the lack of context. The point of the summary is to help me determine if I should read the alerts. LLMs break down when we try to make them independent agents instead of advanced power tools. Alot of people enjoy navel gazing and hand waving about ethics, “safety” and bias… then proceed to do things with obvious issues in those areas.
  - mewpmewp2 2y ago
    Larger LLMs can summarize all of this quite well though.
  - hansvm 2y ago
    Determinism isn't the issue though. Many responses are fine. The displayed one is bad, whether chosen deterministically or not. Some alternatives: - Passed out drunk - Crashed in bed - Slacking because drunk ... The issue isn't a lack of context; it's that even the available context was handled poorly.
samstave 2y ago
I want a tiny_phone_based LLM to do thought tracking and comms awareness.. I actually applied to YC in like ~2014 or such for thus; -JotPlot - I wanted a timeline for basically giving a histo timeline of comms btwn me and others - such that I had a sankey-ish diagram for when and whom and via method I spoke with folks and then each node eas the message, call, text, meta links... I think its still viable - but my thought process is too currently chaotic to pull it off. Basically looking at a timeline of your comms and thoughts and expand into links of thought - now with LLMs you could have a Throw Tag od some sort whereby you have the bot do work on research expanding on certain things and plugging up a site for that Idea on LOCAL HOST (i.e. your phone so that you can pull up data relevant to the convo - and its all in a timeline of thought/stream of conscious hopefully you can visualize it...
- radlad 2y ago
  I had a thought that I think some people value social media (e.g. Facebook) essentially for this. Like giving up your Facebook profile means giving up your history or family tree or even your memories. So in that sense, maybe people would prefer a private alternative.
  - samstave 2y ago
    I read this in Sam Wattersons voice with a pipe abt maybey an inch from his beard, (Fyi I was a designer at fb and while it was luxious I still hated what I saw in zucks eyes every morn when I passed him. Super diff from Andy Grove at intel where for whateveer reason we were in the sam oee schekdule (That was me typing with eues ckised as a test (to myself, typos abound
badlibrarian 2y ago
No. Smartphone only spin animated gif while talk to big building next to nuclear reactor. New radio inside make more efficient.
intrasight 2y ago
I expect that the phone will only do the prompt parsing

can we have tiny LLM that can run on smartphone now

rubslopes 2y ago

Is a tiny large language model equivalent to a normal sized one?

tonyhart7 2y ago
Yes is called MLM (Medium Language Model)

winter_blue 2y ago

Apple Intelligence has an LLM that runs locally on the iPhone (15 Pro and up). But the quality of Apple Intelligence shows us what happens when you use a tiny ultra-low-wattage LLM. There’s a whole subreddit dedicated to its notable fails: https://www.reddit.com/r/AppleIntelligenceFail/top/?t=all https://www.reddit.com/r/AppleIntelligenceFail/top/?t=all One example of this is “Sorry I was very drunk and went home and crashed straight into bed” being summarized by Apple Intelligence as ”Drunk and crashed”.

deleted 2y ago
[deleted]
Spooky23 2y ago
I think the real problem with LLMs is we have deterministic expectations of non-deterministic tools. We’ve been trained to expect that the computer is correct. Personally, I think the summaries of alerts is incredibly useful. But my expectation of accuracy for a 20 word summary of multiple 20-30 word summaries is tempered by the reality that’s there’s gonna be issues given the lack of context. The point of the summary is to help me determine if I should read the alerts. LLMs break down when we try to make them independent agents instead of advanced power tools. Alot of people enjoy navel gazing and hand waving about ethics, “safety” and bias… then proceed to do things with obvious issues in those areas.
- mewpmewp2 2y ago
  Larger LLMs can summarize all of this quite well though.
- hansvm 2y ago
  Determinism isn't the issue though. Many responses are fine. The displayed one is bad, whether chosen deterministically or not. Some alternatives: - Passed out drunk - Crashed in bed - Slacking because drunk ... The issue isn't a lack of context; it's that even the available context was handled poorly.

samstave 2y ago

I want a tiny_phone_based LLM to do thought tracking and comms awareness.. I actually applied to YC in like ~2014 or such for thus; -JotPlot - I wanted a timeline for basically giving a histo timeline of comms btwn me and others - such that I had a sankey-ish diagram for when and whom and via method I spoke with folks and then each node eas the message, call, text, meta links... I think its still viable - but my thought process is too currently chaotic to pull it off. Basically looking at a timeline of your comms and thoughts and expand into links of thought - now with LLMs you could have a Throw Tag od some sort whereby you have the bot do work on research expanding on certain things and plugging up a site for that Idea on LOCAL HOST (i.e. your phone so that you can pull up data relevant to the convo - and its all in a timeline of thought/stream of conscious hopefully you can visualize it...

radlad 2y ago
I had a thought that I think some people value social media (e.g. Facebook) essentially for this. Like giving up your Facebook profile means giving up your history or family tree or even your memories. So in that sense, maybe people would prefer a private alternative.
- samstave 2y ago
  I read this in Sam Wattersons voice with a pipe abt maybey an inch from his beard, (Fyi I was a designer at fb and while it was luxious I still hated what I saw in zucks eyes every morn when I passed him. Super diff from Andy Grove at intel where for whateveer reason we were in the sam oee schekdule (That was me typing with eues ckised as a test (to myself, typos abound

badlibrarian 2y ago

No. Smartphone only spin animated gif while talk to big building next to nuclear reactor. New radio inside make more efficient.

intrasight 2y ago

I expect that the phone will only do the prompt parsing