4 ms·

What LLM are you running on your Mi 11 ultra?

theGeatZhopa 2y ago
It's not about running an LLM on a device, it's about the hardware - which is quite good and long lasting. In both cases, the hardware is good. But apple has been caught to slow down their phones so the incentive to buy a new one is higher (like the scandal slow down because of the battery usage of new features.. etc..) The OG just stated to keep his Mi 11 ultra until it degrades, because he/she/it is happy with it and there won't be such games like making it slow or bring you somehow into situation that you need to buy a new iPhone, like now apple does - buy a new if you want to have LLM.. Also, what LLMs are running on your apple device that is not within the 100-90%=10% of all iPhones and other iDevices? OG can use android apps for that, if he needs LLM or other features of AI. It's even possible to side load them. Which is not possible and not sure for Apple. You can ask a question: If a feature is built in into iOS, is it possible to write an app for iOS and having this accepted for AppStore? Look up the terms for developers. It's written: no. So having apple introduce their AI solution, which is built in and sitting on top of openAI... No other app will be accepted that's doing the same . Nor will the apps have deep integration into the OS, because Apple doesn't want it. They also won't change their minds on it. So this is the point of OP. It's not to be answered with in a sense "so? Then tell me what LLMs are running on your Mi11... Because there aren't and Apple now introduced local LLMs.." are you an Apple fanboy/fan girl?
- supermatt 2y ago
  > are you an Apple fanboy/fan girl? Nope. Would that invalidate the facts? Are you an Applephobe? They stated that apple phones are essentially the same as they were years ago, when it is very clear that the hardware is considerably more powerful. Their devices are now capable of on-device inference of a 3B parameter language model. The commenters device is not capable of such a thing, as it is old hardware, which is counter to his position that there is little difference between the new and old phones.
  - theGeatZhopa 2y ago
    Nope, I'm also not a applephobe. I like their devices and software, but I don't like their politics. I think we're aligned here in our thinking may the best product prosper! Your question wasn't that clear. Also, I think Mi11 will be capable of doing such things, because Google is also eager to have their Models running locally. I remember a news from January of this year, that Google brings their Gemma 2B locally on devices. Other LLMs are also eager to achieve that. It's not like you stated, Mi11 is an old hardware and nothing will run on it. The next thing to consider is, is really 3B model is running on (only new) devices or is it just a mockup because openAI is in behind doing some magic? What about your comparison, that Apple is capable to do 3B and Mi11 not because it's old hardware.. we just learned, that over 90 percent of all Apple devices can't do 3B.. isn't it the same like to say over 90 percent of all androids can't run 3B? ... Yes.. because their not built to do so. In Future you can assume, all of the new devices of certain price will be able to.. in fact, I didn't see the statistics, but more/less than 10 percent are capable right now and will get the software update soon for ... So, how to counter that arguments? :) Edit: Not from January, but from a month ago: https://medium.com/@samvardhan777/running-lightweight-llms-on-android-devices-b2ee4082557e https://medium.com/@samvardhan777/running-lightweight-llms-o...
    - supermatt 2y ago
      How to counter what argument? That old hardware can’t run some recent stuff? That’s exactly what I said…
      - theGeatZhopa 2y ago
        When it's about RAM, Mi11 have 8gb of RAM.. the Mi11 pro has 12gb RAM. And both have sd slot for upto 256 GB.. so plenty of space.. Snapdragon 888 is also not the baddest one. Let's wait what community will bring us. But with 90 percent of apple old hardware also can't run it, so there also won't be a community to hack on the old hardware.. so what?
KingOfCoders 2y ago
Currently none b/c I think every NPU you can currently buy will be rapidly old tech. It's like iPhone 1 (had one), iPhone 2 (3G, had one) [...], iPod 1 (hard drive, had one), iPod 2 (had one) [...], or MacBook Pro 1 (white, had one), MacBook Pro 2 (aluminium , had one), [...] all rapidly changing over the first generations. After some iterations of something truly new, as (local) LLMs, things will slow down.

It's not about running an LLM on a device, it's about the hardware - which is quite good and long lasting. In both cases, the hardware is good. But apple has been caught to slow down their phones so the incentive to buy a new one is higher (like the scandal slow down because of the battery usage of new features.. etc..) The OG just stated to keep his Mi 11 ultra until it degrades, because he/she/it is happy with it and there won't be such games like making it slow or bring you somehow into situation that you need to buy a new iPhone, like now apple does - buy a new if you want to have LLM.. Also, what LLMs are running on your apple device that is not within the 100-90%=10% of all iPhones and other iDevices? OG can use android apps for that, if he needs LLM or other features of AI. It's even possible to side load them. Which is not possible and not sure for Apple. You can ask a question: If a feature is built in into iOS, is it possible to write an app for iOS and having this accepted for AppStore? Look up the terms for developers. It's written: no. So having apple introduce their AI solution, which is built in and sitting on top of openAI... No other app will be accepted that's doing the same . Nor will the apps have deep integration into the OS, because Apple doesn't want it. They also won't change their minds on it. So this is the point of OP. It's not to be answered with in a sense "so? Then tell me what LLMs are running on your Mi11... Because there aren't and Apple now introduced local LLMs.." are you an Apple fanboy/fan girl?

supermatt 2y ago
> are you an Apple fanboy/fan girl? Nope. Would that invalidate the facts? Are you an Applephobe? They stated that apple phones are essentially the same as they were years ago, when it is very clear that the hardware is considerably more powerful. Their devices are now capable of on-device inference of a 3B parameter language model. The commenters device is not capable of such a thing, as it is old hardware, which is counter to his position that there is little difference between the new and old phones.
- theGeatZhopa 2y ago
  Nope, I'm also not a applephobe. I like their devices and software, but I don't like their politics. I think we're aligned here in our thinking may the best product prosper! Your question wasn't that clear. Also, I think Mi11 will be capable of doing such things, because Google is also eager to have their Models running locally. I remember a news from January of this year, that Google brings their Gemma 2B locally on devices. Other LLMs are also eager to achieve that. It's not like you stated, Mi11 is an old hardware and nothing will run on it. The next thing to consider is, is really 3B model is running on (only new) devices or is it just a mockup because openAI is in behind doing some magic? What about your comparison, that Apple is capable to do 3B and Mi11 not because it's old hardware.. we just learned, that over 90 percent of all Apple devices can't do 3B.. isn't it the same like to say over 90 percent of all androids can't run 3B? ... Yes.. because their not built to do so. In Future you can assume, all of the new devices of certain price will be able to.. in fact, I didn't see the statistics, but more/less than 10 percent are capable right now and will get the software update soon for ... So, how to counter that arguments? :) Edit: Not from January, but from a month ago: https://medium.com/@samvardhan777/running-lightweight-llms-on-android-devices-b2ee4082557e https://medium.com/@samvardhan777/running-lightweight-llms-o...
  - supermatt 2y ago
    How to counter what argument? That old hardware can’t run some recent stuff? That’s exactly what I said…
    - theGeatZhopa 2y ago
      When it's about RAM, Mi11 have 8gb of RAM.. the Mi11 pro has 12gb RAM. And both have sd slot for upto 256 GB.. so plenty of space.. Snapdragon 888 is also not the baddest one. Let's wait what community will bring us. But with 90 percent of apple old hardware also can't run it, so there also won't be a community to hack on the old hardware.. so what?

KingOfCoders 2y ago

Currently none b/c I think every NPU you can currently buy will be rapidly old tech. It's like iPhone 1 (had one), iPhone 2 (3G, had one) [...], iPod 1 (hard drive, had one), iPod 2 (had one) [...], or MacBook Pro 1 (white, had one), MacBook Pro 2 (aluminium , had one), [...] all rapidly changing over the first generations. After some iterations of something truly new, as (local) LLMs, things will slow down.