5 ms·

>However, if each app actively maintains a server connection, the OS cannot suspend them. This is a limitation with Android and not a fundamental network/OS li

>However, if each app actively maintains a server connection, the OS cannot suspend them. This is a limitation with Android and not a fundamental network/OS limitation. As long as your protocol allows for long enough keep alive latency you can periodically wake up for a short period and allow apps to service their connections (which is what I'd imagine this does anyway just with a singular app.) This works on Linux if you don't have the extra Android crap getting in the way. It's kind of a bummer to see bad OS architecture feed into over-complicated protocol/application design. EDIT: I guess I wasn't clear here. In my experiments I had the background wake up service wake the device up once and applications had just that singular window for all of them to service their connections. The problem with Android is instead applications either keep the phone from suspending entirely or (apparently) have some API allowing them to register their own periodic wake up that isn't shared. EDIT2: I also want to be clear that this isn't a criticism of NTFY. The developers behind it deserve congratulations for solving a serious problem with FOSS Android apps. My complaint is that the poor architecture and unescisary inflexibility of Android made this such a complex problem in the first place.

MayeulC 4y ago
You can do that, but if keep alive are not synchronized (I don't think there is a mechanism to do so), you will wake more often if you keep more connections open. I think the OS could synchronize them if keep alive packets are sent at the TCP level, but that would be hard to do at a higher level. Letting a single program handle persistent connections leaves a lot more potential for energy efficiency optimizations.
- londons_explore 4y ago
  Google famously tried allowing app keepalives to be synchronised... The scheduler allowed a wakeup in 'approximately' X minutes, and the OS would wake up and run all apps callbacks at once. If any app needed a wakeup at a precise time, then other callbacks at nearby approximate times would be run at the same time too. Overall, the thought was that wakeups are expensive, wakeups that involve the network are particularly expensive, so might as well run them all at once (they tend not to be CPU bound - often much of the wakeup handler is spent waiting for the network to connect and send a packet or an ack to arrive back). This turned out to be a very very bad idea. All it takes is some app to have a 'wake up exactly on the hour', and now all other app callbacks will also run exactly on the hour. End result: Mobile networks get 10 million phones all waking up within a few milliseconds of one another, and everyone wanting to connect, and the whole mobile network fails because it was never designed to have 10 million phones all trying to connect within a few milliseconds. That idea was scrapped.
  - jacob019 4y ago
    Seems like adding a random deviation to wakeup times would solve that problem.
    - _flux 4y ago
      Surely we'd be at back square one at that point?
      - usr1106 4y ago
        Why? On a single mobile you want to run all wake-ups together to save energy. That network and the server get them from many users spread out evenly is only a benefit.
- londons_explore 4y ago
  > if keep alive packets are sent at the TCP leve Unfortunately there are lots of bits of hardware in mobile networks that fake acks - ie. TCP level acks will be received very quickly to indicate the mobile network has accepted the data, and will deliver it at some future time out to the internet. The problem is they still send acks even if the remote end is no longer responsive. So you can't reliably use TCP acks to know if a connection is still alive. You need to send actual data and have the application respond.
  - _flux 4y ago
    Well that sounds utterly repulsive. And somewhat pointless. In fact, I choose not to believe it. Is there some document discussing this kind of functionality?
    - londons_explore 4y ago
      Products like this: https://www.enea.com/solutions/traffic-management/tcp-acceleration https://www.enea.com/solutions/traffic-management/tcp-accele...
kitsunesoba 4y ago
I would think that in order to not have an obvious negative impact on battery life, it would be necessary for the OS to coalesce app notification service keepalives, performing them in scheduled batches instead of the moment an app requests one. That should be roughly equal to the current situation with APNS/GCN, where the OS can schedule network maintenance during periods where the device and antenna(s) are likely to be awake already.
jitl 4y ago
> you can periodically wake up for a short period and allow apps to service their connections For this to work, you need apps to somehow conform to a standard timing and respond to a signal that says "hey, the OS is awake for you to do network things". Now each developer needs to implement some special code that understands the situation. After a while, all the devs start using libSleepyNetwork because getting those details right is hard. The libSleepyNetwork people realize that they can use a system daemon to further coalesce work for greater efficiency (one event loop is better than N event loops), so now all the consumer program C APIs are now backed by IPCs to the daemon. This is exactly what's already happened on Android, except libSleepyNetwork is implemented by either Google, or these people. I don't think it's bad OS design at all. It's chasing efficiency via abstraction that encapsulates some complex coordination.
binwiederhier 4y ago
> This is a limitation with Android and not a fundamental network/OS limitation. As long as your protocol allows for long enough keep alive latency you can periodically wake up for a short period and allow apps to service their connections (which is what I'd imagine this does anyway just with a singular app.) The protocol does not specify how the distributor and push server communicate or keep a connection going. It's up to them to define an energy efficient way to do that. For ntfy [1], a single WebSocket or JSON stream connection is used to deliver messages to the device. This is very energy efficient and consumes <1% of battery on most phones. As for regular wakeups (for polling), Android does not allow periodic work to be done more frequently than every 15 minutes, which is obviously not frequent enough to be useful. > It's kind of a bummer to see bad OS architecture feed into over-complicated protocol/application design. This is somewhat true. The restrictive Android eco-system is what led to the creation of UnifiedPush, but it's far from complicated IMHO Disclaimer: I am the maintainer of ntfy [1], one of the UnifiedPush distributors. [1] https://ntfy.sh https://ntfy.sh
- russianGuy83829 4y ago
  could the 15 minute limit be circumvented with multiple identical apps (different package names) installed?
  - binwiederhier 4y ago
    I mean yes, but if Google catches you, you're not going to have a good time. It's also incredibly inconvenient. Also, you cannot specify the exact time when it's supposed to run, so it's likely they will just run at the same time.
jrmg 4y ago
Even if this were all synchronized, you’d still need to have all the apps memory state available - either taking up loads of RAM or requiring a lot of expensive paging.
sandGorgon 4y ago
this is NOT a limitation of Android. Android has supported wakelock based notification managers for a long time. And FCM works pretty much the same as was mentioned in the OP. The FCM manager maintains one long polled connection to Google notification servers and every other app subscribes to wakelocks from that manager. Your app is only woken up by the OS if FCM receives a notification for you. For e.g. Xiaomi will not wake you up even if you get a notification, unless you're on a white list (the much glorified Xiaomi whitelist). When Android is put in power saving mode, the FCM manager optimized this even further.

MayeulC 4y ago

You can do that, but if keep alive are not synchronized (I don't think there is a mechanism to do so), you will wake more often if you keep more connections open. I think the OS could synchronize them if keep alive packets are sent at the TCP level, but that would be hard to do at a higher level. Letting a single program handle persistent connections leaves a lot more potential for energy efficiency optimizations.

londons_explore 4y ago
Google famously tried allowing app keepalives to be synchronised... The scheduler allowed a wakeup in 'approximately' X minutes, and the OS would wake up and run all apps callbacks at once. If any app needed a wakeup at a precise time, then other callbacks at nearby approximate times would be run at the same time too. Overall, the thought was that wakeups are expensive, wakeups that involve the network are particularly expensive, so might as well run them all at once (they tend not to be CPU bound - often much of the wakeup handler is spent waiting for the network to connect and send a packet or an ack to arrive back). This turned out to be a very very bad idea. All it takes is some app to have a 'wake up exactly on the hour', and now all other app callbacks will also run exactly on the hour. End result: Mobile networks get 10 million phones all waking up within a few milliseconds of one another, and everyone wanting to connect, and the whole mobile network fails because it was never designed to have 10 million phones all trying to connect within a few milliseconds. That idea was scrapped.
- jacob019 4y ago
  Seems like adding a random deviation to wakeup times would solve that problem.
  - _flux 4y ago
    Surely we'd be at back square one at that point?
    - usr1106 4y ago
      Why? On a single mobile you want to run all wake-ups together to save energy. That network and the server get them from many users spread out evenly is only a benefit.
londons_explore 4y ago
> if keep alive packets are sent at the TCP leve Unfortunately there are lots of bits of hardware in mobile networks that fake acks - ie. TCP level acks will be received very quickly to indicate the mobile network has accepted the data, and will deliver it at some future time out to the internet. The problem is they still send acks even if the remote end is no longer responsive. So you can't reliably use TCP acks to know if a connection is still alive. You need to send actual data and have the application respond.
- _flux 4y ago
  Well that sounds utterly repulsive. And somewhat pointless. In fact, I choose not to believe it. Is there some document discussing this kind of functionality?
  - londons_explore 4y ago
    Products like this: https://www.enea.com/solutions/traffic-management/tcp-acceleration https://www.enea.com/solutions/traffic-management/tcp-accele...

kitsunesoba 4y ago

I would think that in order to not have an obvious negative impact on battery life, it would be necessary for the OS to coalesce app notification service keepalives, performing them in scheduled batches instead of the moment an app requests one. That should be roughly equal to the current situation with APNS/GCN, where the OS can schedule network maintenance during periods where the device and antenna(s) are likely to be awake already.

jitl 4y ago

> you can periodically wake up for a short period and allow apps to service their connections For this to work, you need apps to somehow conform to a standard timing and respond to a signal that says "hey, the OS is awake for you to do network things". Now each developer needs to implement some special code that understands the situation. After a while, all the devs start using libSleepyNetwork because getting those details right is hard. The libSleepyNetwork people realize that they can use a system daemon to further coalesce work for greater efficiency (one event loop is better than N event loops), so now all the consumer program C APIs are now backed by IPCs to the daemon. This is exactly what's already happened on Android, except libSleepyNetwork is implemented by either Google, or these people. I don't think it's bad OS design at all. It's chasing efficiency via abstraction that encapsulates some complex coordination.

binwiederhier 4y ago

> This is a limitation with Android and not a fundamental network/OS limitation. As long as your protocol allows for long enough keep alive latency you can periodically wake up for a short period and allow apps to service their connections (which is what I'd imagine this does anyway just with a singular app.) The protocol does not specify how the distributor and push server communicate or keep a connection going. It's up to them to define an energy efficient way to do that. For ntfy [1], a single WebSocket or JSON stream connection is used to deliver messages to the device. This is very energy efficient and consumes <1% of battery on most phones. As for regular wakeups (for polling), Android does not allow periodic work to be done more frequently than every 15 minutes, which is obviously not frequent enough to be useful. > It's kind of a bummer to see bad OS architecture feed into over-complicated protocol/application design. This is somewhat true. The restrictive Android eco-system is what led to the creation of UnifiedPush, but it's far from complicated IMHO Disclaimer: I am the maintainer of ntfy [1], one of the UnifiedPush distributors. [1] https://ntfy.sh https://ntfy.sh

russianGuy83829 4y ago
could the 15 minute limit be circumvented with multiple identical apps (different package names) installed?
- binwiederhier 4y ago
  I mean yes, but if Google catches you, you're not going to have a good time. It's also incredibly inconvenient. Also, you cannot specify the exact time when it's supposed to run, so it's likely they will just run at the same time.

jrmg 4y ago

Even if this were all synchronized, you’d still need to have all the apps memory state available - either taking up loads of RAM or requiring a lot of expensive paging.

sandGorgon 4y ago

this is NOT a limitation of Android. Android has supported wakelock based notification managers for a long time. And FCM works pretty much the same as was mentioned in the OP. The FCM manager maintains one long polled connection to Google notification servers and every other app subscribes to wakelocks from that manager. Your app is only woken up by the OS if FCM receives a notification for you. For e.g. Xiaomi will not wake you up even if you get a notification, unless you're on a white list (the much glorified Xiaomi whitelist). When Android is put in power saving mode, the FCM manager optimized this even further.