9 ms·
OpenPush: A Free, Decentralized Push Messaging Framework for Android
- gdeglin 7y agoThis is an interesting idea. Sadly, applications that use this are probably not eligible to be distributed through Google Play. One of the main reasons, as I understand, is that Google wants to discourage lots of apps from each having their own network connections open in the background. https://news.ycombinator.com/item?id=20145206 https://news.ycombinator.com/item?id=20145206
- JoshTriplett 7y agoThere's really no good reason that a network connection open in the background needs to use extra battery, if configured carefully. If configured incorrectly, it absolutely would, but a background connection doesn't need to regularly wake the phone up, just as Google's own push messaging service doesn't.
- _-___________-_ 7y agoMore connections probably does mean more wakeups, in general, though. FCM will coalesce notifications that happen close together in time, for example. Even if OpenPush does that (I haven't read all the details yet), it still means two coalesced batches of notifications rather than one if you have some apps using FCM and some using OpenPush. The situation obviously gets even worse if every app implements push itself. I still think it's worth it, to be clear, and for people who want a fully de-Googled device, hopefully eventually all their apps will use OpenPush and the battery life situation is the same as a device where all the apps use FCM. (Assuming OpenPush implements similar coalescing.)
- izacus 7y agoYou're right, there isn't. But what years of unrestricted background execution have shown us is that developers these days will choose the lazy solution over the right solution about 9 times out of 10. I'd be the richest guy if I'd get a penny every time I've seen an Android app do background polling every 5 minutes (!) instead of using push notifications or even long polling. The usual response to recommendation to improve the design was "Why? It works and it's a lot of work to run a push server!". So as much as I dislike restrictions on my operating systems, Google forcing developers to push notifications has massively improved battery life across the ecosystem.
- boudin 7y agoI think the target will mainly be developers who already provide a version of their app on f-droid. I don't know of it's an obligation to use google's firebase pish notifications though. I might be mistaken, but I think that Conversations use its own push notifications already: https://play.google.com/store/apps/details?id=eu.siacs.conversations&hl=en_US https://play.google.com/store/apps/details?id=eu.siacs.conve...
- _-___________-_ 7y agoThis will probably be used for apps that distribute through, e.g., F-Droid. Apps that also distribute on Google Play could use FCM for the version distributed through Google Play, and OpenPush for the version distributed elsewhere.
- rohan1024 7y agoThen give the user choice between the service he wants to use. How we can't see through this monopoly is beyond me.
- tuukkah 7y agoCounter-example: Telegram is available in Google Play and implements its own push service (kept alive by a permanent background notification).
- izacus 7y agoThe Play Store version users FCM and there's no notification.
- est31 7y agoThere is still a centralized push notification app on the phone. So it's not every app having its own network connection to the push service. The decentralization refers to the fact that the notification app likely will get the ability to configure different push services. That configuration is then sent to the apps that use it to upload it to their upstream backends. Those upstream backends now have to send push notifications according to that configuration (token + hostname).
- osamagirl69 7y agoI am really excited about this project! Having a source of push notifications without routing them through the google servers will really advance the state of open source and/or google free android. It is great to see it being backed by f-droid and apparently funded by the German government (through prototype fund and Federal Ministry of Education)
- TrickyRick 7y agoConsidering the BND's (German equivalent of NSA) love of intercepting traffic and gathering metadata about citizens, I wouldn't put "funded by the German government" in the Pro column.
- tuukkah 7y agoOne, end-to-end encrypted push service can make any intercepted data useless, as you cannot even know if the traffic corresponds to a message or is just a ping, let alone which app or contact the message is from.
- _-___________-_ 7y agoI actually doubt this is the case. Pings probably happen on some relatively-predictable interval, while a back-and-forth conversation between two parties is going to result in push notifications with much shorter intervals than any reasonable ping implementation would use. The metadata might be limited in value, but it still has some value, and I think it could easily be distinguished from pings. Do you have any references for your assertion?
- sliken 7y agoSure a determined attacked with control over hardware in the network path can analyze the timing and packet sizes. However a smart application/protocol writer could make make a ping and a message delivery take the same number of bytes. Which of course means some wasted payload on pings. Additionally if trying to track peers you could every N seconds send a ping OR a message. After all if you succeed in sending a message you know the peer is reachable. So it's possible, but no idea how careful OpenPush is though.
- me551ah 7y agoRunning background services in Android is tricky. Most device manufacturers put their own battery management software on devices, which makes it almost impossible to run long-running services in the background. And since this framework would either perpetually run in the background or run frequently to check for new messages, your app is guaranteed to be throttled(force-killed) by most devices. And being force killed means that your app cannot receive push notifications even from FCM. I'm curious to know if anyone is using this framework in the wild.
- Yizahi 7y agoMy previous phone was Oneplus with such an extreme battery manager (rated by some resource in top 3 of most aggressive managers). There were messaging apps which worked just fine in background and send push notifications even after prolonged periods in background (e.g. Telegram and others) and some apps which were killed in minutes after being minimized (e.g. official Skype app). Point is - I suspect that there is a correct way to make inoffensive apps which will work in background, but certain minority of developers simply don't give a damn about battery usage and proper coding guidelines.
- me551ah 7y agoOneplus incidentally maintains a whitelist of apps which are allowed to run in the background and only apps in that list can run for extended periods. While it is possible for users to add apps to that list, it comes with a set of apps already whitelisted ( like whatsapp). I found it out the hard way while working on Android. You can read more about it here in case you are curious: https://hackernoon.com/notifications-in-android-are-horribly-broken-b8dbec63f48a https://hackernoon.com/notifications-in-android-are-horribly...
- EwanToo 7y agoThe reason why some apps work is they've been whitelisted by the manufacturer, not a choice by the developers (except maybe a choice not to pay to be whitelisted?) If you're not whitelisted, there's very little you can do except for ask users to take various manual steps to change operating systems.
- mrtksn 7y agoWow, I didn't know that the Apps on Android do implement their own push services in the background. For all the edge cases out there, OpenPush is great news. However, this is a huge no-no for me as it means that I would need to actively manage the device to make sure that things do work as intended. Not happy with Google "owning" the push notifications? Then maybe the solution should be a non-profit independent corporation running it. Having a service-per-app app cannot be as efficient as having a single service serving them all. Both in terms of device resources and user time & experience. Surely having an alternative is a good thing but the issue seems to be institutional(You don't want to have Google in charge), not technical(Google's service works great), therefore the solution out to be institutional too. Or something that acts like an institution, something like bl_______n.
- est31 7y agoThis is a great and important project for F-Droid. The problem is harder than it seems on the surface as idle connections can be terminated by the network operator. There's a paper about this: https://ieeexplore.ieee.org/document/8214213 https://ieeexplore.ieee.org/document/8214213 I guess ultimately one should build a database about phone networks and store measurements of their maximum heartbeat intervals in there so that new connections can use those heartbeat intervals to save battery. Also ideally the app should send its data via WiFi if available. So it has to detect when a WiFi connection is established and open a connection wia WiFi. Ideally the mobile connection is kept alive though for the case the WiFi proves unreliable. E.g. when I leave my home on foot I get further and further away from the WiFi and there's a short time span where my phone keeps trying to use the WiFi connection even though nothing gets through any more. Detecting all these edge cases can be tricky. Fortunately the design makes the protocol between the push server and the on-phone app open to iteration and innovation. It's a good design!
- awelkie 7y agoExcellent point. So how does Google solve the problem? Do they just coordinate the heartbeat intervals with the carriers, or do they bypass the Internet altogether and use some cell-network-specific push mechanism? If the latter, is this mechanism available to everyone or just partners of the carriers?
- est31 7y ago> Do they just coordinate the heartbeat intervals with the carriers, or do they bypass the Internet altogether and use some cell-network-specific push mechanism? IIRC they use the xmpp protocol, so standard TCP over IP. In threads about this topic you often hear claims that Google has special deals with carriers. Whether this is true or not, no idea. I haven't seen any confirmation of this. It would be interesting to conduct measurements whether FCM sends less frequent heartbeat messages than the timeout is for "normal" long lived tcp connections. In any case, deals or not, I guess Google has built a database as I've mentioned. But I'm only guessing here.
- netsharc 7y ago
- TheAceOfHearts 7y agoI have a tangentially related tip or suggestion of which many appear to be unaware: most (all?) cellphone service providers (at least in the US) let you send and receive emails over text message. For example, with Project Fi you'd use $NUMBER@msg.fi.google.com [0]. You can use this as a super simple alert or notification system for scripting and small self-hosted apps without having to setup any additional services or tools. However, it's important to keep in mind that both email and text messaging are unencrypted by default, so you should be careful with what you send. Let's consider a situation in which this might be useful: You have a 15 to 30 minute task running, such as downloading a large file (e.g. Linux ISOs) from a remote server or compiling a small C++ project. Instead of sitting around waiting you could go outside for a short walk in order to get a bit of exercise and sunlight, which helps you get to the recommended minimum of 10k daily steps. Once the task is finished you receive a text message which prompts you to head back inside. If you just want to send yourself notifications then you can actually get pretty far only using email and text messaging. However, I'm aware that OpenPush solves a few different problems, and I love that we're finally getting an open-source self-hosted alternative to GCM. [0] https://support.google.com/fi/answer/6356597?hl=en&ref_topic=6216184 https://support.google.com/fi/answer/6356597?hl=en&ref_topic...
- est31 7y agoThe protocol between the push server and the push app on the device is entirely up to those two. You can use e-mail over SMS if you want.
- commoner 7y agoOpenPush could be very valuable for open source Android apps once it is fully developed. Currently, open source Android apps that do not depend on Google Play Services are using WebSocket or Server-Sent Events (SSE) for push notifications. For example, Signal falls back to WebSocket when Firebase Cloud Messaging (FCM) is not available. Tutanota published a blog post explaining how they deliver push notifications to their Android app with SSE instead of FCM: https://f-droid.org/en/2018/09/03/replacing-gcm-in-tutanota.html https://f-droid.org/en/2018/09/03/replacing-gcm-in-tutanota.... OpenPush would reduce the amount of work needed to implement push notifications compared to the made-from-scratch implementations developed by Signal and Tutanota.
- shujito 7y agoCan websockets be used as a self-hosted push notification alternative?
- hjkgfdfgh 7y agoI wish something like this existed that could additionally proxy FCM notifications. Allowing push notifications without any Google involvement is the ideal long term goal, but one transitional path I dream about would be forcing Google to proxy all their notifications through a host of your choice (ideally that you control). I think I have about 30 false start projects with names like PushMux and rough outlines on potential ways to accomplish it. Tangentially, I also dream of having true one-way push notifications. Send a minimal notification to the target device via FLEX/POCSAG or even one-way Iridium, and allow the device to connect to the internet as needed to fetch the rest of the notification.
- Andrew_nenakhov 7y agoI don't think it'll fly, here is why: Our product is a XMPP client, which can't work if it is not actively running in the background, so we were working with making apps run in background on Android for over 10 years, since version 2.0. The trend is clear: with every update, we get more restrictions. Not only bg services are killed off from memory by aggressive memory managers of crappy phone vendors, Google themselves push developers to rely on proprietary FCM services. The writing is on the wall, and one day background services in Android will simply cease to work, all in the name of better battery life and customer care. So this OpenPush service will be short lived, it'll likely be disallowed to work in this form in Android 12 or 13. The only proper long-term solution here is an anti-monopoly legislation, that would make Google allow Android users to subscribe to a third-party push notification service. Such service is, actually, a relatively simple: current FCM is just a modified XMPP connection that a phone keeps, and it will be easy to replicate and deploy. Apple is guilty of this, too. A proper anti-monopoly legislations should target them as well, and not only make them allow third-party app stores, but third party push notifications as well. This control of push notification services and restriction creep that occurs on both platforms makes Google's and Apple's grip on their respective OSs much tighter than Microsoft's grip on Windows ever was. This tyranny must be stopped, and I don't believe it can be done without legal action, by technical means only.
- est31 7y ago> one day background services in Android will simply cease to work, all in the name of better battery life and customer care. In that instance, having an app which centralizes the push service needs of other apps would be highly useful, as it could then be installed as a system app with the same kind of access as the FCM service. For many phones, custom ROMs are available. Average people probably won't have the skills to install custom ROMs. But the goal of OpenPush is not to replace FCM from all phones everywhere. It's to meet the needs of the subcommunity of people who want to use degooglified Android, maybe also later on integrate with non-Android mobile OSs. In that subcommunity, people likely already use custom ROMs, and among those who don't, the readiness is higher to install one.
- 7y ago