9 ms·
How Tutanota replaced Google’s FCM with their own notification system
- Boulth 8y agoPeople migrating from GCM to FCM need to be aware that in FCM analytics are turned on by default: https://github.com/siacs/Conversations/issues/3041 https://github.com/siacs/Conversations/issues/3041
- closinginverse 8y agoFrom your link: >I’m closing this issue. I think current master successfully disables analytics. Thanks for bringing this to my attention.
- seba_dos1 8y ago"Current master" obviously refers to Conversations, not FCM libs.
- pndy 8y ago> Wouldn’t it be great if the user could just pick a “push notifications provider” in the phone settings and OS managed all these hard details by itself? Having the possibility of choice is a good thing but in this particular case I don't think it would be good if 3rd party notifications providers were available. This should be an integral part of the OS, not a field for competition; knowing that Google itself can harvest data from notifications we can't exclude the possibility that there would appear companies interested only in that activity, at the same time enticing users with pretty and simple design of notifications or whatever else.
- anticensor 8y agoYou would need a standard API like intent API.
- GranPC 8y agoWell, any Android app can harvest data from notifications right now, by setting up a notification listener. I don't see how this would be any different.
- izacus 8y agoOf course this "harvesting" needs to be explicitly approved by the user.
- lucb1e 8y agoThe difference is that it's server side. I'm not aware whether the third party server logs my messages, but I am aware when an app has local permissions (you can see it in the permissions overview) to read my notifications.
- Findus23 8y agoThis is really the chance for a great open source project. So many open source apps (Signal, Mastodon clients, riot.im, telegram-foss and more) have to do hacks to be able to deliver push notifications without GCM/FCM What if there was an open source software one could setup on a server that would provide push services for all these apps and would interact with one open source client running on android. That way one would have the energy saving benefits of only handling one server connection, but the privacy benefits of a private server for oneself/friends/people one trusts. This is definitely not easy and would require coordination between many open source projects and also additions in Android to run on a system level (maybe in lineageos and similar), but I really think it would be worth it.
- enitihas 8y agoAFAIK, signal doesn't work without GCM.
- Findus23 8y agoIt does, but again via an own implementation of web sockets (I think) which creates an additional connection that drains power.
- topkeks 8y agoCorrect. https://github.com/signalapp/Signal-Android/commit/1669731329bcc32c84e33035a67a2fc22444c24b https://github.com/signalapp/Signal-Android/commit/166973132...
- vetinari 8y agoOne of the keys to low-power push system is the ability to manage the radio. It is obviously much easier, if you have a single connection that you manage, know when it is possible to drop it, or when to send the heartbeat. Once multiple processes start doing it, without cooperating together, your battery is going to be shot. Apps also should never send anything private over push notification. It should be only a ping to the app, to check it's event source.
- 8y ago
- EGreg 8y agoFinal thought: Every user should be able to choose a “Notification Provider” for every app At Qbix, we thought deeply about this problem. Ideally, to maintain anonymity, each notification would come from some random endpoint on the cloud. But that’s not how the Internet works these days, you have to connect periodically to SOMETHING. So you can choose your own notification providers. The trouble is if you have many apps, then your device is constantly connected to many servers. What we settled on for now is a background process with WebSockets, but perhaps this works better. The operating systems weren’t designed for this use case and the phones aren’t optimized for it. For example, how would you do the same on iOS? However, what IS possible is tunneling through the native iOS VoIP notifications support and encrypting your notifications. You can even process them on the client side with some “IFTTT” type logic. To do that for Android, however, we had to implement this background process approach. But it’s a hack.
- cobookman 8y agoeach app having its own notification provider cones at the cost of extra cpu cycles, network calls, and interrupts. Worsening battery life. It's likely in the users benefit to only have 1 notification provider
- EGreg 8y agoRight. I think they should have one other mechanism “in the cloud” somewhere and no need to have too many, one for each domain. I am starting to realize that DNS is main source of problems on the Internet. We need to replace it with a DHT using something like Kademlia. It was originally designed to have human-readable domains on the Internet. But it’s become a glorified federated search engine, essentially, whereas we can have lots of different search engines, one per app or location. Today, there is very little benefit to having human-readable URLs. If it’s anything much longer than a hostname, you won’t even say it or read it. So it only works for a tiny subset of internet resources but serves to centralize control in the hands of a few large websites. It leads to centralized databases that attract the NSA and advertisers — and all the stuff from Cambridge Analytica to the Equifax breach are downstream results that can be fixed by apps switching from DNS to a DHT.
- hawski 8y agoI'm wondering how it works with different messaging apps. I'm using mostly Facebook Messenger, because of the network effects. A few months ago I tried to use several apps to communicate with my wife. Both Signal and Wire failed to reliably send messages in under 30 minutes or so. More often then not the message would be delivered after a few hours. That's totally unacceptable. If there are some magic settings in Android they were not properly advertised. Our phones are on quite pure 8.1 and 7 versions. So I'm back to Messenger. Reading this it seems that there is a known solution to this problem. Does Mastodon offer something like private messages? Or is there a messaging app that doesn't use FCM, but works in a way described in article?
- icebraining 8y agoIf your phone has Google Play Services running, Signal should use GCM/FCM; they have a websockets mode, but only for devices lacking the former. I don't use Signal, but I don't think that delay is typical, could there be an issue with the connectivity to their servers?
- metildaa 8y agoNope, this is not normal behavior for Signal, it is a sign that the sending or receiving phone has FCM and data connectivity issues. I have seen this issue on Motorola Android phones with known dying LTE Modems (eg: these phones couldn't maintain a data session for more than a few minutes, GPS off WiFi is also unreliable).
- egypturnash 8y agoMastodon has “private” messages but it is primarily about being Distributed Twitter. I put “private” in quotes because they’re not encrypted in transit between servers, and an unethical admin could easily intercept them.
- interfixus 8y agoDuring so far nearly two years of frequent Signal use, I have had one - one - not getting instantly delivered, and that was due to network breakdown at my end. There's plenty not to like about Signal, but reliability is second to none.
- yegle 8y agoIs it possible for every app developer to stop using FCM? Yes, in China. The result is a chaotic market. App devs will have to use multiple push machinism for different device: on Huawei phones, Huawei's own push service is the most reliable; on other phones, you might want use SDKs from Tencent/Baidu/Alibaba so their app (Wechat/Baidu search/Taobao/Alipay) can "wake up" your app to receive push. Battery life become miserable, push become unreliable. It's to a point where the government start "regulating" the market (the Unified Push Alliance, http://www.chinaupa.com http://www.chinaupa.com, is lead by CAICT, a think tank of Gov of China).
- ktosobcy 8y agoBut do we really need "push providers"? The problem is (IMHO) that app developers don't know how to use the connection efficiently causing the radio to wake up constantly (IDLE socket connection has very low battery impact). Instead of pushing every darn detail queue less important ones and send them out when something more important arrives; your client app can signal the server, that app is in the background (or phone is going to sleep) to send less frequent updates. But - companies will abuse everything they can to notify users about absolutely everything. Currently I'm running phone without google play services and it works just fine (with exception of Viber, which can't deliver messages correctly without push services, but it's not that important for me).
- richardwhiuk 8y agoAs soon as you are maintaining multiple connections, battery life gets worse.
- ucaetano 8y agoIf you don't have a push provider mediating the many apps trying to get notifications, you end up in a tragedy of the commons scenario, where every single app will try to implement its own push service and will prioritize its own push service, quickly resulting in a race to the bottom where the device is constantly connecting to a different push service, draining battery and exhausting your data connection. It isn't very different from preemptive/cooperative multitasking.
- snipem 8y agoAnother difficulty was caused by the Doze mode, introduced in Android M. The Doze, which is turned on after a period of inactivity, among other things prevents background processes to access the network. As you can imagine, this prevents our app from receiving notifications. We mitigate this problem by asking users to make an exemption from battery optimisations for our app. It worked fairly well. This is why we were migrating to FCM from our own custom built MQTT solution that was invented back in 2012.
- craftyguy 8y agoMQTT still requires a network connection to publish to (or receive subscription events from) a MQTT server.. so you still have the exact same problem with Doze preventing that. What are you doing to get around that problem, if not exempting your app from battery 'optimisations' ? Edit: Would anyone care to explain how I am wrong in addition to downvoting me? If Doze blocks network connections to your app when it is backgrounded, how can it receive MQTT events?
- TeMPOraL 8y agoGP knows that. They said they're migrating away from MQTT, to the Google solution.
- craftyguy 8y agoAh, somehow I missed that :(
- mcdevilkiller 8y agoThey are saying that they had to use FCM because MQTT was killed by Doze (i.e. the same you are saying).
- snipem 8y agoWith every release Android makes it harder to have a service running in the background. With the introduction of doze there is no alternative to using FCM. FCM has a privileged service where it is able to wake up the device from doze. A custom push service will never be able to do this. There might be hacks, but I'd rather take Googles approach than to work around on each and every release.
- southerndrift 8y agoWhat's the deal with push notifications if the modem has to be switched on every couple of minutes to keep a connection alive? I was under the impression that push notifications were implemented similar to a call handshake that only activates the phone if there is a call. Are the FCM service and the iOS equivalent also IP based or can they use some lower level, more energy efficient protocols to wake up the phones?
- repolfx 8y agoThey are IP based. There are several problems that make replicating FCM hard, which is one reason why Google tries to push you to using it (wordplay definitely intended). It all boils down to IPv4. IPv4 address space is exhausted, so carriers have had to deploy carrier NAT in front of cell towers, often multiple levels of carrier NAT. Well NAT requires translation tables to be held in RAM and carrier-grade boxes are very expensive in general, so to keep the machines alive they need to aggressively garbage collect dead translations. Otherwise they'd run out of RAM. This is the origin of the 'keep alive' problem - NATs want to close your connection to free up their own resources, and you want the connection to stay open so you can receive push messages. So phones have to wake up every so often and send keepalive packets or do a connection rebuild. Google and Apple have an interesting solution to this problem .... it's a mix of learning and data analytics to figure out what NAT timeouts used by each carrier are, and cutting deals with carriers directly to adjust the timeouts for their IP ranges specifically. Therefore you cannot compete directly with Google or Apple on energy efficiency. This is something a lot of hackers don't realise. Getting to the level of efficiency FCM has is very hard and takes a lot of work, and basically requires you to be a giant company. This is why it's an OS level service. They also use very tight protocols, batch things together and of course provide oodles of server side disk space for buffering messages to disk until the devices return, lots of other MQ type things that are hard to do at scale. IPv6 solves this problem by allowing carriers to lose the NAT boxes. No more machines that need stuff in RAM for every TCP connection, which means connections are no longer a scarce resource that must be culled from time to time. If you only have to maintain connections to devices that support IPv6 you could theoretically maintain very long lived connections if the kernel, radio and server cooperate. Of course there are still timeouts: the TCP connection requires some state server-side, so the servers will kill off connections from time to time because devices may roam across different IP addresses. But it should be a lot more stable and not require cutting deals anymore.
- emilfihlman 8y ago>SSE fits our needs better than WebSocket would (it is cheaper and converges faster, because it’s not duplex). We’ve seen multiple chat apps trying using WebSocket for push notifications and it didn’t seem power efficient. This is just not true unless there is some serious fuckery going on with the websocket implementation. Both require full duplex communications on the transport layer.
- azinman2 8y agoWow reading this seems like android push is a total mess (outside of FCM). Things are much simplier on the iOS side...
- kumarm 8y agoAs an User and Dev of both iOS and Android, I can tell you Push Notifications are miles ahead on Android compared to iOS. The article refers to not wanting use Google Solution for non technical reasons.
- xg15 8y ago> Wouldn’t it be great if the user could just pick a “push notifications provider” in the phone settings and OS managed all these hard details by itself? So every app, which doesn’t want to be policed by the platform owner, didn’t have to invent the system anew? It could be end-to-end encrypted between the app and the app server. There’s no real technical difficulty in that, but as long as our systems are controlled by big players who do not allow this, we have to solve it by ourselves. Fking this. ideally, this would also involve a standard API on the backend for how to send push notifications. E.g., something like: 1) App on phone queries OS for selected push provider. 2) Phone OS returns some metadata about the push provider to the app, including a a backend URL for sending messages. 3) App sends that URL to it's own backend server. 4) When a push message should be sent, the app's backend server invokes that URL with the message to be sent in some standardized way. If - which is likely - push servies require that developers register their apps with them before use, this could be expanded by the phone OS returning a list of providers instead of just one.
- metildaa 8y agoI would encourage app developers to take a look at the network architecture of the major US carriers while determining how to give their app notifications from the server. In the US, the 4 major LTE cellular carriers operate IPv6 only networks with NATed IPv4 access tunneled in as an unsupported afterthought. From my testing with IPv6 enabled VOIP for inbound calls, compared to FCM with calls coming from an IPv4 only server, only the former setup would reliably get calls to a test device on the first ring. If you need very low latency push notifications, IPv6 and FCM/Apple Push Notification are needed for best performance. Thus far, inter-carrier roaming generally does not support IPv6, so you still must support legacy IPv4 to a degree.
- StudentStuff 8y agoTo this end, Apple has been leaning on developers to support iOS devices that are on IPv6 only networks: https://developer.apple.com/support/ipv6/ https://developer.apple.com/support/ipv6/
- gustavmarwin 8y agoThis is a very exciting development, it's great to see more developers take this centralisation issue seriously. On this subject, many people including myself have been failing (hard) to convince Moxie that dropping FCM (or GCM or whatever the name) is the way to go for Signal, if anyone want to give a hand... https://www.reddit.com/r/signal/comments/9ekawn/moxie_drop_the_gcm_requirement_already/ https://www.reddit.com/r/signal/comments/9ekawn/moxie_drop_t... https://github.com/signalapp/Signal-Android/issues/7638 https://github.com/signalapp/Signal-Android/issues/7638