5 ms·
Why WhatsApp Was Able to Support 50B Messages a Day with 32 Engineers
- espio 3y ago[flagged]
- senttoschool 3y agoWhatsapp is a very simple service. Telegram has/had even fewer developers. Simple message apps do not require huge number of developers to support many messages.
- throwaway751 3y ago[dead]
- mg 3y agoMy first question would be if the messages can be sent P2P or if they have to be passed through / stored on servers. If they can be sent and stored on the users devices, the server would only be a discovery service processing the occasional "Hey, I have a new IP" and "Hey, I want to send a message to X, please give me their current IP" messages. If the messages would have to be processed by servers, the system could be set up similar to email servers, where each server can handle - say - 10 million users and be completely independent of the other servers. So scaling would just mean adding a new "email" server for every new 10 million users. If 50B messages amount to - say - 500 million users, that would mean operating 50 servers.
- bakugo 3y ago> If they can be sent and stored on the users devices, the server would only be a discovery service processing the occasional "Hey, I have a new IP" and "Hey, I want to send a message to X, please give me their current IP" messages. You can't just send messages directly to end user devices' IPs unless there's no firewall or NAT in front of them. Good luck forwarding a port on a mobile device.
- tamimio 3y ago> You can't just send messages directly to end user devices' IPs unless there's no firewall or NAT in front of them. Yes you can, through hole punching, I used that technique in several platforms I built to establish a low latency p2p sockets for telem/msgs/and even C2 links for some drones/ugv. https://en.m.wikipedia.org/wiki/UDP_hole_punching https://en.m.wikipedia.org/wiki/UDP_hole_punching
- lclc 3y ago> They eliminated feature creep at all costs. Not anymore. I'm wondering how many engineers they have now to develop and maintain all the non-chat features.
- crop_rotation 3y agoThey are still so much better than almost anyone else in the space at preventing feature creep. It is still a remarkably small and focused app.
- throwitaway156 3y agoI think this is because of FB/Meta. I would assume that they dont have massive financial restrictions from investors due to their data harvesting capability. Other apps like Telegram, however, might work harder to introduce money-making features.
- rpgbr 3y agoIs it? WhatsApp calls home a lot in background, so much it was draining my phone’s battery. Also, business features are coming for a while, bloating the app. Heck, on iOS its installer has 163 MB! Crypto shit aside, I think Signal is way more focused. Element/Matrix, too.
- crop_rotation 3y agoI know very well of 20 people atleast who use Whatsapp a lot and nobody has had an issue with battery due to Whatsapp. You can try Netguard and disable Whatsapp networking to validate your battery hypothesis, but I doubt it is true. Business features are not in the app for normal users (Whatsapp Business is a different app), and I don't even know of a single business feature. Business chats to me are like normal chats. 163 MB is below normal these days on iOS AFAIK. I think both Signal and Element have similar sizes.
- londons_explore 3y agoIt was just a system designed from the start with efficiency in mind, rather than business flexibility. Almost any system can do the same if you start from the basics of "how many bytes of data do I need to move?", and "how many bytes of data can this ethernet cable/CPU transfer each second?" The end result is a system which is typically lower level and less flexible, but has much better money and hardware efficiency. You'll end up delving into the Linux kernel and sockets API's, rather than using NodeJS and MongoDB. It'll take a totally different type of engineer to do so too, so make sure you hire appropriately!
- pharmakom 3y agoNode js is a surprisingly lightweight API layer for manipulating Linux IO
- mschuster91 3y agoThe problem is that the NodeJS ecosystem infrastructure doesn't exactly like efficient programming - you have to go pretty much bare metal to actually be lightweight.
- porsager 3y agoOr maybe NodeJS is just leveraged a lot for cases where this kind of performance isn't required. Doesn't mean it's not useful in scenarios where you do require it [1] [1] https://unetworkingab.medium.com/millions-of-active-websockets-with-node-js-7dc575746a01 https://unetworkingab.medium.com/millions-of-active-websocke...
- zokier 3y ago> you have to go pretty much bare metal to actually be lightweight And yet here we are discussing WhatsApp which notably used Erlang, and I would not call BEAM particularly lightweight thing.
- londons_explore 3y agoAs long as you like having terribly restrictive threads....
- zabzonk 3y agonot a good article - all of the points are extremely cliched and/or just obvious normal practice.
- heisenbit 3y agoThis depends on your definition of good. This seems to be an article aimed at clueless decision makers to persuade them to keep them in the IBM realm. It has all the good messages like IBM got an AI tool for you. You love mainframes. You are not in a dead end where programmers are retiring faster than you can hire replacements and migration is impossible as the remaining crew can't support the additional effort for making the change. > scheduled for release in the fourth quarter, is designed > Refactoring COBOL to Java is a difficult process that can take decades and often fails. IBM expects the AI tool will speed the process by an order of magnitude. The good news with AI is that you usually get an answer even if it is impossible. And the company which way back was taken to court for it's vaporware can now point to the AI providing a schedule which will be impossible to implement. Either that or the DEA should raid it as someone is smoking something illegal there promising 10x schedule compression.
- ceva 3y agosimplicity over complexity at all cost!
- deleted 3y ago[deleted]
- supriyo-biswas 3y agoThis seems like one of those listicle-style, content-free article, all markers of what is possibly a ChatGPT generated article.
- jatins 3y agoYeah, very likely AI generated. Also extremely light and any meaningful concrete details. For examples things like /single responsibility principle" or "small teams" you can just put them anywhere to pad the length
- mypastself 3y agoThere’s something I like to call AI “almost-rightness”, although I’m sure there’s a better term for it out there. It’s a bit like linguistic uncanny valley. It’s when a model spits out a seemingly reasonable text, but it’s slightly off in the context. Nothing about it is incorrect, exactly, but it can be odd if you think about it for a second. So the title poses the question of how a product can be so robust with such a small team maintaning it. Yet one of the answers given in the article body is “having a small team”. If you want to point out that such a team size is an asset rather than weakness, then you should work out the argument fully. This way it ends up feeling almost tautological. Yeah, obviously, if you want a small team to work on a big product, one of the ingredients of that approach is a small team… Not saying this article is necessarily AI-generated, but there’s something almost-right about it.
- pawelduda 3y agoDoes anyone know if they still use Elixir or was it always just Erlang?
- lordnacho 3y agoIf you only focus on sending messages, you will get huge scalability. If you look at it, much of the problem was already solved at a fundamental level: kernel/network developers have been thinking about how to efficiently send messages since forever. Jabber devs had thought about certain problems on top of that as well. Put those together and don't add anything, you should be able to send a huge number of messages. Keep in mind it's not latency sensitive either, people can live with getting their messages 1s or late and with significant jitter. That's not to poop on the success of WhatsApp, I do think it's a good product. But it's a the product that enables the engineering to stay focused.
- DoItToMe81 3y agoIt's missing the most important part. It's based on a mature protocol (XMPP) that is battle tested in handling extremely heavy loads.
- ladyanita22 3y agoI wonder what's the current performance delta between FreeBSD and Linux
- Nellyz 3y ago[dead]