4 ms·

When the guy from Sequoia, who has the inside numbers on the top companies, says it is unheard of, its probably unheard of. The backend service is rather simpl

When the guy from Sequoia, who has the inside numbers on the top companies, says it is unheard of, its probably unheard of. The backend service is rather simple for 50 billion messages a day? hmmm not so much, I'm thinking.

Aloisius 13y ago
With the right architecture, it is largely a horizontal scaling problem. I ran and scaled the Napster server to 80 million users. We pushed about 70 billion "index file" commands an hour alone at peak. We had a chat and messaging system that also pushed millions of messages an hour. There were some pieces of data that had to live on all the servers (what user was connected to what server) which would have caused some problems eventually (though with how much memory you scan stick in today's machines, not for a very long time), but from experience, 10k really was the same as 80 million. The code to do this was a relatively small amount of C++ code. Most of our problems with scaling had to do with bugs in Linux when it was younger and "line-speed" Cisco switches that would blow up just because you pushed a few million packets per second through them.
- gordonguthrie 13y ago
  And Erlang is great and building and managing a horizontally scalable architecture - about building repeatable work units and co-ordinating them. So how many engineers did you have? What were your ratios?
  - Aloisius 13y ago
    The Napster production server engineering team was just me for most of it. Early on there were two other engineers, but one left for management and the other to start a company. I'm personally not familiar with Erlang to know if it would have worked for us. A lot of our scaling was about being very efficient with memory (the whole thing ran on little Pentium 2 machines with 2 GB of RAM each and part of the protocol unfortunately relied on MD5 hashes of files for file transfer which took up most of the memory).
- azth 13y ago
  Wow, you worked on Napster? You mention the server was written in C++. Out of curiosity, do you know what the client was written in? I found a random forum post by an anonymous user that said it was written in Delphi.
  - Aloisius 13y ago
    The Windows client was written in C++ (MFC iirc).
gaius 13y ago
The backend service is rather simple for 50 billion messages a day? Yes and no. It's a solved problem (Tibco RV installations handle this volume without breaking a sweat and have done so for years). On the other hand, this is some impressive price:performance for doing it. Or was; $19Bn would buy you a hell of a lot of Tibco licenses...

Aloisius 13y ago

With the right architecture, it is largely a horizontal scaling problem. I ran and scaled the Napster server to 80 million users. We pushed about 70 billion "index file" commands an hour alone at peak. We had a chat and messaging system that also pushed millions of messages an hour. There were some pieces of data that had to live on all the servers (what user was connected to what server) which would have caused some problems eventually (though with how much memory you scan stick in today's machines, not for a very long time), but from experience, 10k really was the same as 80 million. The code to do this was a relatively small amount of C++ code. Most of our problems with scaling had to do with bugs in Linux when it was younger and "line-speed" Cisco switches that would blow up just because you pushed a few million packets per second through them.

gordonguthrie 13y ago
And Erlang is great and building and managing a horizontally scalable architecture - about building repeatable work units and co-ordinating them. So how many engineers did you have? What were your ratios?
- Aloisius 13y ago
  The Napster production server engineering team was just me for most of it. Early on there were two other engineers, but one left for management and the other to start a company. I'm personally not familiar with Erlang to know if it would have worked for us. A lot of our scaling was about being very efficient with memory (the whole thing ran on little Pentium 2 machines with 2 GB of RAM each and part of the protocol unfortunately relied on MD5 hashes of files for file transfer which took up most of the memory).
azth 13y ago
Wow, you worked on Napster? You mention the server was written in C++. Out of curiosity, do you know what the client was written in? I found a random forum post by an anonymous user that said it was written in Delphi.
- Aloisius 13y ago
  The Windows client was written in C++ (MFC iirc).

gaius 13y ago

The backend service is rather simple for 50 billion messages a day? Yes and no. It's a solved problem (Tibco RV installations handle this volume without breaking a sweat and have done so for years). On the other hand, this is some impressive price:performance for doing it. Or was; $19Bn would buy you a hell of a lot of Tibco licenses...