Take a look at the c10k problem, a classic resource for this problem domain: http://www.kegel.com/c10k.html http://www.kegel.com/c10k.html You probably don't w

Take a look at the c10k problem, a classic resource for this problem domain: http://www.kegel.com/c10k.html http://www.kegel.com/c10k.html You probably don't want to spawn one thread per connection, but get notified by the operating system when one of a bunch of TCP socket connections has an event for you to handle. You can achieve this by the kernel calls epoll, poll, select, kqueue. For Python, I recommend taking a look at tornado (https://github.com/facebook/tornado https://github.com/facebook/tornado), an asynchronous socket handling framework. In particular the "io-loop", which you could use for handling the connections. As for serialization protocols, don't roll your own but consider JSON for maximum flexibility, and protocol buffers for maximum efficiency. For communication between servers consider well known libraries and protocols such as AMQP (eg RabbitMQ) and Redis pubsub. Be sure to read up about Linux kernel settings to improve the number of maximum open sockets, etc. Here are several good readings referenced: http://serverfault.com/questions/10852/what-limits-the-maximum-number-of-connections-on-a-linux-server/10919#10919 http://serverfault.com/questions/10852/what-limits-the-maxim... Also, you'll want to get familiar with some tools you'll be using a lot, such as lsof and netcat. About the infrastructure design: you'll most likely end up with several different server applications and possibly a number of simultaneous running instances. Consider Amazon EC2 micro instances or something similar to start with. I'd recommend to experiment with building several prototypes before the real product, getting to know the approaches, architectures, bottlenecks, etc. Enjoy the learning experience! Finally, spend enough time on the supporting tools, such as automatic deployment (chef, puppet, fabric), automated clients, etc. which will make your life a lot easier! :)