3 ms·

I worked on ICQ project as a backend developer (C/C++) after Mail.Ru bought ICQ from AOL. We had legacy from AOL - AOL's proprietary TCP stack in user space.

I worked on ICQ project as a backend developer (C/C++) after Mail.Ru bought ICQ from AOL. We had legacy from AOL - AOL's proprietary TCP stack in user space. ICQ used AOL's TCP stack for external connections with outer world (i.e. with clients). I asked AOL engineers why they needed TCP stack in user space. They said that back in old days (90s) there was no good scalable TCP implementation which could handle many simultaneous connections properly. Each process which used this TCP stack had to fully own network interface. Of course, we gradually replaced AOL's TCP stack with native stack because nowadays Linux TCP stack is good enough. May be these days someone still need proprietary TCP stack in user land to meet specific needs which aren't available in native Linux implementation. Also if you have TCP stack in user land, you have more control over it.

ams6110 10y ago
AOL had a bespoke webserver too. https://en.wikipedia.org/wiki/AOLserver https://en.wikipedia.org/wiki/AOLserver
- SwellJoe 10y ago
  I really liked AOLServer! My current company's first website ran on OpenACS, which was written in Tcl and ran on aolserver. It was really quite a nice system (and Tcl is a quite nice language). But, the web server worked with the standard Linux network stack; at least, it did by the time I saw it.
  - nulltype 10y ago
    Tcl may be the most under appreciated language.
- retr0h 10y ago
  It was also written in tcl
ktRolster 10y ago
They said that back in old days (90s) there was no good scalable TCP implementation which could handle many simultaneous connections properly. I think that's true, using select() the max number of connections is ~1024. epoll() lets you do many more, but that wasn't available back then.
- wahern 10y ago
  select doesn't have such an inherent limit. The limit is an artifact of the way the fd_set structure and macros are defined and implemented. Most unix-like kernels, including Linux, will accept a much larger fd_set. The userspace libraries (including glibc) even often permit you to redefine FD_SETSIZE at compile time, which makes it even easier. That can cause problems when sharing fd_sets between different libraries, though, so isn't a good idea. IME once you get past an FD_SETSIZE of 8192 the overhead of preparing 3KB of data on every single select call begins to show appreciable CPU load. CPUs are fast enough that you'll rarely get more than a handful of pending events per call even with thousands of active connections, so the cost isn't amortized well. Nonetheless, if you use epoll without actually making use of and benefitting from persistent events (as some naive applications do, including a much-touted "high-performance" platform), select can even be faster than epoll under load. Under load, a naive application which resets events on a descriptor every time it processes that descriptor is pushing much more data through syscalls than select would.
toast0 10y ago
One of the interesting features of the AOL TCP stack is that they listed on all the ports -- pretty easy to do if you're in userspace.

ams6110 10y ago

AOL had a bespoke webserver too. https://en.wikipedia.org/wiki/AOLserver https://en.wikipedia.org/wiki/AOLserver

SwellJoe 10y ago
I really liked AOLServer! My current company's first website ran on OpenACS, which was written in Tcl and ran on aolserver. It was really quite a nice system (and Tcl is a quite nice language). But, the web server worked with the standard Linux network stack; at least, it did by the time I saw it.
- nulltype 10y ago
  Tcl may be the most under appreciated language.
retr0h 10y ago
It was also written in tcl

ktRolster 10y ago

They said that back in old days (90s) there was no good scalable TCP implementation which could handle many simultaneous connections properly. I think that's true, using select() the max number of connections is ~1024. epoll() lets you do many more, but that wasn't available back then.

wahern 10y ago
select doesn't have such an inherent limit. The limit is an artifact of the way the fd_set structure and macros are defined and implemented. Most unix-like kernels, including Linux, will accept a much larger fd_set. The userspace libraries (including glibc) even often permit you to redefine FD_SETSIZE at compile time, which makes it even easier. That can cause problems when sharing fd_sets between different libraries, though, so isn't a good idea. IME once you get past an FD_SETSIZE of 8192 the overhead of preparing 3KB of data on every single select call begins to show appreciable CPU load. CPUs are fast enough that you'll rarely get more than a handful of pending events per call even with thousands of active connections, so the cost isn't amortized well. Nonetheless, if you use epoll without actually making use of and benefitting from persistent events (as some naive applications do, including a much-touted "high-performance" platform), select can even be faster than epoll under load. Under load, a naive application which resets events on a descriptor every time it processes that descriptor is pushing much more data through syscalls than select would.

toast0 10y ago

One of the interesting features of the AOL TCP stack is that they listed on all the ports -- pretty easy to do if you're in userspace.