This is actually a two part problem. oz captured half of it (maintaining hundred of connections in the NAT state table). It can also be caused by saturating you

This is actually a two part problem. oz captured half of it (maintaining hundred of connections in the NAT state table). It can also be caused by saturating your upstream bandwidth, which causes your devices send buffer to fill. When that buffer is full you end up with your packets waiting in the buffer (which is normally fifo). What makes this very noticeable most of the time is the ack part of the tcp connection getting stuck, there by forcing the remote server to wait to send you the next packet. There are ways to mitigate that kind of thing. the simplest being keeping upstream bandwidth levels limited to slightly less than your upstream connection speed. Traffic prioritization schemes also exist letting you have some say in the buffer order ( always putting acks, icmp, and ssh first in line for instance ).