3 ms·

select() should definitely never get used -- it's only even compiled in to the executable if you enable the non-default option of "FILED_NONBLOCK_HTTP" which yo

select() should definitely never get used -- it's only even compiled in to the executable if you enable the non-default option of "FILED_NONBLOCK_HTTP" which you almost certainly don't want. Did you compile filed in this way ? It sounds like you grep'd the source for select() but didn't actually see that it was #ifdef'd out. The number of concurrent connections is limited by your resource limits and the fact that file descriptors are cached. You can tune the cache, but you should update your resource limits if so, this is documented here in the man page available here: http://filed.rkeene.org/fossil/home?name=Manual http://filed.rkeene.org/fossil/home?name=Manual It does support keep-alive and I have no idea why you think it doesn't ( http://filed.rkeene.org/fossil/artifact?ln=1130-1132%201240-1250&name=8f507e131591c7e7 http://filed.rkeene.org/fossil/artifact?ln=1130-1132%201240-... ) Using only 2 threads (way less than the default) is also very sub-optimal since most of those threads will be waiting for the kernel to deliver the file. On average filed does about 2 system calls before asking the kernel to send the file to the client -- One read() of the HTTP request, one write() of the HTTP response header, and then the sendfile() of the contents. There's no reason not to use more threads than 2 other than to limit I/O (since they do not increase CPU utilization significantly).

wtarreau 9y ago
So you need as many threads as you're delivering parallel files ? That reminds me the late 90's when everyone started to discover how easy it was to deal with blocking I/O using threads until they discovered that threads are very slow since you have to suffer from context switches all the time. As a rule of thumb, you must never run more threads than you have CPU cores, or you'll get the taste of context switches. And it's really hard to scale using a model requiring one thread per connection. You'll hardly stand one million concurrent connections this way, and this can definitely happen for large objects.
wtarreau 9y ago
Also, thanks for the link to the code, I found the issue with keep-alive (in fact there are two such issues). The first one is that you're looking for the Connection header with the Keep-Alive value, but this is needed only for HTTP/1.0. In 1.1 it's by default so you will not always get it, it will depend on the browsers, proxies etc. The second thing is that checking for headers and values this way is very unreliable as the connection header can contain other stuff, like "TE" to indicate that the TE request header field is present and must not be forwarded to next hop. In this case, the Connection header tokens are delimited by commas and you don't know either if it will match. One could argue that all these cases are not very common in field but it's the difference between being spec-compliant thus interoperable and working most of the time :-)
wtarreau 9y ago
So I could inject in keep-alive at 100 concurrent threads but it was very slow (2200 req/s at 12% CPU), idling on I-don't-know-what. And at 1000 threads (still 100 concurrent connections), it immediately ate all my memory and got killed by the OOM killer : Out of memory: Kill process 7134 (filed) score 779 or sacrifice child Killed process 7134 (filed) total-vm:7680976kB, anon-rss:7140908kB, file-rss:1988kB, shmem-rss:100kB oom_reaper: reaped process 7134 (filed), now anon-rss:7143588kB, file-rss:1980kB, shmem-rss:100kB 7 GB for 100 connections is a bit excessive in my opinion :-) There are definitely still a number of issues to be addressed before it can be used in production, you definitely need to have a more robust architecture and request parser first.

wtarreau 9y ago

So you need as many threads as you're delivering parallel files ? That reminds me the late 90's when everyone started to discover how easy it was to deal with blocking I/O using threads until they discovered that threads are very slow since you have to suffer from context switches all the time. As a rule of thumb, you must never run more threads than you have CPU cores, or you'll get the taste of context switches. And it's really hard to scale using a model requiring one thread per connection. You'll hardly stand one million concurrent connections this way, and this can definitely happen for large objects.

wtarreau 9y ago

Also, thanks for the link to the code, I found the issue with keep-alive (in fact there are two such issues). The first one is that you're looking for the Connection header with the Keep-Alive value, but this is needed only for HTTP/1.0. In 1.1 it's by default so you will not always get it, it will depend on the browsers, proxies etc. The second thing is that checking for headers and values this way is very unreliable as the connection header can contain other stuff, like "TE" to indicate that the TE request header field is present and must not be forwarded to next hop. In this case, the Connection header tokens are delimited by commas and you don't know either if it will match. One could argue that all these cases are not very common in field but it's the difference between being spec-compliant thus interoperable and working most of the time :-)

wtarreau 9y ago

So I could inject in keep-alive at 100 concurrent threads but it was very slow (2200 req/s at 12% CPU), idling on I-don't-know-what. And at 1000 threads (still 100 concurrent connections), it immediately ate all my memory and got killed by the OOM killer : Out of memory: Kill process 7134 (filed) score 779 or sacrifice child Killed process 7134 (filed) total-vm:7680976kB, anon-rss:7140908kB, file-rss:1988kB, shmem-rss:100kB oom_reaper: reaped process 7134 (filed), now anon-rss:7143588kB, file-rss:1980kB, shmem-rss:100kB 7 GB for 100 connections is a bit excessive in my opinion :-) There are definitely still a number of issues to be addressed before it can be used in production, you definitely need to have a more robust architecture and request parser first.