dbserver swf # ps auxw | grep crawl root 6984 3.0 0.5 215832 5168 pts/5 Sl+ 12:59 0:00 ./crawl I'm using a linked-list in C to store all of the

dbserver swf # ps auxw | grep crawl root 6984 3.0 0.5 215832 5168 pts/5 Sl+ 12:59 0:00 ./crawl I'm using a linked-list in C to store all of the queued-up URLs so all of the threads can mark them as being worked on or whatever. Also, the app and curl lib takes a few megs too. I'm sure that the heap and multi-threadding libs are responsible for some of the resident memory usage. :) the on-disk binary is only 12k if that make a difference. ;) The struct: typedef struct my_url { char * url; int retcode; double old_size; double new_size; status status; struct my_url * next; char * filename; int flashurl_id; int depth; } my_url; I also like keeping them in memory as opposed to a BDB because of access speed - 25+ threads don't have to wait when it's all in memory, but 25+ threads hitting a BDB would be much nastier on the OS.