the bottleneck is twitter's api limits, data-wise and http connection wise I have headroom, lots of. simple http connections to parse/spider the follower recor

the bottleneck is twitter's api limits, data-wise and http connection wise I have headroom, lots of. simple http connections to parse/spider the follower records from public pages is a no-go since twitter blocks the IP then, and scaling this out will eventually not end in a good way.