There isn't a single good answer to this, all depending on what storage you use, the work you need to do etc. But in general I'd think you want the task to be a

There isn't a single good answer to this, all depending on what storage you use, the work you need to do etc. But in general I'd think you want the task to be as granular as sensible, so you can spread the work to many servers. The best thing you can do is to try out, stress-test and benchmark the different ways to do this. As of your description, I'm not even sure if celery is the right tool for the job, but you could join irc.freenode.net #celery to get more information.