3 ms·
The key is to combine cron jobs with Tasks. Tasks are am easy to use and powerful feature of Google Appengine that let you deal with the standard limitations.
by mshafrir 16y ago
The key is to combine cron jobs with Tasks. Tasks are am easy to use and powerful feature of Google Appengine that let you deal with the standard limitations. For example, rather than doing some operation on 1000 entities in one request, you spin off 1000 tasks, each handling the operation on one entity. Read up about them at http://code.google.com/appengine/docs/python/taskqueue/ http://code.google.com/appengine/docs/python/taskqueue/
You can also explore the new Mapper API at http://blog.notdot.net/2010/05/Exploring-the-new-mapper-API http://blog.notdot.net/2010/05/Exploring-the-new-mapper-API
- mmastrac 16y agoAgreed, the Mapper API is the best way to deal with processing large numbers of entities from the datastore. AppEngine requires careful thought of application design at this point. Everything is an HTTP request and subject to the same restrictions. I've heard rumours that they'll be launching longer-lived tasks at some point in the future, but I have no idea when that will happen.