5 ms·

Google App Engine is now much faster, after latest maintenance

125 points by ordinaryman 16y ago

endlessvoid94 16y ago
This headline should say "GETs to the datastore are now much faster". EDIT: I neglected to see the other datastore figures.
- ordinaryman 16y ago
  You should also check out the latency for Put, Update, Delete and Query on that same page.
  - endlessvoid94 16y ago
    Ah, you're right.
- petervandijck 16y ago
  Put, Update and Delete are much faster too.
  - throw_away 16y ago
    looks like they took a 1hr write outage at deployment time, so presumably a fairly beefy change under the hood.
    - mmastrac 16y ago
      Most of the datastore maintenance windows are about an hour long. Previous ones have even gone two hours.
tropin 16y ago
I expected something more than a graphic.
- tropin 16y ago
  Ah, thanks for the hate, ordinaryman.
  - ordinaryman 16y ago
    Not me. And, even if I wanted to, I can't. Also, I guess Google folks would post about the performance boost later.
    - tropin 16y ago
      Ok, please excuse me.
      - YuriNiyazov 16y ago
        In either case - from the HN guidelines: "Resist complaining about being downmodded. It never does any good, and it makes boring reading."
tropin 16y ago
This site, probably NSFW, makes use of the datastore (for the thumbnails) and it's now much faster, it's a pity we can't compare speeds with the old latency. http://wallres.saiswa.com/?tags=girls&pagina=23 http://wallres.saiswa.com/?tags=girls&pagina=23
- ordinaryman 16y ago
  A SFW website with a lot of images too.. yes, considerably faster.. http://my.educonsole.com/free-educational-games-for-kids/identify-fruits-names-and-photos http://my.educonsole.com/free-educational-games-for-kids/ide... (By the way, the above site was my gift for my son on his fourth birthday)
  - tropin 16y ago
    It's amazing the difference low latency can make when dealing with lots of small images.
- jws 16y ago
  My dinky app dropped from 60ms/request to 20ms/request, but over the next 3 hours crept up more or less linearly to about 30ms/request.
  - jspencer 16y ago
    I missed the word "app" when reading your comment and thought it was a clever joke.
mmastrac 16y ago
Our app (http://gri.pe http://gri.pe) feels a lot snappier after this latest bit of datastore maintenance. We've been using as much memcache as possible to avoid datastore access because prior to this maintenance, datastore access was spiky and even the simplest of queries had a small chance of timing out. I can say for sure that some of our datastore-heavy cron jobs were taking > 200ms, but are around 70ms now. It's a shame that the stats for > 12hrs ago seem to have been wiped. I bet we'd be seeing a marked improvement on our ms/request stat.
- bad_user 16y ago
  Unrelated question: how are you doing those cron jobs. I tried using the task queue, but it kept timing out on me, even when I broke my tasks into smaller ones (couldn't figure out a threshold as the behavior of the datastore was totally unpredictable).
  - madflo 16y ago
    Via Cron Jobs ? http://code.google.com/appengine/docs/python/config/cron.html http://code.google.com/appengine/docs/python/config/cron.htm...
    - bad_user 16y ago
      Quote from that article: A URL invoked by cron is subject to the same limits and quotas as a normal HTTP request, including the request time limit. And I'm looking for some tips and tricks to do heavy processing (which is really needed when working with non-relational datastores).
      - mshafrir 16y ago
        The key is to combine cron jobs with Tasks. Tasks are am easy to use and powerful feature of Google Appengine that let you deal with the standard limitations. For example, rather than doing some operation on 1000 entities in one request, you spin off 1000 tasks, each handling the operation on one entity. Read up about them at http://code.google.com/appengine/docs/python/taskqueue/ http://code.google.com/appengine/docs/python/taskqueue/ You can also explore the new Mapper API at http://blog.notdot.net/2010/05/Exploring-the-new-mapper-API http://blog.notdot.net/2010/05/Exploring-the-new-mapper-API
joshzayin 16y ago
Looking at the error rates, I wonder what happened at around 12pm? (Screenshots: http://grab.by/7gJ4 http://grab.by/7gJ4 http://grab.by/7gJ5 http://grab.by/7gJ5 http://grab.by/7gJh http://grab.by/7gJh)
- forgotusername 16y ago
  From the maintenances mailing list: The maintenance work has been completed. The datastore was in a read-only period between 12:02-12:45 for a total of 43 minutes. Some users reported two periods of 500 errors unrelated to datastore writes, but these issues should no longer be occurring. On Nov 6, 12:01 pm, App Engine Downtime Notify <i...@google.com>
smoody 16y ago
the app engine applications gallery is now gone, too. coincidence??? http://appgallery.appspot.com/results?topapps=true http://appgallery.appspot.com/results?topapps=true
- realmojo 16y ago
  enterprisey.
grandalf 16y ago
thank you google! wow, this is amazing.
gfodor 16y ago
As any ops guy will tell you, this is the finest form of graph porn there is.
- pjscott 16y ago
  It would have been sexier on a semi-log-y scale. That said, as someone who's always been annoyed at the high latency of App Engine's datastore, this is huge if they can deliver this level of latency with low variability.
fharper1961 16y ago
I just tried a high traffic site that runs on App Engine (appbrain.com). It used to feel a bit slow, and now it really flies. It definitely makes me reconsider using App Engine for new projects. Disclaimer: I have no connection to Appbrain aside from meeting the developers once.
cageface 16y ago
The sense that the app store has been a neglected, second-tier priority is deep enough now that it will take them a while to regain significant developer trust. AWS is very close to a de-facto standard now. Edit: I mean app engine, the hosted web service. Not the app store. Thanks.
- grandalf 16y ago
  I agree. However I hope they do regain that trust b/c app engine is a far more pleasant platform to develop on.
  - c1sc0 16y ago
    Can't repeat this enough: the main reason I develop on appengine is that it removes server admin headaches completely from the equation. Ok, you get other headaches in return, but most of the time it's stuff I can deal with.
- ordinaryman 16y ago
  app store has been a neglected I guess you are confusing the Google App Engine's App Gallery : http://appgallery.appspot.com/ http://appgallery.appspot.com/, which has been removed for some reasons and the Google Apps Marketplace : http://www.google.com/enterprise/marketplace http://www.google.com/enterprise/marketplace, which is the actual App Store, where you could pay and install apps. App Gallery was just a GAE application show-case center, and most likely was causing confusion to Google Apps customers too. I am still a developer who trusts GAE (my app was on the appgallery and is also listed on marketplace) and believe that they will continue to compete for developer mind-share, as they proved yet again with this performance update. Let go of the fear, uncertainty & doubt and hop on to try an app. You will like it too.
wkornewald 16y ago
It's still too early to tell. This would not be the first time that speed improves after a maintenance and then it slowly gets worse every week until it looks almost like before the maintenance.
- benjaminfox 16y ago
  Is it possible that the degradation in performance is due to more developers building more apps and driving more traffic to App Engine as time goes on?
  - wkornewald 16y ago
    The performance degrades too quickly for this. In two or three months we might be on our old performance level, again. Well, I seriously hope that we'll stay fast this time. Only time will tell.
gmosx 16y ago
that 's great :) hopefully, now they can work on improving support for JDO/JPA and release those great features announced in Google I/O.
k7d 16y ago
Good news but I really hope Google figures out how to solve DeadlineExceededError. They are popping up from time to time for my app (http://www.knowcase.com http://www.knowcase.com) and I don't think it's datastore related.
- ma2rten 16y ago
  This reply from a few days back should shed some light on the issue: http://news.ycombinator.com/item?id=1823362 http://news.ycombinator.com/item?id=1823362
iampims 16y ago
They even solved a bug with the datastore quota not being properly updated after a massive delete of entities. Kudos to the team, and I, too, hope that the latency will remain this low forever and ever…

125 points by ordinaryman 16y ago

This headline should say "GETs to the datastore are now much faster". EDIT: I neglected to see the other datastore figures.

ordinaryman 16y ago
You should also check out the latency for Put, Update, Delete and Query on that same page.
- endlessvoid94 16y ago
  Ah, you're right.
petervandijck 16y ago
Put, Update and Delete are much faster too.
- throw_away 16y ago
  looks like they took a 1hr write outage at deployment time, so presumably a fairly beefy change under the hood.
  - mmastrac 16y ago
    Most of the datastore maintenance windows are about an hour long. Previous ones have even gone two hours.

tropin 16y ago

I expected something more than a graphic.

tropin 16y ago
Ah, thanks for the hate, ordinaryman.
- ordinaryman 16y ago
  Not me. And, even if I wanted to, I can't. Also, I guess Google folks would post about the performance boost later.
  - tropin 16y ago
    Ok, please excuse me.
    - YuriNiyazov 16y ago
      In either case - from the HN guidelines: "Resist complaining about being downmodded. It never does any good, and it makes boring reading."

tropin 16y ago

This site, probably NSFW, makes use of the datastore (for the thumbnails) and it's now much faster, it's a pity we can't compare speeds with the old latency. http://wallres.saiswa.com/?tags=girls&pagina=23 http://wallres.saiswa.com/?tags=girls&pagina=23

ordinaryman 16y ago
A SFW website with a lot of images too.. yes, considerably faster.. http://my.educonsole.com/free-educational-games-for-kids/identify-fruits-names-and-photos http://my.educonsole.com/free-educational-games-for-kids/ide... (By the way, the above site was my gift for my son on his fourth birthday)
- tropin 16y ago
  It's amazing the difference low latency can make when dealing with lots of small images.
jws 16y ago
My dinky app dropped from 60ms/request to 20ms/request, but over the next 3 hours crept up more or less linearly to about 30ms/request.
- jspencer 16y ago
  I missed the word "app" when reading your comment and thought it was a clever joke.

mmastrac 16y ago

Our app (http://gri.pe http://gri.pe) feels a lot snappier after this latest bit of datastore maintenance. We've been using as much memcache as possible to avoid datastore access because prior to this maintenance, datastore access was spiky and even the simplest of queries had a small chance of timing out. I can say for sure that some of our datastore-heavy cron jobs were taking > 200ms, but are around 70ms now. It's a shame that the stats for > 12hrs ago seem to have been wiped. I bet we'd be seeing a marked improvement on our ms/request stat.

bad_user 16y ago
Unrelated question: how are you doing those cron jobs. I tried using the task queue, but it kept timing out on me, even when I broke my tasks into smaller ones (couldn't figure out a threshold as the behavior of the datastore was totally unpredictable).
- madflo 16y ago
  Via Cron Jobs ? http://code.google.com/appengine/docs/python/config/cron.html http://code.google.com/appengine/docs/python/config/cron.htm...
  - bad_user 16y ago
    Quote from that article: A URL invoked by cron is subject to the same limits and quotas as a normal HTTP request, including the request time limit. And I'm looking for some tips and tricks to do heavy processing (which is really needed when working with non-relational datastores).
    - mshafrir 16y ago
      The key is to combine cron jobs with Tasks. Tasks are am easy to use and powerful feature of Google Appengine that let you deal with the standard limitations. For example, rather than doing some operation on 1000 entities in one request, you spin off 1000 tasks, each handling the operation on one entity. Read up about them at http://code.google.com/appengine/docs/python/taskqueue/ http://code.google.com/appengine/docs/python/taskqueue/ You can also explore the new Mapper API at http://blog.notdot.net/2010/05/Exploring-the-new-mapper-API http://blog.notdot.net/2010/05/Exploring-the-new-mapper-API

joshzayin 16y ago

Looking at the error rates, I wonder what happened at around 12pm? (Screenshots: http://grab.by/7gJ4 http://grab.by/7gJ4 http://grab.by/7gJ5 http://grab.by/7gJ5 http://grab.by/7gJh http://grab.by/7gJh)

forgotusername 16y ago
From the maintenances mailing list: The maintenance work has been completed. The datastore was in a read-only period between 12:02-12:45 for a total of 43 minutes. Some users reported two periods of 500 errors unrelated to datastore writes, but these issues should no longer be occurring. On Nov 6, 12:01 pm, App Engine Downtime Notify <i...@google.com>

smoody 16y ago

the app engine applications gallery is now gone, too. coincidence??? http://appgallery.appspot.com/results?topapps=true http://appgallery.appspot.com/results?topapps=true

realmojo 16y ago
enterprisey.

grandalf 16y ago

thank you google! wow, this is amazing.

gfodor 16y ago

As any ops guy will tell you, this is the finest form of graph porn there is.

pjscott 16y ago
It would have been sexier on a semi-log-y scale. That said, as someone who's always been annoyed at the high latency of App Engine's datastore, this is huge if they can deliver this level of latency with low variability.

fharper1961 16y ago

I just tried a high traffic site that runs on App Engine (appbrain.com). It used to feel a bit slow, and now it really flies. It definitely makes me reconsider using App Engine for new projects. Disclaimer: I have no connection to Appbrain aside from meeting the developers once.

cageface 16y ago

The sense that the app store has been a neglected, second-tier priority is deep enough now that it will take them a while to regain significant developer trust. AWS is very close to a de-facto standard now. Edit: I mean app engine, the hosted web service. Not the app store. Thanks.

grandalf 16y ago
I agree. However I hope they do regain that trust b/c app engine is a far more pleasant platform to develop on.
- c1sc0 16y ago
  Can't repeat this enough: the main reason I develop on appengine is that it removes server admin headaches completely from the equation. Ok, you get other headaches in return, but most of the time it's stuff I can deal with.
ordinaryman 16y ago
app store has been a neglected I guess you are confusing the Google App Engine's App Gallery : http://appgallery.appspot.com/ http://appgallery.appspot.com/, which has been removed for some reasons and the Google Apps Marketplace : http://www.google.com/enterprise/marketplace http://www.google.com/enterprise/marketplace, which is the actual App Store, where you could pay and install apps. App Gallery was just a GAE application show-case center, and most likely was causing confusion to Google Apps customers too. I am still a developer who trusts GAE (my app was on the appgallery and is also listed on marketplace) and believe that they will continue to compete for developer mind-share, as they proved yet again with this performance update. Let go of the fear, uncertainty & doubt and hop on to try an app. You will like it too.

wkornewald 16y ago

It's still too early to tell. This would not be the first time that speed improves after a maintenance and then it slowly gets worse every week until it looks almost like before the maintenance.

benjaminfox 16y ago
Is it possible that the degradation in performance is due to more developers building more apps and driving more traffic to App Engine as time goes on?
- wkornewald 16y ago
  The performance degrades too quickly for this. In two or three months we might be on our old performance level, again. Well, I seriously hope that we'll stay fast this time. Only time will tell.

gmosx 16y ago

that 's great :) hopefully, now they can work on improving support for JDO/JPA and release those great features announced in Google I/O.

k7d 16y ago

Good news but I really hope Google figures out how to solve DeadlineExceededError. They are popping up from time to time for my app (http://www.knowcase.com http://www.knowcase.com) and I don't think it's datastore related.

ma2rten 16y ago
This reply from a few days back should shed some light on the issue: http://news.ycombinator.com/item?id=1823362 http://news.ycombinator.com/item?id=1823362

iampims 16y ago

They even solved a bug with the datastore quota not being properly updated after a massive delete of entities. Kudos to the team, and I, too, hope that the latency will remain this low forever and ever…