4 ms·

Google App Engine turns 2

48 points by ordinaryman 17y ago

axiom 17y ago
Until they speed up the datastore by a factor of 10 the app engine just isn't a viable platform. They don't even use it internally for that very reason (this is coming from colleagues who work there.)
- jeff18 17y ago
  How do you reconcile that statement with the fact that tons of developers do actually use it? Myself included. ;)
  - moe 17y ago
    What do the error rates on write operations look like these days? This ticket has held me back from giving appengine a try so far: http://code.google.com/p/googleappengine/issues/detail?id=764 http://code.google.com/p/googleappengine/issues/detail?id=76...
    - jeff18 17y ago
      This was largely solved in 1.3.1. I quote: Reduced error rate with Automatic Datastore Retries - We've heard a lot of feedback that you don't want to deal with the Datastore's sporadic errors. In response, App Engine now automatically retries all datastore calls (with the exception of transaction commits) when your applications encounters a datastore error caused by being unable to reach Bigtable. Datastore retries automatically builds in what many of you have been doing in your code already, and our tests have shown it drastically reduces the number of errors your application experiences (by up to 3-4x error reduction for puts, 10-30x for gets).
  - axiom 17y ago
    We use it too. That's why I can make that statement. A DB request takes over 100ms on the datastore. That's insane and unusable for about 90% of applications. Even their implementation of memcached is several orders of magnitude slower than actual memcached.
    - jeff18 17y ago
      You claim that about 90% of applications require sub-100ms latencies on datastore writes. Can you back this up? On the contrary, it seems to me that very few applications need to be blocked by writing to the datastore, let alone cases where 100 milliseconds make them unusable. Secondly, 3 millisecond memcached access is several orders of magnitude too slow? 3 nanosecond memcache access would be quite a physical achievement. ;)
      - axiom 17y ago
        If you're getting 3ms memcached access then you must be using a different app engine than I am. And it's 100ms reads not writes. Writes are slower. Much slower.
        jeff18 17y ago
        According to the independently verifiable health dashboard, the current memcache latency average for today is about 7ms[1]. The average datastore get() time is around 50ms[2]. The average datastore put() time is around 100ms[3]. Are you accusing Google of falsifying this trivially reproducible and abundant data? That would be quite an interesting story. ;) [1] http://code.google.com/status/appengine/detail/memcache/2010/04/07#ae-trust-detail-memcache-get-latency http://code.google.com/status/appengine/detail/memcache/2010... [2] http://code.google.com/status/appengine/detail/datastore/2010/04/07#ae-trust-detail-datastore-get-latency http://code.google.com/status/appengine/detail/datastore/201... [3] http://code.google.com/status/appengine/detail/datastore/2010/04/07#ae-trust-detail-datastore-put-latency http://code.google.com/status/appengine/detail/datastore/201...
        kalid 17y ago
        I don't think anyone would accuse google of falsifying data, but stats can be misleading. I might have missed it, but I didn't see a standard deviation for those numbers -- if you're doing several reads and they happen to be "above average" then you could see the random appearance of sluggishness. Just checking now (http://imgur.com/8Jb82 http://imgur.com/8Jb82) there are some really long requests (800ms!) which might get smoothed out in an average.
- alecthomas 17y ago
  AppEngine is used quite extensively internally.
- e98cuenc 17y ago
  It depends a lot on each particular project. Panoramio and pubhubsubbub are two of the most well known Google projects built in AppEngine.
- arachnid 17y ago
  We're constantly working on improving datastore performance, but asking for a factor of 10 improvement is outside the bounds of reasonableness: that would bring latencies well under that of a disk seek, so unless you're prepared to pay to have all your data stored in ram, it's effectively impossible for reads.
ordinaryman 17y ago
Apart from being the tipping point - nudging me to start out on my own with its liberal free quotas, I should also thank Google App Engine for not supporting Java during the initial release and thus exposing me to Python. Coming from corporate Java and RDBMS world, the non-RDBMS datastore (BigTable), Python and having to scale while being more resource conscious, have all been pleasant experiences.
- ez77 17y ago
  Just out of curiosity, what tools (text editor, shell, IDE, etc.) do you use when working on GAE?
  - ordinaryman 17y ago
    Emacs & cmd.exe
khelloworld 17y ago
As you begin your 3rd year, here are a few things I wish for you: 1. May you be able to work with your brother, Google Apps, to allow SSL support for custom domains. 2. May you be able to lift your 1 MB cap per urlfetch request to at least 10 MB so that your users can write some cool apps using the YouTube API. 3. May you inherit some snazzy search skills this year from Google Search, your grand daddy, and allow your users to whip up custom search engines for their apps in the blink of an eye (or the press of a button, rather). Happy Birthday GAE! - All of your 250k users.
- simonk 17y ago
  Top two are in the roadmap http://code.google.com/appengine/docs/roadmap.html http://code.google.com/appengine/docs/roadmap.html
- mshafrir 17y ago
  The first one is a non-starter for many startups.
- markkoberlein 17y ago
  I would agree that the lack of support of SSL on custom domains and the size limitations (my main issue is with the upload limits) have kept GAE from really taking off. I have used it for a small non-profit app and I host my personal consulting site on it. However, because of the limitations I'm building my startup on rackspace instead of GAE. I really love GAE as a development platform, especially as a single developer, but I hope they will work through their roadmap faster.
ez77 17y ago
I have the feeling this is an old question, but I can't find a specific answer. Is it possible to host a Google App only with your custom domain while disabling the appspot.com URL? I could understand this being a restriction on free accounts, but does it also apply to paying customers? Thanks PS: If not possible, is there a way to redirect your appspot.com URL to your custom URL?
- ordinaryman 17y ago
  Ideally, if you do not publicize the App ID / appspot domain, it is not likely to be known & requested. Anyway, the simplest way to do this will be to redirect requests received on the appspot domain to your custom domain, from within your code.
- ez77 17y ago
  I found a relevant discussion: http://stackoverflow.com/questions/1364733/block-requests-from-appspot-com-and-force-custom-domain-in-google-app-engine http://stackoverflow.com/questions/1364733/block-requests-fr...
apower 17y ago
Yawn.

48 points by ordinaryman 17y ago

axiom 17y ago

Until they speed up the datastore by a factor of 10 the app engine just isn't a viable platform. They don't even use it internally for that very reason (this is coming from colleagues who work there.)

jeff18 17y ago
How do you reconcile that statement with the fact that tons of developers do actually use it? Myself included. ;)
- moe 17y ago
  What do the error rates on write operations look like these days? This ticket has held me back from giving appengine a try so far: http://code.google.com/p/googleappengine/issues/detail?id=764 http://code.google.com/p/googleappengine/issues/detail?id=76...
  - jeff18 17y ago
    This was largely solved in 1.3.1. I quote: Reduced error rate with Automatic Datastore Retries - We've heard a lot of feedback that you don't want to deal with the Datastore's sporadic errors. In response, App Engine now automatically retries all datastore calls (with the exception of transaction commits) when your applications encounters a datastore error caused by being unable to reach Bigtable. Datastore retries automatically builds in what many of you have been doing in your code already, and our tests have shown it drastically reduces the number of errors your application experiences (by up to 3-4x error reduction for puts, 10-30x for gets).
- axiom 17y ago
  We use it too. That's why I can make that statement. A DB request takes over 100ms on the datastore. That's insane and unusable for about 90% of applications. Even their implementation of memcached is several orders of magnitude slower than actual memcached.
  - jeff18 17y ago
    You claim that about 90% of applications require sub-100ms latencies on datastore writes. Can you back this up? On the contrary, it seems to me that very few applications need to be blocked by writing to the datastore, let alone cases where 100 milliseconds make them unusable. Secondly, 3 millisecond memcached access is several orders of magnitude too slow? 3 nanosecond memcache access would be quite a physical achievement. ;)
    - axiom 17y ago
      If you're getting 3ms memcached access then you must be using a different app engine than I am. And it's 100ms reads not writes. Writes are slower. Much slower.
      - jeff18 17y ago
        According to the independently verifiable health dashboard, the current memcache latency average for today is about 7ms[1]. The average datastore get() time is around 50ms[2]. The average datastore put() time is around 100ms[3]. Are you accusing Google of falsifying this trivially reproducible and abundant data? That would be quite an interesting story. ;) [1] http://code.google.com/status/appengine/detail/memcache/2010/04/07#ae-trust-detail-memcache-get-latency http://code.google.com/status/appengine/detail/memcache/2010... [2] http://code.google.com/status/appengine/detail/datastore/2010/04/07#ae-trust-detail-datastore-get-latency http://code.google.com/status/appengine/detail/datastore/201... [3] http://code.google.com/status/appengine/detail/datastore/2010/04/07#ae-trust-detail-datastore-put-latency http://code.google.com/status/appengine/detail/datastore/201...
        kalid 17y ago
        I don't think anyone would accuse google of falsifying data, but stats can be misleading. I might have missed it, but I didn't see a standard deviation for those numbers -- if you're doing several reads and they happen to be "above average" then you could see the random appearance of sluggishness. Just checking now (http://imgur.com/8Jb82 http://imgur.com/8Jb82) there are some really long requests (800ms!) which might get smoothed out in an average.
alecthomas 17y ago
AppEngine is used quite extensively internally.
e98cuenc 17y ago
It depends a lot on each particular project. Panoramio and pubhubsubbub are two of the most well known Google projects built in AppEngine.
arachnid 17y ago
We're constantly working on improving datastore performance, but asking for a factor of 10 improvement is outside the bounds of reasonableness: that would bring latencies well under that of a disk seek, so unless you're prepared to pay to have all your data stored in ram, it's effectively impossible for reads.

ordinaryman 17y ago

Apart from being the tipping point - nudging me to start out on my own with its liberal free quotas, I should also thank Google App Engine for not supporting Java during the initial release and thus exposing me to Python. Coming from corporate Java and RDBMS world, the non-RDBMS datastore (BigTable), Python and having to scale while being more resource conscious, have all been pleasant experiences.

ez77 17y ago
Just out of curiosity, what tools (text editor, shell, IDE, etc.) do you use when working on GAE?
- ordinaryman 17y ago
  Emacs & cmd.exe

khelloworld 17y ago

As you begin your 3rd year, here are a few things I wish for you: 1. May you be able to work with your brother, Google Apps, to allow SSL support for custom domains. 2. May you be able to lift your 1 MB cap per urlfetch request to at least 10 MB so that your users can write some cool apps using the YouTube API. 3. May you inherit some snazzy search skills this year from Google Search, your grand daddy, and allow your users to whip up custom search engines for their apps in the blink of an eye (or the press of a button, rather). Happy Birthday GAE! - All of your 250k users.

simonk 17y ago
Top two are in the roadmap http://code.google.com/appengine/docs/roadmap.html http://code.google.com/appengine/docs/roadmap.html
mshafrir 17y ago
The first one is a non-starter for many startups.
markkoberlein 17y ago
I would agree that the lack of support of SSL on custom domains and the size limitations (my main issue is with the upload limits) have kept GAE from really taking off. I have used it for a small non-profit app and I host my personal consulting site on it. However, because of the limitations I'm building my startup on rackspace instead of GAE. I really love GAE as a development platform, especially as a single developer, but I hope they will work through their roadmap faster.

ez77 17y ago

I have the feeling this is an old question, but I can't find a specific answer. Is it possible to host a Google App only with your custom domain while disabling the appspot.com URL? I could understand this being a restriction on free accounts, but does it also apply to paying customers? Thanks PS: If not possible, is there a way to redirect your appspot.com URL to your custom URL?

ordinaryman 17y ago
Ideally, if you do not publicize the App ID / appspot domain, it is not likely to be known & requested. Anyway, the simplest way to do this will be to redirect requests received on the appspot domain to your custom domain, from within your code.
ez77 17y ago
I found a relevant discussion: http://stackoverflow.com/questions/1364733/block-requests-from-appspot-com-and-force-custom-domain-in-google-app-engine http://stackoverflow.com/questions/1364733/block-requests-fr...

apower 17y ago

Yawn.