4 ms·

think of it as an integration server for a legacy API you are integrating with: data inside it isn’t going to go missing, but you should expect your connection

by trotsky 16y ago

think of it as an integration server for a legacy API you are integrating with: data inside it isn’t going to go missing, but you should expect your connection to it will break at any point. You need to isolate your users from it, protect you application from it and consider carefully how to protect your data from outages. I get that in terms of practical advice - but the biggest thing I've been wondering in this whole GAE debate is why you'd select a service where failure is such a norm. Is it solely a cost decision because GAE beats someone like AWS by such a large margin? Theoretical development costs seem like they'd be much lower on platforms where you expect lower error rates.

nl 16y ago
It depends. For some applications it's going to suck, and suck badly. OTOH, if you are primarily concerned with interacting with 3rd party HTTP based APIs, then you probably need to build that protection layer anyway. This is a scenario where AppEngine works really well - and once you've built that protection layer it isn't such a big deal to extend it to data access too.
fraserharris 16y ago
The single largest benefit of App Engine is that Google does the sysadmin. I would prefer that Google's highly paid, top 1%, employees are responsible for the servers than whomever I could afford to pay. ;-) If your application fits with the App Engine paradigm, then it can be a great and inexpensive platform.
- loewenskind 16y ago
  Where is this myth coming from that Google employees are highly paid? That's never been the case as far as I know. The number I'd always heard was 10-20% below market because "ZOMG, IT'S GOOGLE!". As far as top 1%, that's a meaningless metric and I've heard they don't even try for that anymore anyway (e.g. people complaining of a lot of "riffraff" coming in.
richardw 16y ago
For me, it's because they move the problems into a realm I feel competent in. If I fix those issues, I get a lot of stuff for free where I have zero competence, and don't care to learn about. Was it Foursquare who had the huge issue a month or so ago? They had many developers worrying about how to re-shard their data because suddenly it couldn't fit into memory. Keep that problem away from me...especially at 2am on a Saturday!
biot 16y ago
> Theoretical development costs seem like they'd be much lower on platforms where you expect lower error rates. Then again, that theoretical becomes quite practical when your app that runs on EC2 breaks down because it isn't as scalable as you thought it might be. Google has already thought about the tough questions like "what happens if an entire datacenter goes offline?" and GAE has that designed in. Developing for failure doesn't make sense when you're operating on a small scale, but it makes a lot of sense when you do need to scale. I'd rather take a bit of a development hit up front when there are few users than take the hit when a site really gains traction.
iampims 16y ago
Regardless of the platform you’re using, defensive programming is a good way to improve the quality of your software, deal with expected and unexpected failures. It is no different from sanitizing user inputs. You wouldn’t expect sane people to type alpha chars in a phone number field, yet they do. So you write code to deal with it. If you expect the datastore to fail every now and then, it helps wrapping it in an error handler. Should it be that way? Certainly not, but software breaks, whether it’s designed by Google or by anyone else, and it’s our job to make it as smooth as possible for our users. note: the datastore issues are not as common as they used to be a few months ago. And latency is low now.
anamax 16y ago
> Theoretical development costs seem like they'd be much lower on platforms where you expect lower error rates. What makes you think that GAE's error rates are higher than "the norm"? Better question - what makes you think that GAE's error rates are higher than what you can get/do yourself? (Yes, I'm sure that someone can beat GAE, but unless you're that person/organization....) Yes, GAE encourages you to program for error. Unless you've got an error free alternative, you're either programming for error or hoping for the best.

by trotsky 16y ago

nl 16y ago

It depends. For some applications it's going to suck, and suck badly. OTOH, if you are primarily concerned with interacting with 3rd party HTTP based APIs, then you probably need to build that protection layer anyway. This is a scenario where AppEngine works really well - and once you've built that protection layer it isn't such a big deal to extend it to data access too.

fraserharris 16y ago

The single largest benefit of App Engine is that Google does the sysadmin. I would prefer that Google's highly paid, top 1%, employees are responsible for the servers than whomever I could afford to pay. ;-) If your application fits with the App Engine paradigm, then it can be a great and inexpensive platform.

loewenskind 16y ago
Where is this myth coming from that Google employees are highly paid? That's never been the case as far as I know. The number I'd always heard was 10-20% below market because "ZOMG, IT'S GOOGLE!". As far as top 1%, that's a meaningless metric and I've heard they don't even try for that anymore anyway (e.g. people complaining of a lot of "riffraff" coming in.

richardw 16y ago

For me, it's because they move the problems into a realm I feel competent in. If I fix those issues, I get a lot of stuff for free where I have zero competence, and don't care to learn about. Was it Foursquare who had the huge issue a month or so ago? They had many developers worrying about how to re-shard their data because suddenly it couldn't fit into memory. Keep that problem away from me...especially at 2am on a Saturday!

biot 16y ago

> Theoretical development costs seem like they'd be much lower on platforms where you expect lower error rates. Then again, that theoretical becomes quite practical when your app that runs on EC2 breaks down because it isn't as scalable as you thought it might be. Google has already thought about the tough questions like "what happens if an entire datacenter goes offline?" and GAE has that designed in. Developing for failure doesn't make sense when you're operating on a small scale, but it makes a lot of sense when you do need to scale. I'd rather take a bit of a development hit up front when there are few users than take the hit when a site really gains traction.

iampims 16y ago

Regardless of the platform you’re using, defensive programming is a good way to improve the quality of your software, deal with expected and unexpected failures. It is no different from sanitizing user inputs. You wouldn’t expect sane people to type alpha chars in a phone number field, yet they do. So you write code to deal with it. If you expect the datastore to fail every now and then, it helps wrapping it in an error handler. Should it be that way? Certainly not, but software breaks, whether it’s designed by Google or by anyone else, and it’s our job to make it as smooth as possible for our users. note: the datastore issues are not as common as they used to be a few months ago. And latency is low now.

anamax 16y ago

> Theoretical development costs seem like they'd be much lower on platforms where you expect lower error rates. What makes you think that GAE's error rates are higher than "the norm"? Better question - what makes you think that GAE's error rates are higher than what you can get/do yourself? (Yes, I'm sure that someone can beat GAE, but unless you're that person/organization....) Yes, GAE encourages you to program for error. Unless you've got an error free alternative, you're either programming for error or hoping for the best.