4 ms·

But read it well. It seems AppEngine comes out very bad, but it was tested for db transaction speed in a way that every insert locks the entire db (one big enti

by koenbok 16y ago

But read it well. It seems AppEngine comes out very bad, but it was tested for db transaction speed in a way that every insert locks the entire db (one big entity group). Any AppEngine developer could tell you not to use it like this. More info: http://code.google.com/appengine/docs/python/datastore/keysandentitygroups.html http://code.google.com/appengine/docs/python/datastore/keysa... It would be very interesting to see what the results would be with some more realistic testing.

axiom 16y ago
Well, maybe you're right, but I'll say the following: 1. Unless I'm missing something here the entity group thing is a red herring. Only if the authors of the paper explicitly setup all entities in the datastore as having the same root entity, then this would be an issue. However, since there isn't anything to suggest that they did this, all datastore entities are their own groups (and hence don't lock up the entire db,) so transactions on them would not have had any speed penalty. Please someone correct me if I'm wrong here. 2. Unfortunately their results are consistent with my experience over the last ~8 months running a startup on the App Engine (and now switching over to EC2 over the next few weeks.)
bad_user 16y ago
> Any AppEngine developer could tell you not to use it like this I'd really wish for some AppEngine developer to tell me how's the proper way of designing entity groups. The only advice given is to not exceed a single user's data, otherwise it's too big. Really, I want to see an example of a request handler that creates data in multiple entities and takes care to not create duplicates. > It would be very interesting to see what the results would be with some more realistic testing. IMHO, my experience with AppEngine's datastore has been pretty bad ... the performance varies greatly based on factors you can't control ... i.e. processing something that does well on slow Fridays, may throw the 30 secs timeout error on Mondays. Also, I don't know how people deploy real-world apps on it ... a non-relational datastore requires custom indexes (i.e. precalculated queries done by building specialized entities out of your quasi-normalized data). You can't do that easily because of the 30 secs limit that also applies to cron or queue jobs. I tried a workaround by having a queue task that processes 5 rows at a time and then reschedules itself. The stuff was so god-damn awful to debug, and the results so unpredictable (30 secs timeout for 5 rows / concurrency issues) that I felt like crying. But yeah, it's fine for hosting a blog engine on top of it.
- axiom 16y ago
  Another issue we're running into now (as a result of having a growing number of users) is that we're noticing that downtime on the App Engine is shockingly bad, and getting worse. Over the last few weeks, we've seen periods of downtime pretty much daily, and serious downtime (i.e. more than a few minutes) monthly. It's grossly under-reported in the App Engine status console, which shows a euphemistic warning (which in practice might mean that all server requests will return errors.) Our software is used in the classroom during lectures, and we've had several customers experience system failure due to App Engine downtime. On top of that we've had more than one customer demo go wrong for the same reason. It's pretty infuriating actually, because 1. you can't do anything other than just sit there and wait it out, and 2. there is no one to talk to at Google. Finally, the problems are only getting worse. We were going to do a slow migration out of the App Engine over the course of a few months (because we've got a ton of code to rewrite.) At this point we're doing an emergency 2 week migration with all devs pulled from their projects for the task. I honestly have no idea why these issues aren't getting more mainstream attention, given how many people are using the App Engine. The only thing I can come up with is that there must be tons and tons of tiny non-critical projects, and very few serious startups betting their business on the App Engine.
  - hamstersoup 16y ago
    I was about to start a big project on App Engine, so your comment is worrying. Anybody else having reliability problems with GAE or is this just an isolated case?
    - axiom 16y ago
      If you're interested in more gory details feel free to email me at mike at tophatmonocle dot com. For what it's worth, we're an angel funded startup with some extremely bright hackers and we were signing praises about the App Engine until about 3 months in, when we started hitting it's various brick walls. Oh, and take a look at: http://code.google.com/status/appengine http://code.google.com/status/appengine Click through the various items with green checkmarks to see the actual data that GAE considers "no significant issues." Including fun stuff like 80% datastore error rates.

by koenbok 16y ago

axiom 16y ago

Well, maybe you're right, but I'll say the following: 1. Unless I'm missing something here the entity group thing is a red herring. Only if the authors of the paper explicitly setup all entities in the datastore as having the same root entity, then this would be an issue. However, since there isn't anything to suggest that they did this, all datastore entities are their own groups (and hence don't lock up the entire db,) so transactions on them would not have had any speed penalty. Please someone correct me if I'm wrong here. 2. Unfortunately their results are consistent with my experience over the last ~8 months running a startup on the App Engine (and now switching over to EC2 over the next few weeks.)

bad_user 16y ago

> Any AppEngine developer could tell you not to use it like this I'd really wish for some AppEngine developer to tell me how's the proper way of designing entity groups. The only advice given is to not exceed a single user's data, otherwise it's too big. Really, I want to see an example of a request handler that creates data in multiple entities and takes care to not create duplicates. > It would be very interesting to see what the results would be with some more realistic testing. IMHO, my experience with AppEngine's datastore has been pretty bad ... the performance varies greatly based on factors you can't control ... i.e. processing something that does well on slow Fridays, may throw the 30 secs timeout error on Mondays. Also, I don't know how people deploy real-world apps on it ... a non-relational datastore requires custom indexes (i.e. precalculated queries done by building specialized entities out of your quasi-normalized data). You can't do that easily because of the 30 secs limit that also applies to cron or queue jobs. I tried a workaround by having a queue task that processes 5 rows at a time and then reschedules itself. The stuff was so god-damn awful to debug, and the results so unpredictable (30 secs timeout for 5 rows / concurrency issues) that I felt like crying. But yeah, it's fine for hosting a blog engine on top of it.

axiom 16y ago
Another issue we're running into now (as a result of having a growing number of users) is that we're noticing that downtime on the App Engine is shockingly bad, and getting worse. Over the last few weeks, we've seen periods of downtime pretty much daily, and serious downtime (i.e. more than a few minutes) monthly. It's grossly under-reported in the App Engine status console, which shows a euphemistic warning (which in practice might mean that all server requests will return errors.) Our software is used in the classroom during lectures, and we've had several customers experience system failure due to App Engine downtime. On top of that we've had more than one customer demo go wrong for the same reason. It's pretty infuriating actually, because 1. you can't do anything other than just sit there and wait it out, and 2. there is no one to talk to at Google. Finally, the problems are only getting worse. We were going to do a slow migration out of the App Engine over the course of a few months (because we've got a ton of code to rewrite.) At this point we're doing an emergency 2 week migration with all devs pulled from their projects for the task. I honestly have no idea why these issues aren't getting more mainstream attention, given how many people are using the App Engine. The only thing I can come up with is that there must be tons and tons of tiny non-critical projects, and very few serious startups betting their business on the App Engine.
- hamstersoup 16y ago
  I was about to start a big project on App Engine, so your comment is worrying. Anybody else having reliability problems with GAE or is this just an isolated case?
  - axiom 16y ago
    If you're interested in more gory details feel free to email me at mike at tophatmonocle dot com. For what it's worth, we're an angel funded startup with some extremely bright hackers and we were signing praises about the App Engine until about 3 months in, when we started hitting it's various brick walls. Oh, and take a look at: http://code.google.com/status/appengine http://code.google.com/status/appengine Click through the various items with green checkmarks to see the actual data that GAE considers "no significant issues." Including fun stuff like 80% datastore error rates.