6 ms·

App Engine JavaScript SDK

74 points by danh 16y ago

axiom 16y ago
If you're thinking of using the App Engine read this first: http://highscalability.com/blog/2010/5/26/end-to-end-performance-study-of-cloud-services.html http://highscalability.com/blog/2010/5/26/end-to-end-perform...
- koenbok 16y ago
  But read it well. It seems AppEngine comes out very bad, but it was tested for db transaction speed in a way that every insert locks the entire db (one big entity group). Any AppEngine developer could tell you not to use it like this. More info: http://code.google.com/appengine/docs/python/datastore/keysandentitygroups.html http://code.google.com/appengine/docs/python/datastore/keysa... It would be very interesting to see what the results would be with some more realistic testing.
  - axiom 16y ago
    Well, maybe you're right, but I'll say the following: 1. Unless I'm missing something here the entity group thing is a red herring. Only if the authors of the paper explicitly setup all entities in the datastore as having the same root entity, then this would be an issue. However, since there isn't anything to suggest that they did this, all datastore entities are their own groups (and hence don't lock up the entire db,) so transactions on them would not have had any speed penalty. Please someone correct me if I'm wrong here. 2. Unfortunately their results are consistent with my experience over the last ~8 months running a startup on the App Engine (and now switching over to EC2 over the next few weeks.)
  - bad_user 16y ago
    > Any AppEngine developer could tell you not to use it like this I'd really wish for some AppEngine developer to tell me how's the proper way of designing entity groups. The only advice given is to not exceed a single user's data, otherwise it's too big. Really, I want to see an example of a request handler that creates data in multiple entities and takes care to not create duplicates. > It would be very interesting to see what the results would be with some more realistic testing. IMHO, my experience with AppEngine's datastore has been pretty bad ... the performance varies greatly based on factors you can't control ... i.e. processing something that does well on slow Fridays, may throw the 30 secs timeout error on Mondays. Also, I don't know how people deploy real-world apps on it ... a non-relational datastore requires custom indexes (i.e. precalculated queries done by building specialized entities out of your quasi-normalized data). You can't do that easily because of the 30 secs limit that also applies to cron or queue jobs. I tried a workaround by having a queue task that processes 5 rows at a time and then reschedules itself. The stuff was so god-damn awful to debug, and the results so unpredictable (30 secs timeout for 5 rows / concurrency issues) that I felt like crying. But yeah, it's fine for hosting a blog engine on top of it.
    - axiom 16y ago
      Another issue we're running into now (as a result of having a growing number of users) is that we're noticing that downtime on the App Engine is shockingly bad, and getting worse. Over the last few weeks, we've seen periods of downtime pretty much daily, and serious downtime (i.e. more than a few minutes) monthly. It's grossly under-reported in the App Engine status console, which shows a euphemistic warning (which in practice might mean that all server requests will return errors.) Our software is used in the classroom during lectures, and we've had several customers experience system failure due to App Engine downtime. On top of that we've had more than one customer demo go wrong for the same reason. It's pretty infuriating actually, because 1. you can't do anything other than just sit there and wait it out, and 2. there is no one to talk to at Google. Finally, the problems are only getting worse. We were going to do a slow migration out of the App Engine over the course of a few months (because we've got a ton of code to rewrite.) At this point we're doing an emergency 2 week migration with all devs pulled from their projects for the task. I honestly have no idea why these issues aren't getting more mainstream attention, given how many people are using the App Engine. The only thing I can come up with is that there must be tons and tons of tiny non-critical projects, and very few serious startups betting their business on the App Engine.
      - hamstersoup 16y ago
        I was about to start a big project on App Engine, so your comment is worrying. Anybody else having reliability problems with GAE or is this just an isolated case?
        axiom 16y ago
        If you're interested in more gory details feel free to email me at mike at tophatmonocle dot com. For what it's worth, we're an angel funded startup with some extremely bright hackers and we were signing praises about the App Engine until about 3 months in, when we started hitting it's various brick walls. Oh, and take a look at: http://code.google.com/status/appengine http://code.google.com/status/appengine Click through the various items with green checkmarks to see the actual data that GAE considers "no significant issues." Including fun stuff like 80% datastore error rates.
- gmosx 16y ago
  You can't just port your application to App Engine. You have to start from scratch. Google enforces all the right constrains to force you to write scalable apps. The killer feature is that there is no need for system administration or maintenance. No need to setup dns rules to make your email go to the recipients, no need to update apache or linux, no need to setup an xmpp serrver, etc. Plus, you can leverage Google's battle tested infrastructure. The icing of the cake is that you can now use the internet programming language, ie JavaScript, while still having access to the vast collection of Java libraries. A match made in heaven if you ask me!
pmjordan 16y ago
I'm always amazed at how popular Rhino is vs. how little active development effort it receives. I'm as guilty as anyone of course.
- gmosx 16y ago
  Indeed, Rhino is such an important project but still it does not receive any attention.
  - boucher 16y ago
    It's unfortunate, but it's also the reason we wrote Narwhal, which now means we rarely need to use Rhino at all. A welcome change.
    - past 16y ago
      Erm, narwhal actually uses rhino as its default engine. Perhaps you meant to say that you can now target the narwhal API, instead of waiting for rhino to expand its API offerings?
      - boucher 16y ago
        No, I meant what I said. We use narwhal-jsc 95% of the time. Rhino is this unfortunate thing we keep around for compatibilities sake. The flexibility of being able to switch engines as needed is also a major win in Narwhal. I suspect in the next year we'll end up moving everything to v8, which should prove to be extremely easy to do.
        gmosx 16y ago
        no prob, thanks to CommonJS it was extremely easy to switch from Narwhal to the RingoJS: www.ringojs.org
    - pmjordan 16y ago
      The killer feature of Rhino for me is the fact that it runs on the JVM, and customers are OK with me using JavaScript, whereas they're not OK with Clojure or so. Too bad the Java integration in Rhino isn't exactly great. (example: you can only use the 0-argument constructor when deriving from a class)
agentultra 16y ago
Cool project. I'll never understand why anyone wants to program in JS though.
- past 16y ago
  Besides being an incredibly powerful combination of a functional and an object-oriented language, I just adore the fast redeployment cycle during development. With appenginejs, I test every change in the backend code by simply refreshing my browser. Not jaw-droppping per se, but certainly a breath of fresh air for a long-time "enterprise" Java developer, like me.
  - sreque 16y ago
    Javascript's functional powers are equivalent to or worse than every other popular dynamic language in existence. Speaking of functions, I prefer not to do argument arity checking on every single function I write(I'm looking at you too, perl!), or accidentally assigning global variables through a simple misspelling. I also really hate it if I accidentally invoke a function on a primitive and it silenty fails by returning null. Use 'use strict', you say? How's the implementation of that feature coming these days? I know firefox still doesn't have it. Does v8 yet? From what I've read, other Prototype-based languages treat Javascript as their ugly step cousin, and I would overall consider it a much inferior OO system to that found in Python, Ruby, CLOS, Perl's Moose, and PLT-Scheme, to name a few. You can easily get rapid redeployment in any dynamic language. You can even get it on the JVM with Java, as the Play framework is showing us. See http://www.playframework.org/ http://www.playframework.org/. If Javascript is your first dynamic language after developing in Java for X years you might think it is neat, but it is really far behind just about every other language out there it directly competes with on the server-side level. It was never designed to be a scalable general purpose programming language and the latest incarnation is still far from reaching that lofty goal.
- artlogic 16y ago
  I used to feel this way. The two people that changed my mind were John Resig and Douglas Crockford. Resig's jQuery project has been, for me, a guide to how JS should be written. Crockford's JS the Good Parts and JSLint have helped as well by providing some solid reference on what to do and, more importantly, what not to do. Lots of folks only see the DOM and the bad parts when working with JS - these two projects have shown me there's so much more.
AlexisT 16y ago
Great stuff!
gmosx 16y ago
some more thoughts on appengineJS here: http://www.gmosx.com/blog/agVnbW9zeHIPCxIHQXJ0aWNsZRipwwEM/on-appenginejs http://www.gmosx.com/blog/agVnbW9zeHIPCxIHQXJ0aWNsZRipwwEM/o...

74 points by danh 16y ago

axiom 16y ago

If you're thinking of using the App Engine read this first: http://highscalability.com/blog/2010/5/26/end-to-end-performance-study-of-cloud-services.html http://highscalability.com/blog/2010/5/26/end-to-end-perform...

koenbok 16y ago
But read it well. It seems AppEngine comes out very bad, but it was tested for db transaction speed in a way that every insert locks the entire db (one big entity group). Any AppEngine developer could tell you not to use it like this. More info: http://code.google.com/appengine/docs/python/datastore/keysandentitygroups.html http://code.google.com/appengine/docs/python/datastore/keysa... It would be very interesting to see what the results would be with some more realistic testing.
- axiom 16y ago
  Well, maybe you're right, but I'll say the following: 1. Unless I'm missing something here the entity group thing is a red herring. Only if the authors of the paper explicitly setup all entities in the datastore as having the same root entity, then this would be an issue. However, since there isn't anything to suggest that they did this, all datastore entities are their own groups (and hence don't lock up the entire db,) so transactions on them would not have had any speed penalty. Please someone correct me if I'm wrong here. 2. Unfortunately their results are consistent with my experience over the last ~8 months running a startup on the App Engine (and now switching over to EC2 over the next few weeks.)
- bad_user 16y ago
  > Any AppEngine developer could tell you not to use it like this I'd really wish for some AppEngine developer to tell me how's the proper way of designing entity groups. The only advice given is to not exceed a single user's data, otherwise it's too big. Really, I want to see an example of a request handler that creates data in multiple entities and takes care to not create duplicates. > It would be very interesting to see what the results would be with some more realistic testing. IMHO, my experience with AppEngine's datastore has been pretty bad ... the performance varies greatly based on factors you can't control ... i.e. processing something that does well on slow Fridays, may throw the 30 secs timeout error on Mondays. Also, I don't know how people deploy real-world apps on it ... a non-relational datastore requires custom indexes (i.e. precalculated queries done by building specialized entities out of your quasi-normalized data). You can't do that easily because of the 30 secs limit that also applies to cron or queue jobs. I tried a workaround by having a queue task that processes 5 rows at a time and then reschedules itself. The stuff was so god-damn awful to debug, and the results so unpredictable (30 secs timeout for 5 rows / concurrency issues) that I felt like crying. But yeah, it's fine for hosting a blog engine on top of it.
  - axiom 16y ago
    Another issue we're running into now (as a result of having a growing number of users) is that we're noticing that downtime on the App Engine is shockingly bad, and getting worse. Over the last few weeks, we've seen periods of downtime pretty much daily, and serious downtime (i.e. more than a few minutes) monthly. It's grossly under-reported in the App Engine status console, which shows a euphemistic warning (which in practice might mean that all server requests will return errors.) Our software is used in the classroom during lectures, and we've had several customers experience system failure due to App Engine downtime. On top of that we've had more than one customer demo go wrong for the same reason. It's pretty infuriating actually, because 1. you can't do anything other than just sit there and wait it out, and 2. there is no one to talk to at Google. Finally, the problems are only getting worse. We were going to do a slow migration out of the App Engine over the course of a few months (because we've got a ton of code to rewrite.) At this point we're doing an emergency 2 week migration with all devs pulled from their projects for the task. I honestly have no idea why these issues aren't getting more mainstream attention, given how many people are using the App Engine. The only thing I can come up with is that there must be tons and tons of tiny non-critical projects, and very few serious startups betting their business on the App Engine.
    - hamstersoup 16y ago
      I was about to start a big project on App Engine, so your comment is worrying. Anybody else having reliability problems with GAE or is this just an isolated case?
      - axiom 16y ago
        If you're interested in more gory details feel free to email me at mike at tophatmonocle dot com. For what it's worth, we're an angel funded startup with some extremely bright hackers and we were signing praises about the App Engine until about 3 months in, when we started hitting it's various brick walls. Oh, and take a look at: http://code.google.com/status/appengine http://code.google.com/status/appengine Click through the various items with green checkmarks to see the actual data that GAE considers "no significant issues." Including fun stuff like 80% datastore error rates.
gmosx 16y ago
You can't just port your application to App Engine. You have to start from scratch. Google enforces all the right constrains to force you to write scalable apps. The killer feature is that there is no need for system administration or maintenance. No need to setup dns rules to make your email go to the recipients, no need to update apache or linux, no need to setup an xmpp serrver, etc. Plus, you can leverage Google's battle tested infrastructure. The icing of the cake is that you can now use the internet programming language, ie JavaScript, while still having access to the vast collection of Java libraries. A match made in heaven if you ask me!

pmjordan 16y ago

I'm always amazed at how popular Rhino is vs. how little active development effort it receives. I'm as guilty as anyone of course.

gmosx 16y ago
Indeed, Rhino is such an important project but still it does not receive any attention.
- boucher 16y ago
  It's unfortunate, but it's also the reason we wrote Narwhal, which now means we rarely need to use Rhino at all. A welcome change.
  - past 16y ago
    Erm, narwhal actually uses rhino as its default engine. Perhaps you meant to say that you can now target the narwhal API, instead of waiting for rhino to expand its API offerings?
    - boucher 16y ago
      No, I meant what I said. We use narwhal-jsc 95% of the time. Rhino is this unfortunate thing we keep around for compatibilities sake. The flexibility of being able to switch engines as needed is also a major win in Narwhal. I suspect in the next year we'll end up moving everything to v8, which should prove to be extremely easy to do.
      - gmosx 16y ago
        no prob, thanks to CommonJS it was extremely easy to switch from Narwhal to the RingoJS: www.ringojs.org
  - pmjordan 16y ago
    The killer feature of Rhino for me is the fact that it runs on the JVM, and customers are OK with me using JavaScript, whereas they're not OK with Clojure or so. Too bad the Java integration in Rhino isn't exactly great. (example: you can only use the 0-argument constructor when deriving from a class)

agentultra 16y ago

Cool project. I'll never understand why anyone wants to program in JS though.

past 16y ago
Besides being an incredibly powerful combination of a functional and an object-oriented language, I just adore the fast redeployment cycle during development. With appenginejs, I test every change in the backend code by simply refreshing my browser. Not jaw-droppping per se, but certainly a breath of fresh air for a long-time "enterprise" Java developer, like me.
- sreque 16y ago
  Javascript's functional powers are equivalent to or worse than every other popular dynamic language in existence. Speaking of functions, I prefer not to do argument arity checking on every single function I write(I'm looking at you too, perl!), or accidentally assigning global variables through a simple misspelling. I also really hate it if I accidentally invoke a function on a primitive and it silenty fails by returning null. Use 'use strict', you say? How's the implementation of that feature coming these days? I know firefox still doesn't have it. Does v8 yet? From what I've read, other Prototype-based languages treat Javascript as their ugly step cousin, and I would overall consider it a much inferior OO system to that found in Python, Ruby, CLOS, Perl's Moose, and PLT-Scheme, to name a few. You can easily get rapid redeployment in any dynamic language. You can even get it on the JVM with Java, as the Play framework is showing us. See http://www.playframework.org/ http://www.playframework.org/. If Javascript is your first dynamic language after developing in Java for X years you might think it is neat, but it is really far behind just about every other language out there it directly competes with on the server-side level. It was never designed to be a scalable general purpose programming language and the latest incarnation is still far from reaching that lofty goal.
artlogic 16y ago
I used to feel this way. The two people that changed my mind were John Resig and Douglas Crockford. Resig's jQuery project has been, for me, a guide to how JS should be written. Crockford's JS the Good Parts and JSLint have helped as well by providing some solid reference on what to do and, more importantly, what not to do. Lots of folks only see the DOM and the bad parts when working with JS - these two projects have shown me there's so much more.

AlexisT 16y ago

Great stuff!

gmosx 16y ago

some more thoughts on appengineJS here: http://www.gmosx.com/blog/agVnbW9zeHIPCxIHQXJ0aWNsZRipwwEM/on-appenginejs http://www.gmosx.com/blog/agVnbW9zeHIPCxIHQXJ0aWNsZRipwwEM/o...