11 ms·

Node.js on Google App Engine Goes Beta

371 points by boulos 11y ago

Strom 11y ago
This seems to be for Managed VM [1] only, so it's a case of better documentation & tooling, but no new functionality. You can already run arbitrary code of any language on Managed VM instances. [1] Managed VM is a sort of meet-half-way service from Google that is quite freeform like Compute Engine, but also offers access to a good chunk of App Engine's services. Still, it's very much not the standard App Engine.
- untog 11y ago
  That's disappointing. I was hoping this might be Google's answer to Amazon Lambda.
  - sehr 11y ago
    Google Cloud Functions are in alpha right now, pretty similar https://cloud.google.com/functions/docs https://cloud.google.com/functions/docs
- noiv 11y ago
  So, no SDK for node.js?
  - justinblat 11y ago
    Of course there's an SDK! It's here: https://github.com/GoogleCloudPlatform/gcloud-node https://github.com/GoogleCloudPlatform/gcloud-node
    - noiv 11y ago
      Well yes, just not an App Engine SDK :(
      - justinblat 11y ago
        What specifically are you looking to do?
      - deleted 11y ago
        [deleted]
- plexicle 11y ago
  Oh, are you kidding me? Man I got so excited when I saw the title, I already spammed the link to a couple buddies. I should have read it first. I want real GAE. Not managed VMs. I want to be able to spin up a quick GAE version on a deploy, and if it doesn't get the traffic to scale up, then it just sits there and doesn't cost me anything. Managed VM has 2+ instances running all month and is an incredible PITA to spin down if I need to temporarily. (The current workaround is to deploy a non-VM app and set that as default, which is just insane.) GAE apps based on Managed VMs need a free quota as well. I have many apps but some of them don't do much most of the time. A couple will spike hard randomly. I don't mind paying when it does. I like the security that if/when something really takes off, GAE has my back. In the meantime, I'll stick to Go on GAE. I'd love to be able to use Node, but things add up quickly if I have a couple hobby apps. I'm glad that they are pushing Node and I'm a HUGE fan of the Google Cloud. I just really want Node as a native GAE thing.
  - thesandlord 11y ago
    There is a easier way to spin them down now: gcloud preview app versions stop main
    - plexicle 11y ago
      Well, that's good at least. Must have been very recent, I was talking to support only a couple weeks ago.
      - thesandlord 11y ago
        Yeah super recent. It was one of my huge pet peeves with Managed VMs, so I'm glad it's fixed.
      - justinblat 11y ago
        I think technically that command is still hidden lol. You can run it, it just doesn't show up in the docs. We'll be officially releasing that one in a few weeks.
- BobTheCoder 11y ago
  I don't mind managed vms, I just wish they had the same pricing model as app engine.
ionwake 11y ago
All I know about GAE is that when my python app started being used by HN users I upgraded my account to a paid one to increase performance. In the transition , without any warning, google took the app down, for a day. Thus killing all chance of acquiring a userbase now that I had hit the frontpage. When I went onto the IRC GAE channel to ask someone for help to unlock or keep the app running during this important moment the reply from a staff member was ' You should have planned better'. I was surprised to hear this response and did not pursue the matter further. Its all in the IRC logs if any of this sounds doubtful. TL;DR; Surprisingly, I did not find the GAE platform to be a supportive environment for startups.
- boulos 11y ago
  That's awful, I'm sorry you experienced that. It's no use to you now, but if you had gotten that message to me, I would have walked over to SRE / Support and we could have reset your quotas (I assume that's what you mean by "Took the app down for a day"). Disclaimer: I work on Compute Engine not on App Engine.
  - michaelbuckbee 11y ago
    I appreciate that you'd be willing to do that, but the fact that it's necessary to get a human (even a technical and helpful one such as yourself) involved to deal with this is a big red flag.
    - boulos 11y ago
      To be clear, the person he reached should have reset / gotten the quota reset (if that's the root cause). There's often a human needed when something goes sideways, which is why we have paid support for Cloud - cloud.google.com/support. The issue is that he reached a human, and it didn't fix his problem. So like I said, sorry! (but I think it's okay to rely on humans fixing things like this)
      - ionwake2 11y ago
ungzd 11y ago
Quirkiness of Javascript + extreme weirdness of App Engine (especially datastore) = best of both worlds.
- boulos 11y ago
  Aww, I like Datastore. But, there's no requirement that you use Datastore. The point of App Engine is that it's "you just give us code, we run it for you". With Managed VMs (built on top of Compute Engine) we're able to offer that programming model with a more flexible environment. As someone mentioned elsewhere, there's even "just hand us a random docker container". We've had people run ffmpeg because you can just hook up App Engine's request/response and autoscaling to an arbitrary thing, and you've got an autoscaling web service. tl;dr: Don't get hung up on the old "extreme weirdness of App Engine", this is different.
  - deleted 11y ago
    [deleted]
smegel 11y ago
Interesting things are happening to GAE these days. They are moving towards custom runtimes using docker that the user can configure...even their datastore API is being replaced by something resembling S3. GAE -> Google Cloud Platform.
- thesandlord 11y ago
  Cloud Datastore is more like DynamoDB, not S3. Cloud Storage is like S3. Yay names! Custom runtimes are trying to give people the "No Ops" of App Engine with the flexibility of Compute Engine. Like always, there are tradeoffs, but more options is usually better :) (I work for Google Cloud Platform)
  - TheLogothete 11y ago
    Do you work on the database products? Please, PLEASE collaborate more with the analytics team. We want export to BQ for regular accounts. It's not a big ask and it will improve the lives and businesses of MANY people. I think it will affect google cloud revenues quite positively too.
cdnsteve 11y ago
What version of node is supported? Currently in all other languages they are usually a major version behind (in everything).
- justinblat 11y ago
  You can specify whatever version of Node.js you want to use in the engines section of your package.json: https://cloud.google.com/nodejs/resources/runtime#engines https://cloud.google.com/nodejs/resources/runtime#engines
  - BinaryIdiot 11y ago
    Frustrating that it doesn't specify how it gets those versions. Is it always direct from the node folks so it's going to always have all versions or do they have their own cache?
    - diggan 11y ago
      Reading the source code of the docker container they use (https://github.com/GoogleCloudPlatform/nodejs-docker/blob/master/base/install_node#L34-L44 https://github.com/GoogleCloudPlatform/nodejs-docker/blob/ma...) it seems like it's reading the available versions from an URL (http://storage.googleapis.com/gae_node_packages/node_versions http://storage.googleapis.com/gae_node_packages/node_version...) which points to some Google hosted storage. So I'm guessing they have some process to update this list after a new release and also caches the versions themselves.
      - dlor 11y ago
        Yep, we update the available versions constantly and just serve them from GCS to reduce download time. I'll get this documented somewhere. Disclosure: I work on GCP.
        BinaryIdiot 11y ago
        Fantastic, thanks!
cdnsteve 11y ago
What version of node is supported? Currently in all other languages they are usually a major version behind (in everything).
jorgecurio 11y ago
is anyone using node.js in enterprise? because everyone seems to be convinced that the enterprise companies are going to uproot all the network of partners, developers, vendors, and billions of dollars of investment made over the past decade? Like it or not, JVM pretty much rules the enterprise space and there's just no way to do it another way. Literally, you will be turned away because you told the IT department they should write everything in javascript and node.js. Why? Because everyone is doing it. Not good enough. I don't know what have your real world experience been? I'm learning Java because outside of rinky dink consumer and 19.99 / month productivity apps for SME, it's viewed in a completely different lime light. Honestly, the shortcomings of Java is greatly exaggerated vs how much of it is dominating the space, and it irks me when a 19 year old CTO on the autistic spectrum claims suddenly node.js is the way to build the world's infrastructure on. javascript is great for async programming but much work has been done to make it feel procedural. My feeling is that node.js is a hard sell for enterprises already deeply entrenched with JVM. Unless you offered the migration at a cost and undercut everyone. Not a great business imho where lower price is the only thing going for you because it never lasts. Eventually, you open source and become a grass eating goat relying on training, certification, support licenses.
- adrianpike 11y ago
  Yes, we are, but caveat; it's only for a few smaller services where the functionality makes lots of sense to be heavily asynchronous, and we're not leveraging many external dependencies. FWIW, deployment and packaging is still a painful issue for node - I've been bitten in the past by people not semver'ing their modules in a non-breaking way.
  - jorgecurio 11y ago
    thanks I think that is a more realistic real world scenario, some people on HN thinks they can change the enterprise world but it's super hard to do it with javascript imho and evident by that it's not being used in wide deployment.... Java is still king in the enterprise world.
- TheLogothete 11y ago
mattbillenstein 11y ago
Don't use AppEngine. I migrated Homejoy off of AppEngine in 2014 and I can't recommend the platform for any reason whatsoever. It's slow, expensive, not based on OSS, and not particularly stable. Bits of it (urlfetch, search, etc) would go down for hours at a time and you're basically hosed until they fix it. I think Heroku is usable with a similar level of effort -- and you have a much easier migration path to something else if you're not using Google's proprietary datastore IMO.
- justinblat 11y ago
  Apologies you had a bad experience. There's no doubt the original App Engine had some... quirks. You may want to take a look at Managed VMs. All of the new runtimes are based on Docker images, are entirely extensible, and don't require the use of the original App Engine APIs: https://cloud.google.com/appengine/docs/managed-vms/ https://cloud.google.com/appengine/docs/managed-vms/ You can use whatever Database or NPM modules you want.
  - edwinnathaniel 11y ago
    Hi Justin, I'm currently trying to get my feet wet on GAE. Judging from your statement, are you suggesting that people should look into Managed VMs and maybe skip the regular GAE (or known as the original GAE)? Perhaps the original GAE will be deprecated in the future?
    - boulos 11y ago
      No, that's not the case. Managed VMs is a more flexible environment than our "GAE Standard" environment, but (at least currently) comes with some of the tradeoffs of a full VM (startup time, minimum size, etc.). If you've got say a vanilla python app that doesn't need custom native modules, GAE Standard is great! If you want to run something we don't currently support, then Managed VMs gives you the flexibility a Compute Engine VM can offer you (run whatever you like). Disclaimer: I work on Compute Engine, not App Engine but I'm familiar enough with their thinking.
itaysk 11y ago
I'm surprised they didn't have it until now...
- joeblau 11y ago
  It's not really that surprising. They started with Python because the inventor of Python worked there. Then they added Java because... well it's Java. Then they added GoLang because they invented it. Then they added PHP because most of the internet still runs on that. So Node is the next logical step... or Ruby probably would have worked as well.
  - aleaxit 11y ago
    Actually, when we started (GAE) with Python, my friend Guido didn't YET work here -- I was finally able to convince him to join, shortly after GAE's launch, in part thanks to the prospect of working on GAE's Python (which is what he ended up doing here, especially developing ndb, but also many other little things). Python was at the heart of Google for long before that (read Levy's "In the 'plex"...!-) -- or I wouldn't have joined Google 11 years ago, BTW. As for Java, well -- guess who announced Java to the world at Moscone Center in SF 21 years ago...? One Eric Schmidt, then Sun's VP for SW products (he reminisced about that this morning at his Next keynote)... who just happened to be Google's CEO when Java was announced as GAE's 2nd language!
esilverberg2 11y ago
I wrote about my (unsuccessful) app engine experience many years ago...hopefully they've improved since then. http://www-cs-students.stanford.edu/~silver/gae.html http://www-cs-students.stanford.edu/~silver/gae.html
- boulos 11y ago
  Good writeup! I think the main change is that App Engine has really morphed into a "You know, you don't have to use Datastore". Cloud SQL was added precisely because of the issues you raise: being on App Engine meant being in a totally foreign world, and not being able to get out easily. If you were to try again today, you'd take stock Django 1.9 via pip install Django -t lib and using vendoring: https://cloud.google.com/appengine/docs/python/tools/libraries27#vendoring https://cloud.google.com/appengine/docs/python/tools/librari... . You point it at your Cloud SQL instance in your DATABASES line (https://cloud.google.com/appengine/docs/python/cloud-sql/django#usage https://cloud.google.com/appengine/docs/python/cloud-sql/dja...) and you're done. tl;dr: Yep! Things have improved in the last 5 years. Disclosure: I work on Compute Engine.
  - sspross 11y ago
    It sounds like you know what you're talking about. Would you mind to take a look at my issue/question on the appengine django skeleton[1]. The Problems start after you've pointed your database to cloud sql und start using GAE specific features like task queue etc. [1] https://github.com/GoogleCloudPlatform/appengine-django-skeleton/issues/14#issuecomment-196490921 https://github.com/GoogleCloudPlatform/appengine-django-skel...
    - boulos 11y ago
      For your first question (playing with Cloud SQL) you've got two options. Directly interact with Cloud SQL from your laptop or like you said have a "development MySQL" separate from prod / staging. Pointing at your Cloud SQL instance (maybe using a staging or development database) can be done in settings.py. The switch between modes let's you specify the IP address for your Cloud SQL instance when you're not on App Engine (which uses a special UNIX domain socket). We also recommend you connect over SSL when doing so outside (see more at the link I provided, or search for cloud sql ssl). Having a local environment for MySQL is also easy enough depending on your box, but has the usual difficulty of needing to have a good test suite / being populated with useful data. I think it's worthwhile if you can pull it off, but I've seen companies big and small decide its not worth it (and instead use a separate table or database on the same server, maybe as a different user as another line of "don't screw up prod"). Finally, the interaction between Django's development server and the App Engine one isn't great. My response was intended to remind people that if you're not looking to use App Engine specific APIs you really can just get going and almost not notice. But you should be able to run your migrations speaking to Cloud SQL remotely without needing to import say Task Queues. So my workflow was to use manage.py for things like that, while using the dev_appserver when I actually needed to emulate App Engine (as opposed to the Django runserver command). Does that help? Edit: I forgot to add that we're working on decoupling the "emulation" of our services so that this kind of interop works better. See https://cloud.google.com/sdk/gcloud/reference/beta/emulators/ https://cloud.google.com/sdk/gcloud/reference/beta/emulators... for Datastore and PubSub (and more on the way).
shubb 11y ago
I used google app engine for a project with very specific requirements - the client wanted an application for internal use that didn't have any ongoing costs (sign off) and would look after itself for a couple of years. The platform has a lot of quirks, and I had to do a lot more learning than I expected to get the fairly simple project done. But once it was up, it did it's thing. I'd definitely recommend for free hosting internal applications and personal projects that are just for you to use. Just understand that you'll need to shape your application around the wierdnesses of the platform.
justinblat 11y ago
Greetings folks, I'm the PM for Node.js on App Engine. If anyone has any questions, feel free to ask me anything!
- tomh- 11y ago
  I have one, how can European individuals use the paid version of any Google cloud services? Last time I checked only commercial entities were allowed to use the paid version.
  - boulos 11y ago
    Warning: IANAL. First, sorry for the pain! As an explanation, it's all about VAT (determining it, collecting it, etc.). Outside of Ireland we dump it on you: https://support.google.com/cloud/answer/6090602 https://support.google.com/cloud/answer/6090602 But specifically: > “Business” status means that you'd like to see a potential economic benefit from your development activities, for example: using the Google Cloud Platform to develop prototypes or applications with a view to generating revenue in the future. Most software developers -- including affiliates, sole traders, self-employed merchants, partnerships, students and others -- use Google Cloud Platform for business purposes. I can't help you decide what your VAT on the use of our services would be (this is the realm of accounts and lawyers). I know AWS makes a different decision than we do, and we're very aware of the friction (again, sorry!). Disclaimer: I work on Compute Engine, but again IANAL.
    - tomh- 11y ago
      Thanks for your answer, although its worrying we need to have a lawyer around to see how we can signup without accidentally violating my local country's VAT regulations as a developer who just wants to pay and build stuff :(
- ralusek 11y ago
  This appears to still just be a Managed VM, not really App Engine. What is new? This has been around for a while... Is there any plan to implement Node fully on GAE?
davidjgraph 11y ago
So many negative views of GAE, I have to add our, overwhelmingly positive, experience. We were paying $200/month for a load balanced EC2 setup in 2011, that's for around 2k unique daily users. The load balancer kept on false triggering new instances, so we went to GAE in 2012. Since then our uptime has been horrifically good, heavily thanks for GAE, trashing that of our competition, who mostly use AWS. Last month we paid $150 for everything on GAE, traffic, CPU, etc. G-Analytics shows a peak of 54k daily uniques last week. All the scaling is dealt with for us, we can handle literally any traffic peak. i.e. x27 more traffic, lower cost, better uptime, spending virtual zero time looking at the operations side of things. This is a mission critical app, although, we've never invoked support, so I can't comment on that side.
- stephenitis 11y ago
  did software change to account for efficiencies?
  - davidjgraph 11y ago
    No, the basic architecture hasn't changed, but the app does a lot, lot more than it did in the AWS days.
- latchkey 11y ago
  Without GAE/GCP my business wouldn't be where it is today and in such a short amount of time. I can't reveal numbers, but our bill and usage is significantly higher than yours. But, it is still just a rounding error compared with our revenue. Thanks to GAE, we focus our work on real features for our partners, which translates directly into profit. A post on this topic that I made a month ago: https://news.ycombinator.com/item?id=11159840 https://news.ycombinator.com/item?id=11159840 Thanks again Google! =)
- PostThisTooFast 11y ago
  Amazon is a mess. But should we care if Node is on Google? What about this: https://news.ycombinator.com/item?id=4495101 https://news.ycombinator.com/item?id=4495101
TheAceOfHearts 11y ago
So, considering that it's beta, where should I expect rough edges? Is there any list that shows what's missing to go from beta to full production? I'm actually really interested in trying this out, but I want to know what potential issues I should expect to deal with. Something I found really awesome about Google's services is that they provide fake local implementations that you can run from the CLI. This means that you can run tests without having to hit the outside world! Somewhat related to that, if you're using S3, check out s3rver [0]. It's a node implementation of fake-s3 [1], which is a local implementation of S3 that you can use during development and testing. Looking through the examples, I'm confused about the deployment strategy for SPAs. Looking at the webpack example [2], you build the frontend app every time the server runs? That's REALLY slow for any reasonably sized SPA :(. It means you generate a new build each time the server restarts, and for something like node that's really common. (AFAIK, the suggested strategy is to just let it crash and restart quickly.) But it also means that you might have multiple servers running different version of your dependencies (since there's no shrinkwrap). Even worse, if you're running multiple servers with slightly different dependencies, it might also mean that you'll end up building slightly different versions of your SPA in every server for any given release! I'd love to see an example where you build and tag the SPA in a single place, and you make sure all the servers are serving the same bundled code. [0] https://github.com/jamhall/s3rver https://github.com/jamhall/s3rver [1] https://github.com/jubos/fake-s3 https://github.com/jubos/fake-s3 [2] https://github.com/GoogleCloudPlatform/nodejs-docs-samples/blob/master/appengine/webpack/package.json https://github.com/GoogleCloudPlatform/nodejs-docs-samples/b...
- eknkc 11y ago
  This is my understanding about Google Cloud services; - Alpha means sub par quality / rough edges / unresolved issues. - Beta means fairly stable but no SLA, no guarantees from Google. So I'd guess it's fine to use now.
  - Xorlev 11y ago
    With Google it's either beta or deprecated.
- 11y ago
exotiik 11y ago
and make sure billing is enabled !
outside1234 11y ago
Well that took forever. Congrats on getting to the right place though Google.
mastazi 11y ago
Wow, I thought that GAE was basically dead and now I find out that, besides the latest addition of Node.js, it supports Python 3, Java 8 and even Docker-based custom runtimes...
- ekso 11y ago
  Why did you think GAE was basically dead?
vskarine 11y ago
Has anyone tried running MeteorJS on GAE?
fibo 11y ago
Great news!
lambdacomplete 11y ago
The only reason why I'm still on AWS: I don't own a credit card.
atonparker 11y ago
We've tried node.js as a managed VM a few times before this. One of the biggest pains is logging. Despite documentation implying that JSON logs are supported (can't find this anymore, maybe it was removed), we still haven't been able to get GCP to read JSON logs. You lose out on all of the nice filtering and request grouping provided by the GCP log viewer. The issue is here[0] and it would seem they still haven't addressed it a year later. [0] https://code.google.com/p/googleappengine/issues/detail?id=11678#c5 https://code.google.com/p/googleappengine/issues/detail?id=1...

371 points by boulos 11y ago

Strom 11y ago

This seems to be for Managed VM [1] only, so it's a case of better documentation & tooling, but no new functionality. You can already run arbitrary code of any language on Managed VM instances. [1] Managed VM is a sort of meet-half-way service from Google that is quite freeform like Compute Engine, but also offers access to a good chunk of App Engine's services. Still, it's very much not the standard App Engine.

untog 11y ago
That's disappointing. I was hoping this might be Google's answer to Amazon Lambda.
- sehr 11y ago
  Google Cloud Functions are in alpha right now, pretty similar https://cloud.google.com/functions/docs https://cloud.google.com/functions/docs
noiv 11y ago
So, no SDK for node.js?
- justinblat 11y ago
  Of course there's an SDK! It's here: https://github.com/GoogleCloudPlatform/gcloud-node https://github.com/GoogleCloudPlatform/gcloud-node
  - noiv 11y ago
    Well yes, just not an App Engine SDK :(
    - justinblat 11y ago
      What specifically are you looking to do?
    - deleted 11y ago
      [deleted]
plexicle 11y ago
Oh, are you kidding me? Man I got so excited when I saw the title, I already spammed the link to a couple buddies. I should have read it first. I want real GAE. Not managed VMs. I want to be able to spin up a quick GAE version on a deploy, and if it doesn't get the traffic to scale up, then it just sits there and doesn't cost me anything. Managed VM has 2+ instances running all month and is an incredible PITA to spin down if I need to temporarily. (The current workaround is to deploy a non-VM app and set that as default, which is just insane.) GAE apps based on Managed VMs need a free quota as well. I have many apps but some of them don't do much most of the time. A couple will spike hard randomly. I don't mind paying when it does. I like the security that if/when something really takes off, GAE has my back. In the meantime, I'll stick to Go on GAE. I'd love to be able to use Node, but things add up quickly if I have a couple hobby apps. I'm glad that they are pushing Node and I'm a HUGE fan of the Google Cloud. I just really want Node as a native GAE thing.
- thesandlord 11y ago
  There is a easier way to spin them down now: gcloud preview app versions stop main
  - plexicle 11y ago
    Well, that's good at least. Must have been very recent, I was talking to support only a couple weeks ago.
    - thesandlord 11y ago
      Yeah super recent. It was one of my huge pet peeves with Managed VMs, so I'm glad it's fixed.
    - justinblat 11y ago
      I think technically that command is still hidden lol. You can run it, it just doesn't show up in the docs. We'll be officially releasing that one in a few weeks.
BobTheCoder 11y ago
I don't mind managed vms, I just wish they had the same pricing model as app engine.

ionwake 11y ago

All I know about GAE is that when my python app started being used by HN users I upgraded my account to a paid one to increase performance. In the transition , without any warning, google took the app down, for a day. Thus killing all chance of acquiring a userbase now that I had hit the frontpage. When I went onto the IRC GAE channel to ask someone for help to unlock or keep the app running during this important moment the reply from a staff member was ' You should have planned better'. I was surprised to hear this response and did not pursue the matter further. Its all in the IRC logs if any of this sounds doubtful. TL;DR; Surprisingly, I did not find the GAE platform to be a supportive environment for startups.

boulos 11y ago
That's awful, I'm sorry you experienced that. It's no use to you now, but if you had gotten that message to me, I would have walked over to SRE / Support and we could have reset your quotas (I assume that's what you mean by "Took the app down for a day"). Disclaimer: I work on Compute Engine not on App Engine.
- michaelbuckbee 11y ago
  I appreciate that you'd be willing to do that, but the fact that it's necessary to get a human (even a technical and helpful one such as yourself) involved to deal with this is a big red flag.
  - boulos 11y ago
    To be clear, the person he reached should have reset / gotten the quota reset (if that's the root cause). There's often a human needed when something goes sideways, which is why we have paid support for Cloud - cloud.google.com/support. The issue is that he reached a human, and it didn't fix his problem. So like I said, sorry! (but I think it's okay to rely on humans fixing things like this)
    - ionwake2 11y ago

ungzd 11y ago

Quirkiness of Javascript + extreme weirdness of App Engine (especially datastore) = best of both worlds.

boulos 11y ago
Aww, I like Datastore. But, there's no requirement that you use Datastore. The point of App Engine is that it's "you just give us code, we run it for you". With Managed VMs (built on top of Compute Engine) we're able to offer that programming model with a more flexible environment. As someone mentioned elsewhere, there's even "just hand us a random docker container". We've had people run ffmpeg because you can just hook up App Engine's request/response and autoscaling to an arbitrary thing, and you've got an autoscaling web service. tl;dr: Don't get hung up on the old "extreme weirdness of App Engine", this is different.
- deleted 11y ago
  [deleted]

smegel 11y ago

Interesting things are happening to GAE these days. They are moving towards custom runtimes using docker that the user can configure...even their datastore API is being replaced by something resembling S3. GAE -> Google Cloud Platform.

thesandlord 11y ago
Cloud Datastore is more like DynamoDB, not S3. Cloud Storage is like S3. Yay names! Custom runtimes are trying to give people the "No Ops" of App Engine with the flexibility of Compute Engine. Like always, there are tradeoffs, but more options is usually better :) (I work for Google Cloud Platform)
- TheLogothete 11y ago
  Do you work on the database products? Please, PLEASE collaborate more with the analytics team. We want export to BQ for regular accounts. It's not a big ask and it will improve the lives and businesses of MANY people. I think it will affect google cloud revenues quite positively too.

cdnsteve 11y ago

What version of node is supported? Currently in all other languages they are usually a major version behind (in everything).

justinblat 11y ago
You can specify whatever version of Node.js you want to use in the engines section of your package.json: https://cloud.google.com/nodejs/resources/runtime#engines https://cloud.google.com/nodejs/resources/runtime#engines
- BinaryIdiot 11y ago
  Frustrating that it doesn't specify how it gets those versions. Is it always direct from the node folks so it's going to always have all versions or do they have their own cache?
  - diggan 11y ago
    Reading the source code of the docker container they use (https://github.com/GoogleCloudPlatform/nodejs-docker/blob/master/base/install_node#L34-L44 https://github.com/GoogleCloudPlatform/nodejs-docker/blob/ma...) it seems like it's reading the available versions from an URL (http://storage.googleapis.com/gae_node_packages/node_versions http://storage.googleapis.com/gae_node_packages/node_version...) which points to some Google hosted storage. So I'm guessing they have some process to update this list after a new release and also caches the versions themselves.
    - dlor 11y ago
      Yep, we update the available versions constantly and just serve them from GCS to reduce download time. I'll get this documented somewhere. Disclosure: I work on GCP.
      - BinaryIdiot 11y ago
        Fantastic, thanks!

cdnsteve 11y ago

What version of node is supported? Currently in all other languages they are usually a major version behind (in everything).

jorgecurio 11y ago

is anyone using node.js in enterprise? because everyone seems to be convinced that the enterprise companies are going to uproot all the network of partners, developers, vendors, and billions of dollars of investment made over the past decade? Like it or not, JVM pretty much rules the enterprise space and there's just no way to do it another way. Literally, you will be turned away because you told the IT department they should write everything in javascript and node.js. Why? Because everyone is doing it. Not good enough. I don't know what have your real world experience been? I'm learning Java because outside of rinky dink consumer and 19.99 / month productivity apps for SME, it's viewed in a completely different lime light. Honestly, the shortcomings of Java is greatly exaggerated vs how much of it is dominating the space, and it irks me when a 19 year old CTO on the autistic spectrum claims suddenly node.js is the way to build the world's infrastructure on. javascript is great for async programming but much work has been done to make it feel procedural. My feeling is that node.js is a hard sell for enterprises already deeply entrenched with JVM. Unless you offered the migration at a cost and undercut everyone. Not a great business imho where lower price is the only thing going for you because it never lasts. Eventually, you open source and become a grass eating goat relying on training, certification, support licenses.

adrianpike 11y ago
Yes, we are, but caveat; it's only for a few smaller services where the functionality makes lots of sense to be heavily asynchronous, and we're not leveraging many external dependencies. FWIW, deployment and packaging is still a painful issue for node - I've been bitten in the past by people not semver'ing their modules in a non-breaking way.
- jorgecurio 11y ago
  thanks I think that is a more realistic real world scenario, some people on HN thinks they can change the enterprise world but it's super hard to do it with javascript imho and evident by that it's not being used in wide deployment.... Java is still king in the enterprise world.
TheLogothete 11y ago

mattbillenstein 11y ago

Don't use AppEngine. I migrated Homejoy off of AppEngine in 2014 and I can't recommend the platform for any reason whatsoever. It's slow, expensive, not based on OSS, and not particularly stable. Bits of it (urlfetch, search, etc) would go down for hours at a time and you're basically hosed until they fix it. I think Heroku is usable with a similar level of effort -- and you have a much easier migration path to something else if you're not using Google's proprietary datastore IMO.

justinblat 11y ago
Apologies you had a bad experience. There's no doubt the original App Engine had some... quirks. You may want to take a look at Managed VMs. All of the new runtimes are based on Docker images, are entirely extensible, and don't require the use of the original App Engine APIs: https://cloud.google.com/appengine/docs/managed-vms/ https://cloud.google.com/appengine/docs/managed-vms/ You can use whatever Database or NPM modules you want.
- edwinnathaniel 11y ago
  Hi Justin, I'm currently trying to get my feet wet on GAE. Judging from your statement, are you suggesting that people should look into Managed VMs and maybe skip the regular GAE (or known as the original GAE)? Perhaps the original GAE will be deprecated in the future?
  - boulos 11y ago
    No, that's not the case. Managed VMs is a more flexible environment than our "GAE Standard" environment, but (at least currently) comes with some of the tradeoffs of a full VM (startup time, minimum size, etc.). If you've got say a vanilla python app that doesn't need custom native modules, GAE Standard is great! If you want to run something we don't currently support, then Managed VMs gives you the flexibility a Compute Engine VM can offer you (run whatever you like). Disclaimer: I work on Compute Engine, not App Engine but I'm familiar enough with their thinking.

itaysk 11y ago

I'm surprised they didn't have it until now...

joeblau 11y ago
It's not really that surprising. They started with Python because the inventor of Python worked there. Then they added Java because... well it's Java. Then they added GoLang because they invented it. Then they added PHP because most of the internet still runs on that. So Node is the next logical step... or Ruby probably would have worked as well.
- aleaxit 11y ago
  Actually, when we started (GAE) with Python, my friend Guido didn't YET work here -- I was finally able to convince him to join, shortly after GAE's launch, in part thanks to the prospect of working on GAE's Python (which is what he ended up doing here, especially developing ndb, but also many other little things). Python was at the heart of Google for long before that (read Levy's "In the 'plex"...!-) -- or I wouldn't have joined Google 11 years ago, BTW. As for Java, well -- guess who announced Java to the world at Moscone Center in SF 21 years ago...? One Eric Schmidt, then Sun's VP for SW products (he reminisced about that this morning at his Next keynote)... who just happened to be Google's CEO when Java was announced as GAE's 2nd language!

esilverberg2 11y ago

I wrote about my (unsuccessful) app engine experience many years ago...hopefully they've improved since then. http://www-cs-students.stanford.edu/~silver/gae.html http://www-cs-students.stanford.edu/~silver/gae.html

boulos 11y ago
Good writeup! I think the main change is that App Engine has really morphed into a "You know, you don't have to use Datastore". Cloud SQL was added precisely because of the issues you raise: being on App Engine meant being in a totally foreign world, and not being able to get out easily. If you were to try again today, you'd take stock Django 1.9 via pip install Django -t lib and using vendoring: https://cloud.google.com/appengine/docs/python/tools/libraries27#vendoring https://cloud.google.com/appengine/docs/python/tools/librari... . You point it at your Cloud SQL instance in your DATABASES line (https://cloud.google.com/appengine/docs/python/cloud-sql/django#usage https://cloud.google.com/appengine/docs/python/cloud-sql/dja...) and you're done. tl;dr: Yep! Things have improved in the last 5 years. Disclosure: I work on Compute Engine.
- sspross 11y ago
  It sounds like you know what you're talking about. Would you mind to take a look at my issue/question on the appengine django skeleton[1]. The Problems start after you've pointed your database to cloud sql und start using GAE specific features like task queue etc. [1] https://github.com/GoogleCloudPlatform/appengine-django-skeleton/issues/14#issuecomment-196490921 https://github.com/GoogleCloudPlatform/appengine-django-skel...
  - boulos 11y ago
    For your first question (playing with Cloud SQL) you've got two options. Directly interact with Cloud SQL from your laptop or like you said have a "development MySQL" separate from prod / staging. Pointing at your Cloud SQL instance (maybe using a staging or development database) can be done in settings.py. The switch between modes let's you specify the IP address for your Cloud SQL instance when you're not on App Engine (which uses a special UNIX domain socket). We also recommend you connect over SSL when doing so outside (see more at the link I provided, or search for cloud sql ssl). Having a local environment for MySQL is also easy enough depending on your box, but has the usual difficulty of needing to have a good test suite / being populated with useful data. I think it's worthwhile if you can pull it off, but I've seen companies big and small decide its not worth it (and instead use a separate table or database on the same server, maybe as a different user as another line of "don't screw up prod"). Finally, the interaction between Django's development server and the App Engine one isn't great. My response was intended to remind people that if you're not looking to use App Engine specific APIs you really can just get going and almost not notice. But you should be able to run your migrations speaking to Cloud SQL remotely without needing to import say Task Queues. So my workflow was to use manage.py for things like that, while using the dev_appserver when I actually needed to emulate App Engine (as opposed to the Django runserver command). Does that help? Edit: I forgot to add that we're working on decoupling the "emulation" of our services so that this kind of interop works better. See https://cloud.google.com/sdk/gcloud/reference/beta/emulators/ https://cloud.google.com/sdk/gcloud/reference/beta/emulators... for Datastore and PubSub (and more on the way).

shubb 11y ago

I used google app engine for a project with very specific requirements - the client wanted an application for internal use that didn't have any ongoing costs (sign off) and would look after itself for a couple of years. The platform has a lot of quirks, and I had to do a lot more learning than I expected to get the fairly simple project done. But once it was up, it did it's thing. I'd definitely recommend for free hosting internal applications and personal projects that are just for you to use. Just understand that you'll need to shape your application around the wierdnesses of the platform.

justinblat 11y ago

Greetings folks, I'm the PM for Node.js on App Engine. If anyone has any questions, feel free to ask me anything!

tomh- 11y ago
I have one, how can European individuals use the paid version of any Google cloud services? Last time I checked only commercial entities were allowed to use the paid version.
- boulos 11y ago
  Warning: IANAL. First, sorry for the pain! As an explanation, it's all about VAT (determining it, collecting it, etc.). Outside of Ireland we dump it on you: https://support.google.com/cloud/answer/6090602 https://support.google.com/cloud/answer/6090602 But specifically: > “Business” status means that you'd like to see a potential economic benefit from your development activities, for example: using the Google Cloud Platform to develop prototypes or applications with a view to generating revenue in the future. Most software developers -- including affiliates, sole traders, self-employed merchants, partnerships, students and others -- use Google Cloud Platform for business purposes. I can't help you decide what your VAT on the use of our services would be (this is the realm of accounts and lawyers). I know AWS makes a different decision than we do, and we're very aware of the friction (again, sorry!). Disclaimer: I work on Compute Engine, but again IANAL.
  - tomh- 11y ago
    Thanks for your answer, although its worrying we need to have a lawyer around to see how we can signup without accidentally violating my local country's VAT regulations as a developer who just wants to pay and build stuff :(
ralusek 11y ago
This appears to still just be a Managed VM, not really App Engine. What is new? This has been around for a while... Is there any plan to implement Node fully on GAE?

davidjgraph 11y ago

So many negative views of GAE, I have to add our, overwhelmingly positive, experience. We were paying $200/month for a load balanced EC2 setup in 2011, that's for around 2k unique daily users. The load balancer kept on false triggering new instances, so we went to GAE in 2012. Since then our uptime has been horrifically good, heavily thanks for GAE, trashing that of our competition, who mostly use AWS. Last month we paid $150 for everything on GAE, traffic, CPU, etc. G-Analytics shows a peak of 54k daily uniques last week. All the scaling is dealt with for us, we can handle literally any traffic peak. i.e. x27 more traffic, lower cost, better uptime, spending virtual zero time looking at the operations side of things. This is a mission critical app, although, we've never invoked support, so I can't comment on that side.

stephenitis 11y ago
did software change to account for efficiencies?
- davidjgraph 11y ago
  No, the basic architecture hasn't changed, but the app does a lot, lot more than it did in the AWS days.
latchkey 11y ago
Without GAE/GCP my business wouldn't be where it is today and in such a short amount of time. I can't reveal numbers, but our bill and usage is significantly higher than yours. But, it is still just a rounding error compared with our revenue. Thanks to GAE, we focus our work on real features for our partners, which translates directly into profit. A post on this topic that I made a month ago: https://news.ycombinator.com/item?id=11159840 https://news.ycombinator.com/item?id=11159840 Thanks again Google! =)
PostThisTooFast 11y ago
Amazon is a mess. But should we care if Node is on Google? What about this: https://news.ycombinator.com/item?id=4495101 https://news.ycombinator.com/item?id=4495101

TheAceOfHearts 11y ago

So, considering that it's beta, where should I expect rough edges? Is there any list that shows what's missing to go from beta to full production? I'm actually really interested in trying this out, but I want to know what potential issues I should expect to deal with. Something I found really awesome about Google's services is that they provide fake local implementations that you can run from the CLI. This means that you can run tests without having to hit the outside world! Somewhat related to that, if you're using S3, check out s3rver [0]. It's a node implementation of fake-s3 [1], which is a local implementation of S3 that you can use during development and testing. Looking through the examples, I'm confused about the deployment strategy for SPAs. Looking at the webpack example [2], you build the frontend app every time the server runs? That's REALLY slow for any reasonably sized SPA :(. It means you generate a new build each time the server restarts, and for something like node that's really common. (AFAIK, the suggested strategy is to just let it crash and restart quickly.) But it also means that you might have multiple servers running different version of your dependencies (since there's no shrinkwrap). Even worse, if you're running multiple servers with slightly different dependencies, it might also mean that you'll end up building slightly different versions of your SPA in every server for any given release! I'd love to see an example where you build and tag the SPA in a single place, and you make sure all the servers are serving the same bundled code. [0] https://github.com/jamhall/s3rver https://github.com/jamhall/s3rver [1] https://github.com/jubos/fake-s3 https://github.com/jubos/fake-s3 [2] https://github.com/GoogleCloudPlatform/nodejs-docs-samples/blob/master/appengine/webpack/package.json https://github.com/GoogleCloudPlatform/nodejs-docs-samples/b...

eknkc 11y ago
This is my understanding about Google Cloud services; - Alpha means sub par quality / rough edges / unresolved issues. - Beta means fairly stable but no SLA, no guarantees from Google. So I'd guess it's fine to use now.
- Xorlev 11y ago
  With Google it's either beta or deprecated.
11y ago

exotiik 11y ago

and make sure billing is enabled !

outside1234 11y ago

Well that took forever. Congrats on getting to the right place though Google.

mastazi 11y ago

Wow, I thought that GAE was basically dead and now I find out that, besides the latest addition of Node.js, it supports Python 3, Java 8 and even Docker-based custom runtimes...

ekso 11y ago
Why did you think GAE was basically dead?

vskarine 11y ago

Has anyone tried running MeteorJS on GAE?

fibo 11y ago

Great news!

lambdacomplete 11y ago

The only reason why I'm still on AWS: I don't own a credit card.

atonparker 11y ago

We've tried node.js as a managed VM a few times before this. One of the biggest pains is logging. Despite documentation implying that JSON logs are supported (can't find this anymore, maybe it was removed), we still haven't been able to get GCP to read JSON logs. You lose out on all of the nice filtering and request grouping provided by the GCP log viewer. The issue is here[0] and it would seem they still haven't addressed it a year later. [0] https://code.google.com/p/googleappengine/issues/detail?id=11678#c5 https://code.google.com/p/googleappengine/issues/detail?id=1...