9 ms·
Node.js is now available on App Engine standard environment
- sam0x17 8y agoBeen waiting on this for years.
- deleted 8y ago[deleted]
- BrandiATMuhkuh 8y agoIsn't firebase cloud-functions from google doing mostly the same?
- pspeter3 8y agoI think Firebase Functions are more restricted in what/when they can run
- sudhirj 8y agoFirebase cloud functions are more AWS Lambda competition, where you write JS functions that follow a particular interface and can do certain things. This is being able to deploy regular NodeJS servers with no special changes or modifications, and have them run an autoscaling sandbox that does a really good job of keeping the server alive (and charging you) only when absolutely necessary. Also scales up infinitely (up to your budget, anyway) in a pinch.
- tmpz22 8y agoWasn't Ingress/Niantic using App engine when they were going through their scalability problems? Nothing scales infinitely.
- vemacs 8y agoif you are talking about Pokemon GO, then no. i was on GKE (https://cloudplatform.googleblog.com/2016/09/bringing-Pokemon-GO-to-life-on-Google-Cloud.html https://cloudplatform.googleblog.com/2016/09/bringing-Pokemo...).
- filleduchaos 8y agoYou can deploy a regular NodeJS server via firebase cloud functions as well.
- thesandlord 8y agoThe execution and billing model is a bit different. App Engine is great for full HTTPS servers, and is billed on an instance basis (it does scale to zero when unused) GCF is great for responding to events in the cloud (Pub/Sub, Storage, webhook, Firebase database) and is billed on a per function execution basis. The actual runtimes are quite similar, though GCF is still Node 6 based for now. (I work for GCP)
- oomkiller 8y agoVery nice. Now, maybe allow it to connect to a typical SQL database via Cloud SQL? It seems like the only hosted database you can use with this is Cloud Datastore, a proprietary NoSQL database.
- rfks 8y agoLooks like this is supported https://cloud.google.com/appengine/docs/standard/nodejs/using-cloud-sql https://cloud.google.com/appengine/docs/standard/nodejs/usin... ?
- oomkiller 8y agoCool, this has previously been an issue. They haven't updated the docs I was looking at yet https://cloud.google.com/sql/docs/postgres/connect-app-engine https://cloud.google.com/sql/docs/postgres/connect-app-engin...
- steren 8y ago(PM on the team) Good catch, I'm sending a pull request.
- frabcus 8y agoWhile you're here... It's really unclear from this what the security best practice is for where to put the Cloud SQL password. Do we just assume the password is public info and rely on IAM? This discusses really clunky methods to use the legacy datastore or a bucket to store a password. https://stackoverflow.com/questions/22669528/securely-storing-environment-variables-in-gae-with-app-yaml https://stackoverflow.com/questions/22669528/securely-storin... Seems odd when competitors (Heroku, Beanstalk) have a special system for keeping environment variables with the password in.
- oomkiller 8y agoAt this point I believe Google has a deep aversion to storing secrets in environment variables. The best way I have found so far is to use Cloud KMS to encrypt the credentials and have the deploy process be able to fetch and decrypt them on the fly. Some folks also store that and other config in Datastore, although I think that's really clunky. You can also use Google Cloud KMS-backed Hashicorp Vault: https://cloud.google.com/solutions/using-vault-for-secret-management https://cloud.google.com/solutions/using-vault-for-secret-ma...
- pspeter3 8y agoAre there type definitions for the Google Cloud packages? Edit: For those also interested, here is the relevant issue https://github.com/GoogleCloudPlatform/google-cloud-node/issues/952 https://github.com/GoogleCloudPlatform/google-cloud-node/iss...
- kylehotchkiss 8y agoLook at the deploy time for Express in Node.js standard compared to Flex! Much faster. Thank you, Googlers.
- steren 8y ago(PM on the team) Thanks! Also, subsequent deployments are faster than the first thanks to some caching in the build process.
- pier25 8y agoAnyone can comment on AppEngine vs Heroku?
- d3sandoval 8y agothe lack of availability of node.js on GAE standard was one of the main reasons I decided to deploy my next.js app on Heroku instead. I may have to try it again to compare head to head... Is there anything in particular that you'd like to know? E.g. performance, ootb tooling, etc?
- pier25 8y agoEverything, but performance and pricing in particular.
- friedman23 8y agoThe article talks about upsides to using Standard over Flexible but what (if there are any) are the downsides?
- teich 8y agoPM on the team. Flex allows bigger instance sizes, VPC access, custom base docker images with custom binaries. Generally, it's a bit more flexible. :)
- friedman23 8y agoThanks for the response! You all are doing some good work.
- frew 8y agoDoes this runtime have access to the Standard APIs? It doesn't seem like the docs have any mention of Task Queue/Memcache/etc.
- thefounder 8y agoNo. The real issue is that these APIs/platforms are closed source. Sooner or later you may need to move a service from appengine to a non-gae environment and you will find that nothing works. If you invest in tools/libs (i.e wrappers/ORM around the GAE APIs) the loss is even bigger.
- sudhirj 8y agoThe other standard runtimes have special packages, but I think for Node everything is just being lumped into the companion package https://cloud.google.com/nodejs/docs/reference/libraries https://cloud.google.com/nodejs/docs/reference/libraries
- yathern 8y agoIt only supports Node v8 (which sounds like an upside to me). The main difference is that you can't customize the run time image.
- stefano 8y agoI wish they had python 3. It's still stuck on python 2, which will be unsupported in less than two years.
- stewart27 8y agoHi, I'm a Product Manager on the App Engine team. We're actually (finally!) gearing up to offer Python 3. Would you like to be an alpha tester? Here's the sign-up form: https://docs.google.com/forms/d/e/1FAIpQLSfLXViTUmtY6Ed_Gm9HohVVe7HMKcT37_MvzgPpKIDQTkZNjg/viewform?usp=sf_link https://docs.google.com/forms/d/e/1FAIpQLSfLXViTUmtY6Ed_Gm9H...
- lostmsu 8y agoHow about .NET Core?
- rattray 8y agoOoc, why run .net core on gcp instead of azure?
- tracker1 8y agoI'd presume the most common are probably other experience with GCP, other applications on GCP, or prefer GCP tooling to Azure. I haven't compared pricing, so no comment there. Lastly, anti-ms pov.
- mcny 8y agoFor me there's main reasons are 1. Free tier 2. Features in that order It is perfect for "hobby" projects that have no revenue today. If a "revenue strategy" emerges, it isofcourseeasy to enable billing but there is no 12 month $300 trial.
- wikibob 8y agoCan you share even a smidgen of what took so long (10 years!)? From various comments over the years here and elsewhere, I’ve been led to the understanding that Python 3 was “coming soon” several times. 1. Why now? 2. Is it true this time? 3. Will Python 3 keep up to date with new releases, or 5 years from now will we be stuck on 3.6? 4. What is the future and roadmap of AppEngine? The feeling in the community is that AppEngine is another Google Reader, and death may be imminent. Can you reassure us? 5. What is the future of Standard and Flex? Should all new projects be started on Flex? Is standard ever going to keep up with language releases? Thanks
- derefr 8y agoCan someone from GCP explain why the App Engine standard environment has such long lead-times in terms of language support? What is required, infrastructurally, to ship a new standard-environment language?
- steren 8y ago(PM on the team) To be able to deliver Node.js on App Engine Standard, we had to develop an entirely new sandbox environment. You can learn more about it by watching this I/O talk (starting at 15min): https://youtu.be/ogexnfng_hE?t=14m48s https://youtu.be/ogexnfng_hE?t=14m48s
- mattsoldo 8y agoAnother PM on App Engine here. Historically supporting a language on App Engine required modifying the languages libraries to work on proprietary parts of Google infrastructure. This entailed a decent bit of effort on our part, and also meant that some apps & code wouldn't execute correctly. We now use technology based on gVisor (https://github.com/google/gvisor https://github.com/google/gvisor) in production for App Engine standard. This allows us to release new language runtimes unmodified. Node.js support is the first runtime based on this new stack. We completely agree that we should be releasing support for new languages & major versions more frequently...
- bpicolo 8y agoAre there plans to use the new tech for Go app engine? Go on app engine was a real struggle with the non-standard bits.
- patwolf 8y agoThis surprises me. I had assumed incorrectly that flexible environments were the future of App Engine. I'm pretty happy with using flexible environment for nodejs. However, the fact that this supports scaling to zero, and has headless chrome built-in, I could see using this for a low-cost headless Chrome environment. I've had several projects where I've wanted a node server separate from the main app server on which I could use headless Chrome for pdf generation. It's never easy to justify the cost of an extra instance for that purpose. But with a zero scale capability, this could presumably be cheaper and easier to justify.
- waleedka 8y agoSame here! I was going with the assumption that standard environments are going away since they haven't been updated for years.
- zer0faith 8y agoSounds like this is basically PCF/Docker for Google?
- deklerk 8y agoApp Engine is a PaaS (platform as a service), as is PCF. Docker is a containerization platform, though, which is entirely something else. e.g. a PaaS might use Docker (many do).
- SoulMan 8y agoStandard, in my opinion, is still the best bet of you want most of the things out of the box from GCP. Flex does allow a lot of customization but you miss out on Memcache, Task Queues, Auto tracing in Stackdriver etc which are sometimes really backbone of your highly scalable architecture given that you don't want to manage your own infrastructure.
- sdhgaiojfsa 8y agoNot to mention that standard is enough to run a pretty significant free-tier site. I'd way rather program in JS than Python [edited: thanks yathern], especially now that async/await is a thing in Javascript. Very cool work by the Appengine team.
- yathern 8y ago> rather program in Node than JS Node is JS, of course (with a different standard library). Did you perhaps mean to compare Node to python or something?
- sdhgaiojfsa 8y agoThanks for the correction! I mean, "Rather program in JS than Python," but my hands typed faster than my brain was keeping up. Oopsie.
- thefounder 8y agoYeah but the "standard" env is always behind(at least on the Go version) and on top of that you are even more locked in. The only real advantage of the standard environment is that you pay only for what you use. On the "fkexible" environment you must keep at least one instance live all the time. IMHO you should avoid GAE.
- jchw 8y agoPerhaps you should avoid GAE for hobby projects, but if you can afford it, it can save you some money in not needing any ops. For hobby sites you probably barely need ops.
- z3t4 8y agoCan you deploy a email or dns server/client on this or is it only for http servers !?
- steren 8y ago(PM on the team) App Engine is designed for stateless HTTP workloads. for an email or DNS server, I recommend using Google Kubernetes Engine or Google Compute Engine.
- Welytech 8y agoWowoo this is great improvement.
- 11235813213455 8y agoHow to handle app secrets though?
- davidspiess 8y agoIs it possible to downgrade an existing flexible app to standard without changing A records? It seems there is a dedicated page for that, but it leads to a 404. https://cloud.google.com/appengine/docs/standard/nodejs/upgrading https://cloud.google.com/appengine/docs/standard/nodejs/upgr...
- outside1234 8y agoWait, they didn't have this already???? Welcome to the present.
- guiomie 8y agoI'm debating between GAE standard or Firebase now for my next project :/ I've always been confortable with nodejs but didn't want to bother with GCP (GAE flex). One part that I liked about Firebase is how it's easy to support multiple login providers (twitter,fb...etc). Is there something similar in GAE standard nodejs ?
- testb 8y agoI don't quite understand how GAE would affect using Firebase or not. Even on standard you would still use Firebase Auth, there is no GAE specific authentication.
- cavisne 8y agoThere actually is some gae specific authentication https://cloud.google.com/appengine/docs/standard/python/config/appref#handlers_login https://cloud.google.com/appengine/docs/standard/python/conf... But it has nowhere near the features of Firebase Auth.
- steren 8y ago(PM on the team) These are not incompatible, you can use Firebase authentication with App Engine. Get started at https://firebase.google.com/docs/auth/ https://firebase.google.com/docs/auth/. The firebase Authentication Admin SDK has a Node.js client library that is compatible with App Engine (https://firebase.google.com/docs/auth/admin/ https://firebase.google.com/docs/auth/admin/).
- Jonovono 8y agoWe use Firebase (and Firebase Functions :fire:) for most of our stuff. Then have a few things running on App Engine. What are you building? I'm a big fan of firebase if it makes sense for what you are building!
- vira28 8y agoFirebase user here. Played with GAE in the past and now running all the backend code in functions.
- 8y ago
- starforce 8y agoAre there differences in the scaling characteristics of the nodejs standard environment and the python 2 standard environment? Does one environment scale faster than the other?
- steren 8y ago(PM in the team) Both runtimes are using the same autoscaler, we recently updated it: https://cloudplatform.googleblog.com/2018/05/Increase-performance-while-reducing-costs-with-the-new-App-Engine-scheduler.html https://cloudplatform.googleblog.com/2018/05/Increase-perfor...
- westoque 8y agoWow! I've been waiting for this for a long time. This is helpful to people starting new projects. Using the Standard environment would cost significantly less than the Flexible environment which is one of the reasons I avoided GAE before.
- wasd 8y agoWow! Any chance we'll see Ruby in the standard environment?
- samblr 8y agoIm using flex with node. love simplicity of app engine and suite of tools in gcp - cloud debugger is dope. Does cloud fumctions support node 8 yet? Will make life easier
- doomslice 8y agoI have a NodeJS app running on flex that I always thought was a little pricey, so I'll probably move it to standard. However, I do have one question. My app server currently polls another external endpoint periodically and does something based on the results. If I move it to standard will this still work? I assume not since it can scale to zero.
- mylesborins 8y agoYou can always set up a cron service https://cloud.google.com/appengine/docs/standard/nodejs/scheduling-jobs-with-cron-yaml https://cloud.google.com/appengine/docs/standard/nodejs/sche... https://github.com/GoogleCloudPlatform/nodejs-microservices-demo/blob/master/task-scheduler/cron.yaml https://github.com/GoogleCloudPlatform/nodejs-microservices-... (I work for GCP)
- doomslice 8y agoHow about eventsource or websocket connections? Will those work (both websockets to the server and to external servers)?
- tejohnso 8y agoRequests are expected to complete within 60s so a websocket connection is probably not sensible. https://cloud.google.com/appengine/docs/standard/java/how-requests-are-handled#quotas_and_limits https://cloud.google.com/appengine/docs/standard/java/how-re...
- jkarneges 8y agoFor handling incoming long-lived connections, you could pair with Fanout [1] (disclosure: founder). Basically the client sees a raw WebSocket, and your backend gets woken up with HTTP requests when things happen. I haven't seen anything like this for outbound connections though. [1] https://fanout.io https://fanout.io
- steren 8y ago(Pm on the team) App Engine Standard does not support websockets at the moment.
- spullara 8y agoWell, the out of the box experience for me is pretty bad. Just created a TypeScript based React App, using npx, which runs locally just fine and when deployed to app engine it just 500s with nothing in the log as to why.
- steren 8y ago(PM here), if you share a GitHub repository that reproduce the issue, we can investigate and fix issues. Be sure to perform the build step before deploying.
- duxup 8y agoSo as a coding n00b I tend to put my stuff on Heroku if only because at the moment that's what I know... and it is free. Any advantage to jumping into Google's App Engine?
- dangoor 8y agoApp Engine autoscales. To be sure, this isn't a key feature for a lot of sites which don't achieve a ton of scale, but Google can keep spinning up new frontend instances and the Data Store essentially "just scales" if you use it properly.
- paxy 8y agoHeroku is no doubt the easiest to set up and use. I find their free tier too restrictive, however, since (1) apps are put to sleep pretty aggressively and take time to wake up, resulting in failed requests and (2) they have a set quota for total free hours/month across all apps on your account. GCP free tier provides a lot more in comparison (app engine hours, cloud storage, memcache, pub/sub, cron, egress, cloud functions and more), but you have to be willing to tolerate the vendor lock-in.
- duxup 8y agoYeah I'm slowly becoming more annoyed by how quickly Heroku puts things to sleep.
- freedomben 8y agotake a look at gigalixir.com. I'm moving an app over there right now and they don't do the annoying sleep thing, or the daily restarts. You also get a free 10k row db. Pricing is reasonable if/when you want to scale later too. More expensive than rolling it on GCE yourself, but you get all the convenience of the heroku workflow.
- duxup 8y agoI will check them out. Thank you.
- ilaksh 8y agoCool but being stuck on version 8 is a big issue for me so I don't think I could use it. Would prefer something like that over having to do a custom Dockerfile though if Node 9 or 10 were available.
- kondro 8y ago8 is the current LTS version.
- deleted 8y ago[deleted]
- PudgePacket 8y agoDoes it support running as a gRPC server?
- lorderikir 8y agoHey look, my quote is on there <3
- dosy 8y agoThat's great news. I've been using AWS EB (also great) but love datastore and making calls across to Datastore on GAE. Now maybe I can just put Node on GAE. But damn those free AWS credits !!! Edit: just saw that OS has the right dependencies for running headless Chrome / puppeteer. This is huge. Because GAE Standard was always quite sandboxed.
- amitagarwal 8y agoMy headless Chrome app is currently running on GAE Flex. It requires a few extra packages like fonts-liberation and fonts-indic. Does GAE standard allow that?
- steren 8y agoYes, Node.js on GAE Standard comes with all the OS packages needed to run Headless Chrome. Learn more at https://cloud.google.com/appengine/docs/standard/nodejs/using-headless-chrome-with-puppeteer https://cloud.google.com/appengine/docs/standard/nodejs/usin...
- tyscorp 8y agoLooks like it doesn't support yarn.
- zawerf 8y agoCan someone give me some estimated costs? I was playing with the flex environment with an express app (nothing special: cloud sql, cloud storage, etc). It ended up costing like ~$80 when I forgot to shut down a zero traffic dev instance for a month. It starts two instances by default and even if I set manual scaling to 1, it still costs like $40 a month. (I ended up going with a $5 VPS instead) Is standard environment any cheaper? Curious about some real numbers for development and when deployed at scale.
- deejbee 8y agoMy experience has been it is way over priced. It costs over $40/month for a 1CPU/1GB RAM node while linode, amongst others is $5 + $20 for a load balancer. I noticed in the google pricing calculator that you can set the Cores/vCPUs per hour to 0.05 and it goes down to $7/month! I'm not if I can do anything do with 0.05 of a core though, or if it automatically increases if it decides to.
- fibo 8y agoWhat a good new, finally! Is it possible to run a socket.io server?