8 ms·

I'm the tech lead and primary software engineer on the Go runtime. If you have any questions, please don't hesitate to ask! I'm super thrilled to announce that

• lvillani 8y ago
  Yes! Thanks for releasing this and making my life easier :) I have an old app deployed to App Engine. The app itself is rock solid and chugging along fine (I touch it once a year or so), but I dreaded having to deploy it due to the lack of vendoring support and other... peculiarities of the Go runtime. Glad to see this is no longer an issue!
• bpye 8y ago
  Several of your links are broken, they have been elipsised.
  • deleted 8y ago
    [deleted]
  • enocom 8y ago
    The ellipsis in full is "go-differences" for the migration guide.
  • buss 8y ago
    Sorry, I copy pasted without double checking. Now fixed!
• skybrian 8y ago
  I take it the datastore change is just an api change? (There isn't any data migration needed, is there?) It looks like memcache is moving to a third-party service now instead of a built-in api. Is performance different? Does anyone have experience with this?
  • buss 8y ago
    > I take it the datastore change is just an api change? (There isn't any data migration needed, is there?) Yup! > It looks like memcache is moving to a third-party service now instead of a built-in api. Is performance different? Does anyone have experience with this? I'm not sure about the performance impact. But I can tell you that we're working with the Cloud Memorystore team to have a better memcache story for the App Engine second generation runtimes
    • Strom 8y ago
      Both google.golang.org/appengine/datastore and cloud.google.com/go/datastore connect to the same data? Is there a list of incompatibilities somewhere? I remember seing some serialization differences between these two datastore libraries.
      • steren 8y ago
        Hi (App Engine PM here). Yes both libraries connect to the same database (Cloud Datastore). cloud.google.com/go/datastore is preferred as it uses the Cloud Datastore API instead of the App Engine specific API, so your code will be portable. You are right that the devil is in the details, and there might be some slight serialization differences. I cannot find an exact list. Feel free to post here if you need more help.
      • chrisbroadfoot 8y ago
        You remember right - I think you might be referring to this issue: https://groups.google.com/forum/#!topic/google-api-go-announce/79jtrdeuJAg https://groups.google.com/forum/#!topic/google-api-go-announ... > the cloud datastore package will default to writing your nested structs as entity values, while the appengine datastore packages will only write your nested structs as flattened sets of attributes. Other than that, there are no differences. (GCP Gopher)
• selljamhere 8y ago
  Does the move to gVisor pave the way to connecting CloudSQL instances? The removed limitations sound like they were road blocks.
  • merb 8y ago
    you already could connect to cloudsql instances. however you needed a proxy, but not because of AppEngine, you need it because of security. either you whitelisted your ips or you used the proxy (which go appengine could already use). so with go you just needed to use https://github.com/GoogleCloudPlatform/cloudsql-proxy https://github.com/GoogleCloudPlatform/cloudsql-proxy which means you just need to change the query string. however a new feature is emerging where you can have a private ip that is connected to your internal services, however it is still a beta feature. Source: myself, but not a googler, just a regular user
    • chrisbroadfoot 8y ago
      Notable, though, that this new runtime better supports connecting to Postgres instances through the unix socket (/cloudsql/). (GCP Gopher)
      • deleted 8y ago
        [deleted]
  • buss 8y ago
    For cloudsql in particular, you can connect via the `/cloudsql` Unix socket. But anything that needs raw socket access can now be used from App Engine. See https://cloud.google.com/appengine/docs/standard/go111/using-cloud-sql https://cloud.google.com/appengine/docs/standard/go111/using... and scroll way down to the sample code.
    • selljamhere 8y ago
      Awesome!
• merb 8y ago
  when will appengine support setting environment variables with gcloud command or inside the gcloud console instead of in the app.yaml, i.e. so that each environment can have their own variables without copying app.yaml's?!
  • steren 8y ago
    (App Engine PM here) At this time, app.yaml is the only way to set env vars. If you want to separate environments, you could use different .yaml files, I personally use app.staging.yaml and app.prod.yaml
    • dfischer 8y ago
      Not worried about secrets in repo?
      • CydeWeys 8y ago
        Tech Lead of Google Registry ( https://registry.google https://registry.google ) here. We run on GCP. You definitely don't want to check secrets into the repo for the same reason you don't set them as environment variables: It's not secure. The solution is to use Cloud Key Management Service. More info here: https://cloud.google.com/kms/ https://cloud.google.com/kms/ We use that to either store secrets directly, or to decrypt encrypted entities in a DB (e.g. Datastore). You can see how we use it here (our project is open source): https://github.com/google/nomulus/tree/master/java/google/registry/keyring/kms https://github.com/google/nomulus/tree/master/java/google/re...
        • asciimike 8y ago
          Another great resource: https://sethvargo.com/secrets-in-serverless/ https://sethvargo.com/secrets-in-serverless/ Shows how to encrypt env vars with Cloud KMS then decrypt them in memory.
          • dfischer 8y ago
• Strom 8y ago
  > I'm the tech lead and primary software engineer on the Go runtime. Hey thanks a lot for your efforts! Your direct communication about the Go runtime progress has been a breath of fresh air compared to the years preceding you. I wonder though, how many developers are there on the App Engine Standard Go team?
• throwaway9981 8y ago
  is gRPC supported now or is that more of an upstream HTTP 1.1 constraint?
  • steren 8y ago
    Hi (App Engine PM here) You can use any go package in the Go 1.11 runtime, including the ones relying on GRPC. Did I answer your question?
    • rhodysurf 8y ago
      You cant run a GRPC server though still right? Thats all I really want to be able to do especially now that the app engine endpoints libraries are no longer supported on python
  • buss 8y ago
    Inbound GRPC will be limited to HTTP1.1; that's a limitation of the platform rather than the language runtime. We're working on HTTP2 support, but I can't promise any timelines.
    • rhodysurf 8y ago
      By inbound GRPC you mean when running an app engine instance as a client right? So running a GRPC server is completely out of question still I would presume?
      • buss 8y ago
        I mean running a GRPC server, yeah. It's a limitation of our networking stack which we're working on fixing.
• wiradikusuma 8y ago
  It's always been my impression that Go on GAE is the most lightweight compared to Java/Python, as it not "memory hungry" like Java nor "slow" like Python. Am I correct?
  • buss 8y ago
    It is quite lightweight, yes! Though...I wouldn't necessarily call Python slow! Python keeps getting faster, and we now support Python 3.7 on our second gen runtimes: https://cloud.google.com/blog/products/gcp/introducing-app-engine-second-generation-runtimes-and-python-3-7 https://cloud.google.com/blog/products/gcp/introducing-app-e...
    • melling 8y ago
      Does anyone have numbers to compare? Go is a compiled language. Should perform much better. Python might be perfectly acceptable of course.
      • tomcam 8y ago
        I would absolutely love those numbers too. Always wondered about the working set of a minimal program, and also one with database access.
        • pkroll 8y ago
          Well, there's the Techempower benchmarks at: https://www.techempower.com/benchmarks/ https://www.techempower.com/benchmarks/ You can go to the filter, turn off all languages but Go and Python, and get a general idea. There's also the old Computer Benchmark Game, which should be taken with a whole barrel of salt: https://benchmarksgame-team.pages.debian.net/benchmarksgame/faster/go-python3.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
          • eiieirurjdndjd 8y ago
            I actually looked at this before making my other comment. You’re absolutely right about the salt. One of the go benchmarks is basically pure cgo using vector intrinsics. It naturally blows idiomatic JS out of the water. So you might get those kind of results if that’s the kind of code you’re going to write, but it’s far from what I consider idiomatic. Another language’s proponent would be fair to point out that most languages have C FFIs.
• B-Scan 8y ago
  Any plans to support the cloud endpoints with the standard env? Wondering if the new runtime would be able to be embed that. That would be only reason for me not to switch from flex env.
  • asciimike 8y ago
    (GAE PM) Cloud Endpoints currently runs on GAE Standard Gen 1 and GAE Flex. We're investigating a solution for GAE Standard Gen 2 (which go 1.11 is, along with python 3.7, node 8, etc.) and GCF.
    • B-Scan 8y ago
      Thanks for the clarification. Wasn’t aware that it works with Go on Standard Gen 1. Any ETA for Gen 2 or alpha signup?
      • asciimike 8y ago
        Clarification: GAE Standard uses the Endpoints framework, which is Python and Java only (https://cloud.google.com/endpoints/docs/frameworks/about-cloud-endpoints-frameworks https://cloud.google.com/endpoints/docs/frameworks/about-clo...). Sorry for the confusion. Flex uses ESP (https://github.com/cloudendpoints/esp https://github.com/cloudendpoints/esp), which is deployed as a sidecar, and works with any runtime. As for EAP: unfortunately not. Personally, I'd like to see a managed version as opposed to the current framework or sidecar approach, as it'll be easier to implement across products, but that means it'll likely take a little longer.
• warrentr 8y ago
  How long will the legacy App Engine APIs continue to work? Will they be removed before the runtime is GA? These new runtimes seem amazing (and fix a lot of problems), but we also lose a ton of functionality that made app engine so desirable (easy users auth, images api, built in email sending, search, cron, and others). It kind of feels like we've thrown the baby out with the bathwater.
  • steren 8y ago
    (App Engine PM here) The App Engine APIs will be present in the Go 1.11 runtime when it goes GA and until it is turned down. We do not know exactly which future version of the runtime will stop supporting them. We are working on Google Cloud standalone products to replace most missing features (e.g. Cloud Scheduler for cron jobs). Not all of them are ready yet. App Engine APIs were great when there was no alternative (when GCP did not exist, or when App Engine could not use arbitrary packages), but at the same time, contributed to the "lock-in" of App Engine, which was one of the main criticism. We believe it is in the long term benefit of our users to use standalone Google Cloud or third party services, instead of replying on APIs only accessible in App Engine.
    • warrentr 8y ago
      Thanks for the explanation. It makes sense and hopefully replacements are generally available before we lose the old apis.
• Sir_Cmpwn 8y ago
  Hey buss, thanks for stopping by. How do you feel about the distinction between Google and Go? It sometimes feels like they're joined at the hip, something which historically had made people queasy (Java and Oracle, .NET and Microsoft, etc). Though Go is a project which came from Google, I personally feel that it better serves the interests of the Go community to treat them separately. Google receives special treatment from Go - the announcement that AWS Lambda would support Go did not receive similar fanfare on golang.org. Side note: there is some activity on the Go blog which is more mutual cooperation and less Google hivemind: https://blog.golang.org/go-cloud https://blog.golang.org/go-cloud But I still wonder why this stuff belongs here.
  • deleted 8y ago
    [deleted]
  • justinclift 8y ago
    Agreed. To even submit content to the Go Blog requires a Google account. Ugh.
  • jhabdas 8y ago
    Three smart guys made go. Google only has one or two left.
• heromat 8y ago
  I just tried to use go111, but I get the following message: >[7] Access Not Configured. Cloud Build has not been used in project <project> before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudbuild.googleapis.com/overview?project=<project> https://console.developers.google.com/apis/api/cloudbuild.go... then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry. So go111 cannot be used without Cloud Build, for which I have to activate billing?
  • buss 8y ago
    Ah, yeah, looks like you have to activate billing. You'll be solidly within the free tier, though, so you won't be charged.
    • chrisbroadfoot 8y ago
      You can set a $0 limit on your billing account, too.
• paulddraper 8y ago
  I thought Go was statically compiled....i.e. it doesn't have a separate runtime from the program. Does anyone have any elucidation?
  • chrisbroadfoot 8y ago
    Correct, Go's runtime is compiled into the program's binary. However, there's still an environment needed to run that binary. For Go on App Engine, this includes the sandbox (backed by gVisor) and the operating system (Ubuntu). We also tend to call everything in the toolchain the "runtime" - this includes the CLI for uploading and staging your app, and the builder used to compile your program. Not the strictest definition :) (GCP Gopher)
    • paulddraper 8y ago
      So App Engine is a build system as well?
      • chrisbroadfoot 8y ago
        Yes, with App Engine, you upload source code, and it's compiled for you remotely. For this newer runtime, it happens inside a Cloud Build invocation. For App Engine flexible, you can skip that build step and provide your own container, but that isn't possible for App Engine standard. (If you're interested in something like that, see https://g.co/serverlesscontainers https://g.co/serverlesscontainers)
• openbasic 8y ago
  Vendoring! Modules! Finally. Do you have any examples with proposed solutions for local module setups, without having to rely on GOPATH changes?
  • TheDong 8y ago
    This is a go question, not an app engine question. See this page on the go wiki for an example of how to use go modules: https://github.com/kubernetes/kubernetes/pull/58098 https://github.com/kubernetes/kubernetes/pull/58098 Note: you need to have go1.11, and by default you need to be in a directory NOT in your $GOPATH for modules to work in go1.11
    • TheDong 8y ago
      Edit: https://github.com/golang/go/wiki/Modules#how-to-use-modules https://github.com/golang/go/wiki/Modules#how-to-use-modules Copy paste error I guess.
• iamgopal 8y ago
  Does context.background() works ?
  • chrisbroadfoot 8y ago
    (GCP Gopher) Yes, you don't need to use the appengine context for anything (except to access the services at google.golang.org/appengine/...) You can use the standard net/http package to make HTTP requests, for example.
• jostein 8y ago
  This is a big improvement, but the restriction that prevents streaming responses unfortunately still apply: https://cloud.google.com/appengine/docs/standard/go/how-requests-are-handled#streaming_responses https://cloud.google.com/appengine/docs/standard/go/how-requ... Would be great if that was solved.