5 ms·

Hey all - Seth from Google here. We're really excited to bring Go to Google Cloud Functions (GCF). Please try it out, give us feedback, and let us know if you h

Hey all - Seth from Google here. We're really excited to bring Go to Google Cloud Functions (GCF). Please try it out, give us feedback, and let us know if you have any questions!

vira28 8y ago
As we all know, it took Google so long to get this. Can you overview on what are some of the technical challenges faced? Just curious. Thanks.
- stewart27 8y ago
  Cloud Functions PM here. I can give some insight: * We've been running a private early access preview/alpha since last August. * This was our first compiled language on Cloud Functions, which came with its own set of challenges. * It took us a while to find the right approach for supporting dependencies (both Go Modules and vendoring are supported). Unlike other providers, when you deploy your source code, Cloud Functions will automatically install dependencies listed in your go.mod file. * Our testers gave us a ton of feedback that helped us polish the developer experience -- we identified and fixed many rough edges related to error messages during deployment/build and errors at runtime. Serverless products can be a bit opaque (since you can't just ssh into a machine), so getting this right is important. I'd like to say that there was one big, interesting challenge that we had to tackle. But the reality is that we worked through many small details that only became apparent during testing. We wanted to address these so that we could offer a high quality experience for our public beta launch. We owe our alpha testers major credit for helping us find and solve issues. Speaking of testers -- if you have feedback on the runtime, we'd love to hear from you in our Cloud Functions Beta Tester group [1]. [1] https://cloud.google.com/functions/docs/getting-support#beta_tester_group https://cloud.google.com/functions/docs/getting-support#beta...
  - vira28 8y ago
    Thanks for the response. Would like to see more granular triggers (event types). Especially with Firebase. Also, would like to see more examples with firestore.
    - stewart27 8y ago
      Which ones would you like to see? We're always looking for new use cases to support.
      - vira28 8y ago
        These are not specific to Go. firebase authentication -> user account from disabled to enabled. firebase authentication -> when a new phone number is associated. Firestore -> field level triggers. Right now we have only document level trigger.
        elithrar 8y ago
        > Firestore -> field level triggers. Right now we have only document level trigger. For what it's worth, if you have an onUpdate trigger on a Firestore database, the event you receive has the before & after state of the change, which would include field-level changes: https://firebase.google.com/docs/functions/firestore-events#handle_event_data https://firebase.google.com/docs/functions/firestore-events#... Is there a specific use-case that this doesn't handle for you? Note: Work at GCP, not on Functions.
        vira28 8y ago
        well, if I have onUpdate trigger, which means my function is going to be triggered for every update on the document (even for the fields that I don't care) You can say that triggering functions shouldn't cost you much, but thats not the way to do right. Correct me if I am wrong. My use-case is simple. Want to get my CF triggered if the value of a particular field in a doc changes. Thanks.
  - sjeanpierre 8y ago
    Is there a reason dependency management happens like this? We currently deploy Go Lambda functions in AWS with the help of the Serverless Framework and it just uploads the cross compiled Binary and not the whole project. Why wouldn't the binary be the deployed unit in this case?
    - pstuart 8y ago
      Maybe Google Native Client support? https://github.com/golang/go/wiki/NativeClient https://github.com/golang/go/wiki/NativeClient
      - chrisbroadfoot 8y ago
        No, it's not related to this. GOOS=linux and GOARCH=amd64 for GCF deployments. The sandboxing technology used is based on gVisor. We're working on arbitrary binary deployments, you can sign up for that here: g.co/serverlesscontainers Disclaimer: I work on GCP.
    - ehsankia 8y ago
      While it's a lot more work for them, and some may already have the infrastructure setup for deploying their own binary, I think having GCP handle the end-to-end there is more user friendly in general. I can quickly write a Cloud Function from any computer, without having to deal with setting up the tool chains. If you want to just run binaries, sounds like Cloud Functions isn't what you're looking for.
  - askver 8y ago
    Does this effort take you closer to the (supposed) goal of running arbitrary X86/ARM Linux binaries as cloud functions, or is that a completely different direction?
    - chrisbroadfoot 8y ago
      At the sandboxing level, it's already possible (you can upload any binary and fork/exec it from one of the supported languages). That's made possible by gVisor, which is the underlying sandbox technology used in GAE and GCF. As for making that an actual product, we're working on that, too. Sign up for the alpha here: g.co/serverlesscontainers Disclaimer: I work on GCP.
      - askver 8y ago
        Nice!
      - wasd 8y ago
        Do you know of any alphas running for App Engine? I'm interested in testing.
        chrisbroadfoot 8y ago
        There's a beta for Go 1.11 on App Engine... https://cloud.google.com/blog/products/application-development/go-1-11-is-now-available-on-app-engine https://cloud.google.com/blog/products/application-developme... If you didn't mean Go, specifically, then the alpha I linked above might be up your alley.
dstaley 8y ago
IIRC only the Node.js runtime has the ability to be emulated locally. Is this still the case? If I wanted to test my function locally, how would I go about doing that?
- chrisbroadfoot 8y ago
  Use regular "go test" for unit testing. If you're using metadata inside your function (e.g., to extract an event ID) you can use metadata.NewContext to pass into your function while testing: https://godoc.org/cloud.google.com/go/functions/metadata#NewContext https://godoc.org/cloud.google.com/go/functions/metadata#New... If you just want to execute your function, create another package main and start an HTTP server, as you'd normally do. There are some examples on this page: https://cloud.google.com/functions/docs/bestpractices/testing#functions-prepare-environment-go https://cloud.google.com/functions/docs/bestpractices/testin... We're thinking about ways to improve the local development/testing story, and are keen to hear any ideas/feedback. Disclaimer: I work at Google, and worked on this product.
  - gobuildship 8y ago
    Just do what Azure does with functions. Have used their VS Code local debug and "just worked" out of the box
wasd 8y ago
What's next on the roadmap for cloud functions? Some things I'm curious about is work related to cold starts and future run times (specifically Ruby) but happy to hear about anything.
- stewart27 8y ago
  While I can't share details publicly, I can confirm that we're actively working on new runtimes and performance improvements.
w8rbt 8y ago
I think AWS Lambda supports Go 1.x. Does GCF plan to stick to Go 1.11.x or broaden support to other Go versions in the future? What will happen to 1.11.x support when 1.12 comes out?
- sethvargo 8y ago
  We will include support for more runtimes and language versions in the future.
- chrisbroadfoot 8y ago
  We use unpatched langauge toolchains/runtimes for GCF and second-generation GAE. That means a drastically reduced time to release for new language versions. (Maintaining patched runtimes was a significant burden to making new versions of Go available on GAE.) For long-term support of various languages, see the GCP deprecation policy: https://cloud.google.com/terms/deprecation https://cloud.google.com/terms/deprecation Especially this sentence: "excluding [...] support for a programming language version that is no longer publicly supported by, or receiving security updates from, the organization maintaining that programming language" So, that means we're committed to supporting the two most recent major version releases of Go, as per the Go release policy: https://github.com/golang/go/wiki/Go-Release-Cycle#release-maintenance https://github.com/golang/go/wiki/Go-Release-Cycle#release-m... Disclaimer: I work on GCP and used to work on Go language releases.
bdibs 8y ago
Hey Seth, I was wondering if you guys ever discussed the previous AWS vs GCP performance post that was on here a while back.
- sethvargo 8y ago
  We worked with the original authors and they updated their original benchmarks. We are still compiling data and running our own benchmarks in other areas. Things were on pause for a bit as many Googlers took time off for the holidays to spend time with friends and family. I’ll check in on the team though. Thanks for the reminder!
- sofaofthedamned 8y ago
  Do you have a link to this please?
  - bdibs 8y ago
    https://news.ycombinator.com/item?id=18673541 https://news.ycombinator.com/item?id=18673541
    - sofaofthedamned 8y ago
      Awesome,thank you.
MrSaints 8y ago
This might be a very specific question, but any plans to include packages in the Go runtime for headless Chrome? I know it is already available in Node.js.
- tbpalsulich 8y ago
  The base set of system packages should be the same between the two runtimes. I work at GCP and on this project.

Hey all - Seth from Google here. We're really excited to bring Go to Google Cloud Functions (GCF). Please try it out, give us feedback, and let us know if you have any questions!

vira28 8y ago

As we all know, it took Google so long to get this. Can you overview on what are some of the technical challenges faced? Just curious. Thanks.

stewart27 8y ago
Cloud Functions PM here. I can give some insight: * We've been running a private early access preview/alpha since last August. * This was our first compiled language on Cloud Functions, which came with its own set of challenges. * It took us a while to find the right approach for supporting dependencies (both Go Modules and vendoring are supported). Unlike other providers, when you deploy your source code, Cloud Functions will automatically install dependencies listed in your go.mod file. * Our testers gave us a ton of feedback that helped us polish the developer experience -- we identified and fixed many rough edges related to error messages during deployment/build and errors at runtime. Serverless products can be a bit opaque (since you can't just ssh into a machine), so getting this right is important. I'd like to say that there was one big, interesting challenge that we had to tackle. But the reality is that we worked through many small details that only became apparent during testing. We wanted to address these so that we could offer a high quality experience for our public beta launch. We owe our alpha testers major credit for helping us find and solve issues. Speaking of testers -- if you have feedback on the runtime, we'd love to hear from you in our Cloud Functions Beta Tester group [1]. [1] https://cloud.google.com/functions/docs/getting-support#beta_tester_group https://cloud.google.com/functions/docs/getting-support#beta...
- vira28 8y ago
  Thanks for the response. Would like to see more granular triggers (event types). Especially with Firebase. Also, would like to see more examples with firestore.
  - stewart27 8y ago
    Which ones would you like to see? We're always looking for new use cases to support.
    - vira28 8y ago
      These are not specific to Go. firebase authentication -> user account from disabled to enabled. firebase authentication -> when a new phone number is associated. Firestore -> field level triggers. Right now we have only document level trigger.
      - elithrar 8y ago
        > Firestore -> field level triggers. Right now we have only document level trigger. For what it's worth, if you have an onUpdate trigger on a Firestore database, the event you receive has the before & after state of the change, which would include field-level changes: https://firebase.google.com/docs/functions/firestore-events#handle_event_data https://firebase.google.com/docs/functions/firestore-events#... Is there a specific use-case that this doesn't handle for you? Note: Work at GCP, not on Functions.
        vira28 8y ago
        well, if I have onUpdate trigger, which means my function is going to be triggered for every update on the document (even for the fields that I don't care) You can say that triggering functions shouldn't cost you much, but thats not the way to do right. Correct me if I am wrong. My use-case is simple. Want to get my CF triggered if the value of a particular field in a doc changes. Thanks.
- sjeanpierre 8y ago
  Is there a reason dependency management happens like this? We currently deploy Go Lambda functions in AWS with the help of the Serverless Framework and it just uploads the cross compiled Binary and not the whole project. Why wouldn't the binary be the deployed unit in this case?
  - pstuart 8y ago
    Maybe Google Native Client support? https://github.com/golang/go/wiki/NativeClient https://github.com/golang/go/wiki/NativeClient
    - chrisbroadfoot 8y ago
      No, it's not related to this. GOOS=linux and GOARCH=amd64 for GCF deployments. The sandboxing technology used is based on gVisor. We're working on arbitrary binary deployments, you can sign up for that here: g.co/serverlesscontainers Disclaimer: I work on GCP.
  - ehsankia 8y ago
    While it's a lot more work for them, and some may already have the infrastructure setup for deploying their own binary, I think having GCP handle the end-to-end there is more user friendly in general. I can quickly write a Cloud Function from any computer, without having to deal with setting up the tool chains. If you want to just run binaries, sounds like Cloud Functions isn't what you're looking for.
- askver 8y ago
  Does this effort take you closer to the (supposed) goal of running arbitrary X86/ARM Linux binaries as cloud functions, or is that a completely different direction?
  - chrisbroadfoot 8y ago
    At the sandboxing level, it's already possible (you can upload any binary and fork/exec it from one of the supported languages). That's made possible by gVisor, which is the underlying sandbox technology used in GAE and GCF. As for making that an actual product, we're working on that, too. Sign up for the alpha here: g.co/serverlesscontainers Disclaimer: I work on GCP.
    - askver 8y ago
      Nice!
    - wasd 8y ago
      Do you know of any alphas running for App Engine? I'm interested in testing.
      - chrisbroadfoot 8y ago
        There's a beta for Go 1.11 on App Engine... https://cloud.google.com/blog/products/application-development/go-1-11-is-now-available-on-app-engine https://cloud.google.com/blog/products/application-developme... If you didn't mean Go, specifically, then the alpha I linked above might be up your alley.

dstaley 8y ago

IIRC only the Node.js runtime has the ability to be emulated locally. Is this still the case? If I wanted to test my function locally, how would I go about doing that?

chrisbroadfoot 8y ago
Use regular "go test" for unit testing. If you're using metadata inside your function (e.g., to extract an event ID) you can use metadata.NewContext to pass into your function while testing: https://godoc.org/cloud.google.com/go/functions/metadata#NewContext https://godoc.org/cloud.google.com/go/functions/metadata#New... If you just want to execute your function, create another package main and start an HTTP server, as you'd normally do. There are some examples on this page: https://cloud.google.com/functions/docs/bestpractices/testing#functions-prepare-environment-go https://cloud.google.com/functions/docs/bestpractices/testin... We're thinking about ways to improve the local development/testing story, and are keen to hear any ideas/feedback. Disclaimer: I work at Google, and worked on this product.
- gobuildship 8y ago
  Just do what Azure does with functions. Have used their VS Code local debug and "just worked" out of the box

wasd 8y ago

What's next on the roadmap for cloud functions? Some things I'm curious about is work related to cold starts and future run times (specifically Ruby) but happy to hear about anything.

stewart27 8y ago
While I can't share details publicly, I can confirm that we're actively working on new runtimes and performance improvements.

w8rbt 8y ago

I think AWS Lambda supports Go 1.x. Does GCF plan to stick to Go 1.11.x or broaden support to other Go versions in the future? What will happen to 1.11.x support when 1.12 comes out?

sethvargo 8y ago
We will include support for more runtimes and language versions in the future.
chrisbroadfoot 8y ago
We use unpatched langauge toolchains/runtimes for GCF and second-generation GAE. That means a drastically reduced time to release for new language versions. (Maintaining patched runtimes was a significant burden to making new versions of Go available on GAE.) For long-term support of various languages, see the GCP deprecation policy: https://cloud.google.com/terms/deprecation https://cloud.google.com/terms/deprecation Especially this sentence: "excluding [...] support for a programming language version that is no longer publicly supported by, or receiving security updates from, the organization maintaining that programming language" So, that means we're committed to supporting the two most recent major version releases of Go, as per the Go release policy: https://github.com/golang/go/wiki/Go-Release-Cycle#release-maintenance https://github.com/golang/go/wiki/Go-Release-Cycle#release-m... Disclaimer: I work on GCP and used to work on Go language releases.

bdibs 8y ago

Hey Seth, I was wondering if you guys ever discussed the previous AWS vs GCP performance post that was on here a while back.

sethvargo 8y ago
We worked with the original authors and they updated their original benchmarks. We are still compiling data and running our own benchmarks in other areas. Things were on pause for a bit as many Googlers took time off for the holidays to spend time with friends and family. I’ll check in on the team though. Thanks for the reminder!
sofaofthedamned 8y ago
Do you have a link to this please?
- bdibs 8y ago
  https://news.ycombinator.com/item?id=18673541 https://news.ycombinator.com/item?id=18673541
  - sofaofthedamned 8y ago
    Awesome,thank you.

MrSaints 8y ago

This might be a very specific question, but any plans to include packages in the Go runtime for headless Chrome? I know it is already available in Node.js.

tbpalsulich 8y ago
The base set of system packages should be the same between the two runtimes. I work at GCP and on this project.