13 ms·
Portable Cloud Programming with Go Cloud
- alongwittheflow 8y agoWhy is Google advertising its own product through official golang blog? It looks very amateurish.
- spraak 8y agoHow so? Go Cloud works with AWS and soon others. That seems pretty mature to me.
- stirner 8y agoBecause it's not a Go project, it's a Google project[1]. It's like Apple posting new iOS features on the Swift blog, e.g. [1] https://github.com/google/go-cloud https://github.com/google/go-cloud
- tomcam 8y agoFWIW I’m the target market and delighted to learn about it on the blog.
- MoOmer 8y agoThe examples reference the use of an Amazon Web Service product for the abstraction. I don’t see a problem here.
- pkaye 8y agoThe primary example they gave was on AWS.
- spraak 8y agoMaybe it's a biased and distorted perception that I have, but Go seems to be the most consistent and unsurprising language I've used, and I mean this positively. I haven't written any Go for over a year since my current job mostly uses Node, but I feel immediately comfortable jumping in to the examples in this blog post. I don't think that I'm explaining what I mean very well, but it's something I don't think is so true for Node/JS and other languages I've used.
- pkaye 8y agoOur company uses Go. Most people who join have no experience in Go yet usually they can get comfortable with the language in 1-2 weeks to start contributing. I know nothing about Node/JS but I'm sure that takes much longer?
- nostalgeek 8y ago> but I'm sure that takes much longer? Your people can pick up go reflect package or go routines/channels and co in 1-2 weeks and start contributing? impressive... most people can pick up for loops, if,switch statements, variable and function declaration in a week in any language as well... In fact, a lot of non programmers use JS regularly to write scripts in the browser. I imagine your people know how to program before hand... So what do you think takes longer to learn? mastering go threading model or being able to add interactivity to the DOM?
- golangnews 8y agogo reflect is not commonly used, goroutines are pretty simple to pick up. I have used several large codebases with zero use of reflection. The point of go is it restricts you to 25 keywords (the basics that you sneer at above) so senior people can't get fancy and produce an impenetrable mess/marvel that beginners will take months to orient themselves in. It is definitely easier to write well than js IMO, and harder to make a big ball of mud in.
- deleted 8y ago[deleted]
- pkaye 8y ago> So what do you think takes longer to learn? mastering go threading model or being able to add interactivity to the DOM? For me, the go threading model is easier. I just can't stand DOM and web programming in general. Currently working on a problem doing some computation across dozens of cores on multiple servers and go makes it a lot easier to spawn the 1000 goroutines and coordinate between them. I guess everyone has different interests and perhaps it is similar for those we hire.
- NiceGuy_Ty 8y agoSeems like a very useful library. Any similar libraries for other languages?
- spiderPig 8y agoIsn’t this just centralizing the cloud provider SDKs? Instead of AWS, GCP, Azure designing their own SDKs, they have a common contract to adhere to. I’d imagine as the cloud becomes more mainstream, you’ll see similar support in Java and .Net (a System.Cloud package for example)
- colek42 8y agoIt looks like it, but think of the power. If you write your app for S3 and Amazon raises their prices 100x you can be running on GCP in no time. Of course, you just create interfaces for your data model and implement them with your favorite service, but nobody has time for that.
- sciurus 8y agoIn this specific case you'd already be able to do that, since GCP's Cloud Storage service supports S3's API. https://cloud.google.com/storage/docs/interoperability https://cloud.google.com/storage/docs/interoperability
- gnufied 8y agoThere is Fog library for ruby.
- ryukafalz 8y agoIt's a lot like Libcloud for Python: https://libcloud.apache.org/ https://libcloud.apache.org/
- sciurus 8y agolibcloud (python), jclouds (java), and fog (ruby) are three that come to mind.
- stcredzero 8y agoI'm working on a new kind of container that allows for lambdas with very-very low latency. (Suitable for running a game loop.) This is absolutely awesome! Programming against the auto-generated AWS golang APIs is pretty painful. It works, but it's not idiomatic and overly verbose. I'm going to give it a try and see if switching to GCP is that easy.
- tmpz22 8y agoFWIW there is an AWS-go-sdk-2 our there that’s more idiomatic.
- audi0slave 8y agoHave you looked at this: https://www.usenix.org/system/files/conference/atc18/atc18-oakes.pdf https://www.usenix.org/system/files/conference/atc18/atc18-o...
- didip 8y agoEven though this is nothing but a centralized client-side libraries, it allows Go developers to live in a multi-cloud environment sooner. I am pretty excited about it.
- cflewis 8y agoHey ho. I worked on this project. The majority of the team is in SF for Google Cloud Next to present this. I'm about to hop into the car to head up myself to watch, so I can do a little AMA throughout the day. Reply to this to make sure I see your question! Here's a question I've already seen: > Is this a Google project or a Go project? It is a project led by the Go team, and we are very aware of our responsibility to be cloud provider neutral. At the same time, while the project is so new and evolving so rapidly, we need to have working examples that we can change atomically in a single repository (atomic changes necessary for rollbacks and such). We don't want to have cloud provider-specific code in the Go project, so the code is currently sitting under the Google GitHub org. When the interfaces to different products stabilize, the intention is to graduate them up to the Go GitHub org. Then anyone can code implementations to that interface for whatever cloud they like in separate repos, similar to the way `database/sql` is structured. The cloud provider-specific code will always live outside the Go project. But things are very new, so change is possible.
- gant 8y agoI'm a little bit concerned because the last package that Google seems to have purpose-built for Go+Google (AppEngine?) was context, that gave us things like context.Context, a generic KVS that needs reflection, heavily relies on convention and doesn't work in linear time and "just make every function take a context and use context.TODO if you don't need it". It also seems to be one of the biggest points of controversy for Go 2, with calls from "everything should have a context" to "context should go away".
- ironjunkie 8y agoyeah, that context thing is a huge mess. The libraries that support it have now all their function signature way longer, and it is not well understood if it is a "Request for cancellation" or something else
- atombender 8y agoContext is a pet peeve of mine. It's a design mistake as implemented, even if it serves an important function. It infects everything, forcing every method to pass contexts around, and it conflates multiple separate concerns (cancellation, deadlines, and key/value data). Given that all of a Go program sooner or later will need contexts somewhere, it would be much cleaner to let goroutines have this stuff implicitly. I don't know enough about the intervals to say whether this is feasible with the current runtime, though.
- arendtio 8y agoI like to see how Go seems to getting some traction within Google. For years, I had the impression that Google was more Java/JS/Python driven and I was wondering why they weren't using their own language for more projects :-D
- robbrit 8y agoGoogle is actually C++ centric - every other language is a bit of a "second class citizen" when interacting with infrastructure. Go is still new and Google is "old", so there is surprisingly little Go code at Google - it doesn't make sense to rewrite critical infrastructure just because you have a shiny new language. The derivative is positive though, and slowly Go is being used for more and more new development.
- erik_seaberg 8y agoWhen I left, Google was nearly 50/50 C++/Java with some lingering low-volume Python. Language-neutral RPC was key to not being stuck in one language.
- pjmlp 8y agoI can assure you that on Android world C++ feels pretty much "third class citzen".
- skybrian 8y agoYep. Google has built multiple ecosystems and they each have their own preferences. The grandparent was talking about Google's internal, server-side ecosystem. People working in that area sometimes assume that's it's the default. Client-side development is a whole different thing. It's fragmented between browser, Android, and iOS. (And each of those is further fragmented, with multiple languages available.)
- pinkgodzilla 8y agoI wonder what kind of monitoring and alert system must exist if any cloud provider changes their SDK or APIs (signature, logic, capability etc.) upstream while your go binaries are in production. Did Google manage to get cloud providers on board to register and update a 1 to 1 match between the upstream api and the go cloud api? Secondly on the article's mention on multi cloud usage, you would need to be a pretty large place to need (or even bothering to assess and convince yourself that you need) to use multiple cloud providers at once. Just learning and tweaking settings in the cloud providers GUI console is half the battle won sometimes. Saying all this, this, together with the data portability announcement, it is definitely great for competition and going to bring great resilience in your code base to be SDK agnostic.
- deleted 8y ago[deleted]
- fishywang 8y agoI don't think cloud providers would just change their SDK or APIs without notice. It will break anyone using their SDK, not only go-cloud. For your second point, it's normal for small startups to shop around and pick the cloud provider that's more suitable for their needs. For example on my last job at a small startup, we switched from AWS to GCP for cost reasons, and I implemented an abstraction (similar to go-cloud's) between S3 and GCS because of that.
- pinkgodzilla 8y agoThey never change over the course of say 3 years? 5 years? If it changes frequently that's great - someone could be put to task to monitor. If it changes very rarely, you will have a surprise outage although I am guessing debugging and fixing it would not be an issue unless you have SLAs you need to keep up with.
- fishywang 8y agoBy "change" I mean "breaking change". Backward compatible changes doesn't matter in the original question.
- pwaai 8y agoWhy should I learn go and use go cloud over AWS SDK?
- barrongineer 8y agoThey are trying to promote portability.
- TheBlight 8y agoLooks like they're trying to promote ease of moving customers from AWS to GCP to me but sure.
- skybrian 8y agoInteresting that they built a new dependency injection framework, much like similar things in Java. I guess this doesn't replace Context, though? https://github.com/google/go-cloud/tree/master/wire https://github.com/google/go-cloud/tree/master/wire
- enjoylife 8y agoDoesn't replace context, and neither is it alone in terms of being a DI library, [1]. [1] https://github.com/uber-go/fx https://github.com/uber-go/fx
- latchkey 8y agoAs a Java engineer, I really missed DI in golang. Then I found fx. Once I wrapped my brain around fx, I really liked it and used it heavily in a project I wrote. It was especially good for initializing 'things' of the same 'type'. Like a bunch of controllers for a web app. One nice thing about fx is that it does not rely on code generation (uses reflection instead). I'm very curious what other features wire has over fx.
- gant 8y agoOh no, not another code generator. That seems to be a common thing in Kubernetes too, if you don't want to spend the time writing actual code and Go is too "boilerplate heavy" for you, just write a code generator and bring the complexity of other languages right back!
- liveoneggs 8y agoironically terraform is already written in golang. https://github.com/hashicorp/terraform https://github.com/hashicorp/terraform Go 96.1%
- pageald 8y agoTerrafirm isn't provider neutral. It is just a declarative wrapper for cloud provider APIs. Code using one provider is not easily translated to another. I do think it's an excellent configuration management and provisioning tool, but Go Cloud solves a different problem.
- cflewis 8y agoYes, we are solving a different problem. Terraform is about provisioning resources, Go Cloud is about interacting with them. We actually use Terraform for provisioning in our samples, and we see the two working hand in hand.
- erik_seaberg 8y agoA frustrating choice. I spend a few minutes per week running Terraform on a recent laptop. If it were written in Python, it might use more memory and require a few dependencies (which I don't care about at all) but then I could write plugins without having to deal with IPC and weird language-specific serialization (which Java should have convinced us is a bad idea).
- liveoneggs 8y agoheh I thought you were making a funny sarcastic joke and then realized you were serious :)
- smnplk 8y agoerr, are you not nil ? err, are you not nil ? err, are you not nil ? err, are you not nil ? err, speak to me...
- smnplk 8y agoI need one more downvote :)
- bhamta 8y agoI like how Google tries to masquerade their corporate goals behind the 'portability' banner. Google Cloud is in a distant third place behind AWS and Azure, so ofcourse they don't want developers to have tight coupling to their competitors. This portable cloud strategy is a mirage. I haven't seen anyone migrate providers easily (we tried in my last company). In reality, you have to break the cloud agnostic abstraction to leverage provider specific functionality. And then you are stuck with unmaintainable mess of 'cloud agnostic' APIs mixed cloud specific APIs.
- mfer 8y agoNote, Azure support is filed under "unplanned"... https://github.com/google/go-cloud/issues/76 https://github.com/google/go-cloud/issues/76 Interesting that the #2 cloud provider is listed as unplanned in a library coming out of the Go team.
- spraak 8y agoI mean, AWS is #1 so that makes sense, and then their own platform as their second choice to implement, that makes sense too. I don't think it's like a conspiracy that they haven't released the Azure support yet.
- vsekhar 8y agoSee: https://news.ycombinator.com/item?id=17604358 https://news.ycombinator.com/item?id=17604358
- skybrian 8y agoHaving some common types available seems useful for people writing portable libraries. Maybe it's not going to change the world for most developers, but it could help defragment the library ecosystem a bit.
- pietroglyph 8y ago> I like how Google tries to masquerade their corporate goals behind the 'portability' banner. This may further Google's corporate goals, but that doesn't mean it wont actually increase portability, or that it's actually a bad thing for people who want to avoid lock-in. > In reality, you have to break the cloud agnostic abstraction to leverage provider specific functionality. I agree this is a difficulty, but I think that it's also avoidable if you make a disciplined commitment to interoperability. Just as you have to put in the effort to use this new "portable" API, you also have to choose not to defeat the former investment. Saying that this API is useless because you end up using provider specific functionality anyway is like complaining loudly that something is broken when you're using it wrong in the first place.
- kissgyorgy 8y agoMy understanding about a cloud provider lock in that it is a well thought-out, conscious decision (at least it should be) and after you made that step, you can use the specific services of the given cloud provider and you never move out. Are those hybrid/mixed cloud solutions or move between cloud provider viable or that common? Is there anybody went through a huge cloud migration which went well? If you need to move, there are huge amount of data you have to move for example between S3 and Google Cloud Storage, is that even possible to do beyond an early stage? I can't imagine.
- ljnelson 8y agoIs this a Go-flavored version of something like https://jclouds.apache.org/ https://jclouds.apache.org/?
- jhabdas 8y agoIt took us a while to get here. But it appears as if we've finally recreated the guestbook. And now it runs on someone else's machine.
- la6470 8y agoWait until Microsoft introduces their voodoo magic into their APIS that it would be impossible to write wrappers around them.
- dschulz 8y agoI bet y'all would enjoy reading the article after installing this Chrome Extension [Cloud To Butt Plus](https://chrome.google.com/webstore/detail/cloud-to-butt-plus/apmlngnhgbnjpajelfkmabhkfapgnoai https://chrome.google.com/webstore/detail/cloud-to-butt-plus...).
- solipsism 8y agoAm I the only person not excited by this? It seems to be based on the idea that you can just paper over the chasms between vastly different implementations. Putting aside the fact that the differences are important and you can't ignore them, doing this well is simply not scalable. How could it ever be all of: comprehensive, correct, up to date, elegant?
- pjmlp 8y agoYou're not alone. This is like those DB abstraction layers because "we need to change DB", but in the end almost no one doesn't the products lifetime. Also given that most of our clients don't use these fashionable cloud APIs, rather their own infrastructure, makes also not that much relevant to me.
- freespirit77 8y agoWill this compete directly with something like Terraform?
- sandGorgon 8y agoIs this the equivalent of Terraform +Boto3 in the python world ?
- dev_dull 8y agoSimple NCSA-style logging webserver. So important! I can't believe this wasn't part of the standard lib.
- h8liu 8y agoPackage list with dependency graph @smallrepo.com https://smallrepo.com/repos/github.com/google/go-cloud https://smallrepo.com/repos/github.com/google/go-cloud
- kjgkjhfkjf 8y agoDisgustingly ingenuous! Any language can be used to implement the factory design pattern.
- hiacynt 8y agoOh no, it comes with its own dependency injection framework. This is just bad, non-idiomatic Go.
- bborud 8y agoThis is tempting in the same way ORMs were once tempting. But just like with ORMs, this is solving the problem at the wrong abstraction level.
- devj 8y agoHow is it different from Pulumi(https://www.pulumi.com/ https://www.pulumi.com/)?