14 ms·
Show HN: Go Micro – A distributed systems development framework
- throwerawa174 6y agoNot dissing your efforts, but you keep posting your Go Micro stuff, under different account names (chunk) in Show HN and normal threads. 90% of these posts add nothing and are just duplicates. Maybe add a blog post or something of value instead of the same old link every time.
- asim 6y agoThanks for your comment. I post a link to the github repo periodically because the project is moving incredibly fast and actively evolving to become something that's fundamentally valuable to those building distributed systems, cloud services and want to build applications with Go using a framework. People also come and go on HN so don't always get a chance to see it or dont' remember the project. I'm also always actively looking for feedback and so appreciate your comments and will look to publish a blog post next time with valuable content as I've done in times past on the blog https://micro.mu/blog/ https://micro.mu/blog/. Otherwise as someone who started this project 5 years ago as his first open source endeavour and was really scared about the reaction from a developer community about the quality of my software or whether there would be any interest at all, I'd hope that positive encouragement is something we advocate for rather than negativity. And I hope others are not disuaded by your comment and actively post on HN to get positive feedback on their project! Otherwise thanks for your comment. Oh and also now being an old man I wanted to use an account with my real name rather than a pseudonym :)
- lugged 6y agoDon't apologize for selling yourself. If they're complaining, its working.
- asim 6y agoThank you :)
- jbverschoor 6y agoAka spam
- giancarlostoro 6y ago> the project is moving incredibly fast and actively evolving Not that I'm hating, but one of the comments above made a valid point that this part of your comment suggests is correct: maybe you should be blogging about progress of the project and gain plenty of eyeballs? One thing I saw that fascinated me is you guys are using mDNS for discovery, I was debating this discovery approach for a small microservices foundation for D but I couldnt even find a decent library so I havent really attempted much work in that approach. I did something that in a sense feels like a micro service architecture entirely with mDNS but in Python, then months later I worked with microservices in Java and it felt so bloated and overcomplicated compared to mDNS.
- parhamn 6y agoIf you can only confront someone using a throwaway account, you’re probably being unnecessarily mean. Glad they’re trying to get word out there on what they’re working on. Let the voting do the voting. Thank for your OS work OP, upvote from me. Looks like a cool project!
- asim 6y agoThank you!
- jiofih 6y agoI understand the positive viewpoint, but don’t feel it’s in the spirit of HN to repost your own project every couple months. Back in the day even just posting your own stuff (instead of someone else who found it interesting) was frowned upon. IMO a culture of curiosity vs blatant promotion is an asset for this board and should be preserved. I counted at least 26 reposts of go-micro. At what frequency should we just consider it a recurring (free) ad? https://news.ycombinator.com/submitted?id=chuhnk https://news.ycombinator.com/submitted?id=chuhnk
- city41 6y agoWhen has self promotion here been frowned upon? It’s even officially supported with “Show HN” posts.
- parhamn 6y agoI don’t disagree but that is the exact purpose of forum software: voting, spam detection, etc. The fact that 99.999% of HN users had no idea these posts were made (most have no votes) shows it’s working correctly. On the other hand, to a large extent getting traction on HN is a somewhat random process reliant on a few high momentum votes in the early drop. A lot of the stuff we see on the front page is pushed by voting rings. Anyway given this persons hard work on a project and desire for traction, I’d rather they try/learn to market at the expense of a couple of bytes on a DB and a few scanning eyes on /new. And it seems like this post might be their best yet :)
- 6y ago
- ABS 6y agogenuinely curious to know why you created a throwaway account solely to post this?
- city41 6y agoThat is very common practice on HN. Submitting a link here is a lottery that one rarely wins.
- bsaul 6y agoit looks interesting, but unfortunately the doc doesn't provide much beyond general diagrams. As an example, i wonder what are the properties of the pub/sub system in terms of delivery guarantees and tradeoffs. Or, if the goal is to have a third-party system implement the interface and just plug-it in, then at least give a list of available wrappers ( eg : "we currently have wrappers available for rabbitmq or kafka if you want to use them as your pub/sub system").
- asim 6y agoHi, author here. Thanks for your comment. You're right the readme doesn't go into detail and that's on purpose. I only wanted to provide a high level summary there. I need to do better and write some docs. There is a development repo where we're trying to slowly flesh out design docs https://github.com/micro/development https://github.com/micro/development. Please let me know if any of that helps and what we can add. On the specific thing you're talking about. Yes exactly. We want to define high level abstractions for distributed systems which are pluggable. The idea being that we can't replicate the specific complexity and absolutely phenomal work of the teams building rabbitmq, kafka, etc but we can provide abstractions on top to simplify the developer experience and make it pluggable so those who want to use rabbitmq can but those who want to use kafka can swap that out instead. The built in broker interface at the moment focuses on lightweight pubsub, notifications and events in a way that's fire and forget moreso than delivery guarantees, queuing, offsets, etc. You can use something like kafka or rabbitmq to get these guarantees but we want to write an Event interface that will be more inline with what people expect from that requirement also. Hope that helps!
- mrath 6y agoI do a little hobby work in Go. I looked at micro several times. For some reason I felt it is bit too heavy or unnecessary complexity. It does not feel fit in go ecosystem. But may be a lot of people are liking it. I could be wrong because I have not done much real work in Go. Does anyone use it for production apps?
- asim 6y agoI think the Go community in its early years advocated for libraries over frameworks. 1. Because of the very powerful standard library which dictated the development model and culture of the community. 2. Because Go was still such a young project. I think its only natural over time that frameworks and tools emerge to simplify the development experience in every language. We realistically cannot ask people to piece together complex systems in that way. If you imagine the thing you will end up writing in 200-300 lines of code in your main function, we're just encapsulating in a single function for you. The thing most companies end up building, a common shared library is by any other name a framework and so we're just piecing together that common experience for everyone else. Now the key thing to note is you can use any of these packages independently. In fact you can think of Go Micro as a standard library for distributed systems development. Its just that at the top level we import the common features and create the construct of a "Service" and give you one way to initialise it with `micro.NewService`. But the great thing is you have the freedom of choice to use things like go-kit, go-cloud, or any of the other thousands of tools in the ecosystem :)
- atmosx 6y agoThe “powerful standard library” to my experience is a myth that holds no water. Even trivial projects use external packages to handle trivial things like an HTTP server or logging. No project I saw, however small, uses just the standard library.
- frou_dh 6y agoWell objective data on this is out there if anyone can be bothered trawling through it: https://github.com/search?o=asc&p=4&q=language%3Ago&s=stars&type=Repositories https://github.com/search?o=asc&p=4&q=language%3Ago&s=stars&...
- wdb 6y agoI am wondering if a similar framework exists for Node.js or Deno? I am really interested in Go but I have a lot of learning to do before I am confident to write solutions with it.
- pfalafel 6y agoYou advertise "a powerful pluggable architecture." But Go binary plugins cannot be unloaded. At least not yet. How do you overcome this?
- asim 6y agoGo Micro is a framework/library as opposed to a binary or server. You import the framework in code and it gives you the ability to add additional plugins via new imports. These can then be swapped out via environment variables or flags. We make use of Go interfaces to define the abstractions so all you have to do is create a plugin that implements the interface and that effectively makes it pluggable. Here's the community led Go Micro plugin effort https://github.com/micro/go-plugins https://github.com/micro/go-plugins
- pfalafel 6y agoWhat exactly does it mean to say that plugins "can then be swapped out"? I assume a binary Go plugin can not in fact be unloaded from memory. So modifying and reloading a Go plugin will result in multiple versions of it to then stay in memory. This is a shortcoming of the current Go implementation. Is my understanding correct?
- konart 6y agoI think you are confusing whatever go micro is offering with https://golang.org/pkg/plugin/ https://golang.org/pkg/plugin/ I'm pretty sure go micro's plugins are just additional packages each for its purpose ready to be imported and used in your project. Basically a collection of community maintained packages for the most common utils\middleware\MQ clients etc
- asim 6y agoDo not assume "Go Plugin" in this context. We are not using the Go plugin system. Plugin is a name used to describe something that can be "plugged in" or that is "plug and play". In our case as mentioned, we use the Go interface as an abstraction and the "Plugin" is an implementation of that interface which can replace what you're using. You will import plugins for rabbitmq, kafka, etc as a Go import. You can then use env vars and flags to choose which you want to use. Internally we are taking an initialiser for New[Plugin] from a list and executing it to set it up. There is no Go language plugin loading, its all still compiled code. Its just how we use the abstraction. Please see the readme for our plugins for more detail https://github.com/micro/go-plugins https://github.com/micro/go-plugins
- npiit 6y agoDunno why the adversity in the comments. Great job, OP. I think there is a real need for something like this as the GitHub stars show, I considered using Kubernetes itself as a distributed infrastructure but it may be a bit heavy for many use cases. I think this can be a serious alternative for distributed systems that do not have intention to be built over Kubernetes.
- latenightcoding 6y agoReminds me of https://github.com/atomix/atomix https://github.com/atomix/atomix
- eatonphil 6y agoWe used go micro in a production system for a while at a previous company. The nicest thing I found is that it abstracts a few common pubsub systems. But besides that I found it overly abstract. It's not really that hard to write glue for messaging systems as needed and that's the only part of go micro we were using.
- chuhnk 6y agoYou're right. It's not very useful if you're just using it for pubsub. Go Micro was always intended as a full fledged framework for distributed systems development and you'll find if you build systems at scale you end up developing a similar framework in house. For those just using messaging or a database my advise would be to go directly to that system. Abstractions are only useful when they work together collectively for the foundation of something more. In our case when everyone is building Micro services we can actually run these absolutely anywhere regardless infrastructure and share them with anyone as a reusable building block.
- wpietri 6y agoAre these things necessarily opposed? It doesn't seem impossible to me that a "full-fledged framework" also addresses the most common uses cases in ways that don't leave people befuddled.
- asim 6y agoI think its really a matter of opinion and choice of development. Some people want to build it themselves, others would prefer to use a standard. Its the difference between using Spring or writing tools around Java. Using small libraries or Ruby on Rails.
- wpietri 6y agoAgain, I don't think they are necessarily opposed. The examples you give are, sure, but that's a library vs framework distinction. As a contrary example, take a lot of Unix command-line tools. Things like ls and ps have very sensible defaults that cover up very complex models. Or my experience with Python's Twisted is that it is very rich, but it's simple to do simple things. And cryptography libraries are a great example of where well-chosen defaults are absolutely vital. Same with Wireguard. Is what it's up to very complex? Definitely. Do I need to understand the details to get good results? No.
- aloknnikhil 6y agoPrevious discussion: https://news.ycombinator.com/item?id=21878659 https://news.ycombinator.com/item?id=21878659
- aloknnikhil 6y ago> Service Discovery - Automatic service registration and name resolution. Service discovery is at the core of micro service development. When service A needs to speak to service B it needs the location of that service. The default discovery mechanism is multicast DNS (mdns), a zeroconf system. > Load Balancing - Client side load balancing built on service discovery. Once we have the addresses of any number of instances of a service we now need a way to decide which node to route to. We use random hashed load balancing to provide even distribution across the services and retry a different node if there’s a problem. Trying to understand this a bit here. Is micro a replacement for k8s? Because I don't see any deployment capabilities here. So I'm assuming it's complementary to that. If that's the case, then I'm not sure how service discovery and load balancing would co-exist with the CNI in k8s.
- asim 6y agoNot everything is k8s centric but also k8s also has very rudimentary models for discovery and load balancing, this is why things like service mesh have emerged to handle those challenges. As a framework Go Micro builds in client side discovery and load balancing which are completely separate to any third party system. You can imagine the next distributed database to be written with Go Micro which can operating independent of k8s. In the case of k8s a lot of people swap out the load balancing for something like dns or integrate it with an envoy or linkerd considering our protocol usage is gRPC. But otherwise our service discovery acts as a central registry for all services. In a development centric environment this allows us to share feature rich metadata along with endpoints, description, and anything like team info, pager details, etc. All purely written in code rather than through yaml.
- aloknnikhil 6y agoAgreed. I'm not saying that this needs to be integrated w/ k8s. I'm merely comparing it to k8s which promises the same abstraction albeit declaratively. The biggest plus point, IMO, for k8s is it makes managing deployments across physical hosts easy. So, I was asking if there was anything equivalent in micro that would help manage my deployments. And your comment on client side discovery and Load balancing partly addresses my original question. My main point here being, I don't see anything in the linked doc on why I'd choose micro over existing frameworks. EDIT: https://github.com/micro/micro/blob/master/README.md https://github.com/micro/micro/blob/master/README.md This page is more informative on why Micro over anything else. So I think this answers my second question.
- solarkraft 6y agoWhy is "microservice" not mentioned anywhere on the page? Sure, it's not a great term, but it would make identification of its goal immediate instead of taking a few seconds.