4 ms·

I've been porting a project from Ruby to Go in search of making it a bit lighter. The project is about a 1000 lines now so I'm not allowed to criticize Go yet,

by tinco 11y ago

I've been porting a project from Ruby to Go in search of making it a bit lighter. The project is about a 1000 lines now so I'm not allowed to criticize Go yet, but so far it's very nice. I picked the language up in just a few hours as I went and the whole project took just about a week and a half. So as someone who is clearly in no position to be criticizing other projects yet, isn't Heka exactly the sort of project you shouldn't do in Go? I say that because I have the feeling you should use Go only for very concrete cases, given its lack of proper abstractions. I.e. writing a tool that receives log lines over HTTP, extracts metrics and forwards those to StatsD? Perfect use case for Go. But writing a tool that lets you plug in arbitrary frontends to forwards to arbitrary backends? Perhaps they got it to work nice, but that sounds more like a case for a more general language.

windlep 11y ago
For heka, one of the things we wanted besides performance, was a small, easy-to-distribute binary you could drop on a system and 'just run'. Go is fantastic for distribution thanks to the small staticly linked binaries you can get out of it. To accommodate the pluggable filters and such later, a Lua sandbox system was added... and oddly some of the other Go pieces ended up being faster in Lua which is why Lua decoders/inputs are now an option for heka as well. Note: I gave this talk.
- mdup 11y ago
  Not advocating anything, but could C++ have been a good choice? You'd get top performance, small memory footprint, opt-in static linking, at the price of a less "fun" language and verbosity. Regarding memory leaks, it's easy to write a leak-free C++ program these days with smart pointers. However, it doesn't come "batteries included" for net-related stuff, and ASIO can give a couple of headaches.
  - windlep 11y ago
    It could've been. Though now that so much of heka is involved in moving data between the Lua layer, turning it into plain C is a better option. trink, one of the heka authors has a project called hindsight that does just that.
- methodover 11y ago
  I don't have much to add, other than I went to the talk this past Wednesday (Python meetup in SF) and really enjoyed it. You did a great job. I agree with the currently-most upvoted comment here that I really value these kinds of war stories. Making technology choices is a huge part of what I have to do on a week to week basis and it was great to hear your experience.
aikah 11y ago
> Perfect use case for Go Go shouldn't have "use cases". One should be able to do almost everything with ease with a language built in 2008/9. And unfortunatly that is not the case. Go has excellent concurrency features, but is limited by dumb language design decisions which make it painfull to test and to write good reusable and composable libraries for. I'd love to replace my entire stack with Go but I can't. Something I would write in Ruby in 10 days takes 2 months in Go. And worse, I cant write the code the exact way I want which does piss me off. Give me some choice, not random constraints. Aside from concurrency , I shouldn't have to ask myself how I write something in a specific language. This is the goal of refactoring, and it comes later. I will personnally invest in Crystal and dump Go as soon as it runs on Windows. It has channels, and that's all I need.
- Luminarys 11y ago
  > Something I would write in Ruby in 10 days takes 2 months in Go. Have you looked into Elixir? It has Ruby like syntax, but uses an actor model for concurrency(it runs off of the Erlang VM). For handling concurrent tasks it tends to benchmark around Go's speed, but is much nicer to do things in. While it is admittedly immature, the ecosystem still has a decent amount of packages and the tooling isn't bad.
  - rubiquity 11y ago
    Being a Ruby to Go to Elixir convert myself, I can only second this. Elixir is a really great language.
- larzang 11y ago
  > Go shouldn't have "use cases". One should be able to do almost everything with ease with a language built in 2008/9. Assuming that Go is a general-purpose high-level language, yes, but is it? It was created to replace C++ in critical infrastructure, not Python/Ruby as the end-all be-all of default platforms for every situation. Its syntax simplicity and speed absolutely makes it attractive to a wider audience, sure, but if it does the job it was designed to do very well, should we be angry that it doesn't do all jobs well? Developer time and execution time are both important metrics when considering a language, and Go is very well situated when the major developer time gains offset the minor execution time losses vs C/C++. That it's less well situated when the developer time losses vs Python/Ruby are incurred on a project when the execution time gains are irrelevant isn't a failing of the language, it's a trade-off.

by tinco 11y ago

windlep 11y ago

For heka, one of the things we wanted besides performance, was a small, easy-to-distribute binary you could drop on a system and 'just run'. Go is fantastic for distribution thanks to the small staticly linked binaries you can get out of it. To accommodate the pluggable filters and such later, a Lua sandbox system was added... and oddly some of the other Go pieces ended up being faster in Lua which is why Lua decoders/inputs are now an option for heka as well. Note: I gave this talk.

mdup 11y ago
Not advocating anything, but could C++ have been a good choice? You'd get top performance, small memory footprint, opt-in static linking, at the price of a less "fun" language and verbosity. Regarding memory leaks, it's easy to write a leak-free C++ program these days with smart pointers. However, it doesn't come "batteries included" for net-related stuff, and ASIO can give a couple of headaches.
- windlep 11y ago
  It could've been. Though now that so much of heka is involved in moving data between the Lua layer, turning it into plain C is a better option. trink, one of the heka authors has a project called hindsight that does just that.
methodover 11y ago
I don't have much to add, other than I went to the talk this past Wednesday (Python meetup in SF) and really enjoyed it. You did a great job. I agree with the currently-most upvoted comment here that I really value these kinds of war stories. Making technology choices is a huge part of what I have to do on a week to week basis and it was great to hear your experience.

aikah 11y ago

> Perfect use case for Go Go shouldn't have "use cases". One should be able to do almost everything with ease with a language built in 2008/9. And unfortunatly that is not the case. Go has excellent concurrency features, but is limited by dumb language design decisions which make it painfull to test and to write good reusable and composable libraries for. I'd love to replace my entire stack with Go but I can't. Something I would write in Ruby in 10 days takes 2 months in Go. And worse, I cant write the code the exact way I want which does piss me off. Give me some choice, not random constraints. Aside from concurrency , I shouldn't have to ask myself how I write something in a specific language. This is the goal of refactoring, and it comes later. I will personnally invest in Crystal and dump Go as soon as it runs on Windows. It has channels, and that's all I need.

Luminarys 11y ago
> Something I would write in Ruby in 10 days takes 2 months in Go. Have you looked into Elixir? It has Ruby like syntax, but uses an actor model for concurrency(it runs off of the Erlang VM). For handling concurrent tasks it tends to benchmark around Go's speed, but is much nicer to do things in. While it is admittedly immature, the ecosystem still has a decent amount of packages and the tooling isn't bad.
- rubiquity 11y ago
  Being a Ruby to Go to Elixir convert myself, I can only second this. Elixir is a really great language.
larzang 11y ago
> Go shouldn't have "use cases". One should be able to do almost everything with ease with a language built in 2008/9. Assuming that Go is a general-purpose high-level language, yes, but is it? It was created to replace C++ in critical infrastructure, not Python/Ruby as the end-all be-all of default platforms for every situation. Its syntax simplicity and speed absolutely makes it attractive to a wider audience, sure, but if it does the job it was designed to do very well, should we be angry that it doesn't do all jobs well? Developer time and execution time are both important metrics when considering a language, and Go is very well situated when the major developer time gains offset the minor execution time losses vs C/C++. That it's less well situated when the developer time losses vs Python/Ruby are incurred on a project when the execution time gains are irrelevant isn't a failing of the language, it's a trade-off.