38 ms·

3.5 Years, 500k Lines of Go

326 points by natefinch 10y ago

zalmoxes 10y ago
Great point about time. At work we've adapted github.com/WatchBeam/clock and it's helped a lot. Thanks for blogging about your work on juju! Despite Go already being five years old, many of the patterns around building large applications are only emerging now.
- akerro 10y ago
  Go was released in 2007, it's 10 years old. Rust is closer to be 5yo, it's 7yo according to wikipedia.
  - bpicolo 10y ago
    Wasn't released to public at all until late 2009.
  - jimktrains2 10y ago
    It only became stable in mid-2015. https://blog.rust-lang.org/2015/05/15/Rust-1.0.html https://blog.rust-lang.org/2015/05/15/Rust-1.0.html
    - NateDad 10y ago
      Comparing 1.0's is a much better comparison, except in cases where there was huge adoption before 1.0.
      - jimktrains2 10y ago
        I wouldn't say rust was hugely adopted before 1.0 (not that it's hugely adopted after, either).
        NateDad 10y ago
        Yes, sorry, wasn't saying that either Go or Rust were an exception, just trying to forestall the inevitable "but what about X thing that was used by everyone and is still 0.9" :) For Go vs. Rust, I think a comparison of 1.0 releases is perfect.
        stymaar 10y ago
        I think gp meant that for some platforms it can make sense to count even 0.x releases (nodejs is the best example I think), even though for most it doesn't (including Rust and Go).
  - f2f 10y ago
    Go was announced to the world on November 10th, 2009.
- weberc2 10y ago
  This is a minor nit, but it seems to me that it would have been easier to configure the unit with a shorter timeout duration rather than mocking out the time functions. Am I mistaken?
  - zalmoxes 10y ago
    It really depends. You might want to test something that requires days/weeks expiration. I would agree that the Clock interface is a bit large. In personal projects I prefer to just have something simple like timer func() time.Time
    - weberc2 10y ago
      I don't understand your argument. What's special about a days/weeks that would prevent you from shortening this value for your tests? In other words, if I want to test my timeout logic, it should be independent of the timeout value, so I should be free to use a shorter timeout value in my timeout tests.
  - jerf 10y ago
    Mocking out the time functions means you don't get any race conditions. If you try to just twiddle with how long the time functions run, you still never quite know how long to set them. Bear in mind that even rather generous timeouts like a full millisecond to set a map entry in another goroutine can still fail if your system's CPU is loaded or the system is in swap, and spurious test failures are spurious test failures regardless of their cause. Often with this sort of code you're testing in one goroutine and the test code is in one or more other goroutines. If your test code can emit something down a synchronous channel, then your test code can also be sure it is staying in sync. I mean, even to test with "extra special small test timeouts" still means you're writing in extra test structure, you might as well just use one of these time mocking things. (In fact, you should generally always use time mocking libraries. In all languages, direct access to the system clock that can not be easily redirected for testing is at least a code smell and could possibly even rise to the level of "antipattern".) I have a lot of places in my Go test code where my test code is deliberately pushing things down channels. It's one of the top reasons my test code often still ends up being in the same package, so it gets private access to those channels for safe, properly-sync'ed testing. (Not that I'm really all that concerned about trying to test only the public interface; I don't have a lot of troubles with that anyhow. YMMV.) I also have some places in my code where I have channels in the private interface of some goroutine server whose sole purpose in life is to sync with the tests. This generally appears when I have some server that I am sending a message that I expect to change the state of the server, but for which there is no reply. In order to verify that the proper changes have occurred in the data structures of the server, I need to sync with the change before I do the check. Having a simple struct{} channel whose sole purpose in life is to synchronize works well enough for that.
therealmarv 10y ago
What do people think about the future of Juju?
- brianwawok 10y ago
  Been a few years, and it was cool in the day, but very complex. With k8 now, not sure why one would choose Juju first. They are slightly different problem domains, but not exactly.
- alsadi 10y ago
  For automation of deployments we prefer ansible (ex. it's agent-less) For container orchestration it's clear that kubernetes is the winner (if you have k8s deployed you don't need juju) For managing the whole thing manage iq have much to offer.
  - noir_lord 10y ago
    I like Ansible and use it for deploying vagrant setups and such but I've found quite a few things broken enough that I'm wary of using it to do deploy production stuff for quite a while I had issues with apt: so much so I ended up using shell, last time I wrote a new playbook they had resolved that though so things do get fixed.
ben_pr 10y ago
I'm looking forward to my first project with GO. It appears to offer a lot with minimal complexity. > Because Go has so little magic, I think this was easier than it would have been in other languages. You don’t have the magic that other languages have that can make seemingly simple lines of code have unexpected functionality. You never have to ask “how does this work?”, because it’s just plain old Go code. That lack of magic and his comparison to C# sounds like a really good mix.
- kasey_junk 10y ago
  > That lack of magic I have a really hard time understanding what people mean when they say magic. In every language I've ever worked in I spend a fair bit of time saying "how does this work". Go doesn't seem any different in that regard to me.
  - johnfn 10y ago
    When people say "magic", what they often mean is "code over here can effect the execution of code over there in an implicit way". Like in Ruby, I could conditionally monkey-patch a function into an object someone way over there was using, causing code to break. Other languages, like those with stronger type systems, will not allow this to happen.
    - brightball 10y ago
      Yea, monkey patching is helpful when dealing with a 3rd party library that needs to be tweaked 10 layers up the inheritance chain without having to change the object type all over the whole system. If it gets overused it causes problems but there are times when it is close to a miracle. That said, there is a reason ruby devs are so test conscious.
      - johnfn 10y ago
        Yeah - of course monkey patching has good uses :) The problem is that when you're trying to debug an issue, it's another thing that you'll have to remember - "is anyone monkey patching something in here?"
grabcocque 10y ago
This entire piece sounds like the Blub Paradox made real. http://paulgraham.com/avg.html http://paulgraham.com/avg.html It's written with knocking down a very specific set of straw men in mind, but rather carefully avoids coming anywhere close to addressing the legitimate criticisms of Go as a language. One of the things that's most irritating about Go enthusiasts is the way they try to close ranks on legitimate critique and reframe their language's warts as "simplicity". Also: 20 bonus blub points for pulling the old 'I don't need generics, therefore NOBODY does' gambit.
- aaron-lebo 10y ago
  edit: there are other more useful responses
  - grabcocque 10y ago
    Actually, I can't deny that I'm a language snob. I also don't think that's a bad thing. I find Go's magical maps and slices, its null pointers, its verbose error handling and its lack of generics to all be in fairly poor taste. So, yeah, I'm a snob.
    - deleted 10y ago
      [deleted]
    - eeZah7Ux 10y ago
      Few people would accuse airline pilots, surgeons, construction engineers of snobbery for demanding high standards. When it comes to software, challenging this "everything goes" culture is called "snobbery". People are called "senior engineer" after 2 years of copypasting javascript from stack overflow - and become "proficient" in a language in a week. And yet we wonder why so much software is bloated, unreliable, insecure, overly expensive to develop.
      - aaron-lebo 10y ago
        There's having standards and there's snobbery. Rejecting a language simply because it doesn't fit your conception of what a language should be is snobbery. I've chosen Go for projects specifically because it produces better software (within certain contexts). A fast simple binary, a simple language (so co-workers can look and hack on the code), etc. To act like people are picking Go because they are ignorant or stupid or lazy (which is what the notion of "blub" always is) is unfair and lazy.
mylons 10y ago
And half are from dependencies? ;)
- lossolo 10y ago
  Read the article: As of this writing, the main repo for Juju, http://github.com/juju/juju http://github.com/juju/juju, is 3542 files, with 540,000 lines of Go code (not included in that number is 65,000 lines of comments). Counting all dependencies except the standard library, Juju is 9523 files, holding 1,963,000 lines of Go code (not including comments, which clock in at 331,000 lines).
  - mylons 10y ago
    It was a joke.
mynegation 10y ago
Off topic rant: I don't know much about the details of godeps hash file but I do wish that there were a better infrastructure for merging contents of various file formats. Built into git or shipped as a separate repository. I wasted too much time merging vcxproj.filters files just because in its XML representation one item of a sequence of folder assignments occupies 3 lines (opening tag, contents, closing tag) instead of having everything on one line. Similar problems with JSON files when new items added concurrently to the end of the array.
- rogpeppe1 10y ago
  The godeps file is just a dependency-per-line, tab-separated values, deliberately so it's easily amenable to shell script processing. Aside: the conflicts mentioned in the article should never be a real problem because you can always resolve the conflict by just recreating the dependences.tsv file (you should never be editing it manually anyway).
  - NateDad 10y ago
    That's not exactly true, Rog. If two people have changed the file, which causes a conflict, then you have resolve the conflict manually. Recreating the file with godeps would populate it with whatever commit happens to be in your gopath right now, which has no relevance to what is conflicted in the file and could be a completely different hash.
    - rogpeppe1 10y ago
      I tend to check out one branch, run godeps -u, then check out the other one and run godeps -N -u. Then you've got the newest deps from both branches. I still wouldn't resolve the conflict manually.
      - NateDad 10y ago
        TIL I learned about godeps -N.... a month too late :) Cool, though, that definitely would help with that problem.
rjammala 10y ago
Wonder how long does it take to build juju?
- f2f 10y ago
  dave cheney has been tracking juju build times since the regression in the compiler when it was rewritten from C to go: older article here: https://dave.cheney.net/2016/04/02/go-1-7-toolchain-improvements https://dave.cheney.net/2016/04/02/go-1-7-toolchain-improvem... spreadsheet: https://docs.google.com/spreadsheets/d/1mczKWp3DUuQvIAwZiORD29j5LRb96QZC4mZDn72gAE4/edit#gid=0 https://docs.google.com/spreadsheets/d/1mczKWp3DUuQvIAwZiORD...
  - infogulch 10y ago
    Wow the latest revision is down to 140% of 1.4.3. I'm impressed.
- NateDad 10y ago
  42 seconds. Just tried it with go 1.8 and a clean gopath (after downloading). That's on a 3 year old quad core i7 laptop w/ SSD. That's `go install github.com/juju/juju/...` which actually builds two binaries - the client and the server.
excepttheweasel 10y ago
I think the go language has taken a position along the lines of "re usability and abstraction are overrated". I certainly think there is some truth to that, I am really enjoying working with go on smaller projects and it is this philosophy that has made understanding the code much easier. But I wonder how it really scales on a large code base like this? Some of the best projects I've worked on leverage usability more effectively to create a sort of vocabulary. They're far more concise, there is rarely more than one source of truth, they're far easier to change and improve. Does this hold true for 540,000 lines of go code?
- jrs95 10y ago
  Well, it'd definitely be easier to change than something that had the wrong abstractions.
- NateDad 10y ago
  functions and methods are the original abstraction. Interfaces allow you to abstract implementations of sets of methods. I fail to see how that is saying that abstraction and usability are overrated. They're not. They're important.
  - 2_listerine_pls 10y ago
    Yeah, I guess some need to review the definition of abstraction.
    - sbov 10y ago
      Sometimes I feel like the OO world has redefined abstraction to only mean an interface.
      - pjmlp 10y ago
        It was already known in the 90's as component based programming. https://en.wikipedia.org/wiki/Component-based_software_engineering https://en.wikipedia.org/wiki/Component-based_software_engin... https://www.amazon.com/Component-Software-Object-Oriented-Programming-Szyperski/dp/B011DCDV40/ref=sr_1_6?ie=UTF8&qid=1490373711&sr=8-6&keywords=component+software+beyond+object-oriented+programming https://www.amazon.com/Component-Software-Object-Oriented-Pr...
Gys 10y ago
3542 files 540,000 lines of Go code 65,000 lines of comments So on average only 170 lines per file, including 17 lines of comments. Are this normal ratios ?
- fishywang 10y ago
  This sounds normal in go. In go a minimal import (package) is a while directory, not a single file, so it's normal to split packages into smaller files, each file implements a smaller set of features.
- adtac 10y ago
  I think so? Assuming a regular function (a method should do one thing and one thing only) is about ~20 lines, we're looking at ~6 functions (170 lines - 17 comments - about 15 lines of newlines, imports and so on = 130-odd). Six functions sounds very reasonable to me.
- butabah 10y ago
  Using cloc against the Go source code, I found that it has 934025 loc with 3080 files. This averages to about 300 loc per file. There are 149688 lines of comments, which averages to about 50 per file. I'd say those ratios are very similar.
- michaelmcmillan 10y ago
  I try to keep my files under 100 loc, but I mostly build CRUD applications.
- sulam 10y ago
  The comment ratio is going to vary heavily depending on the problem the project is trying to solve. If it was a library, I would expect the ratio of comments to go up to 30% or more. The LOC per file average seems slightly high for my taste but okay.
k__ 10y ago
Are generics really that big of a thing if you got structural typing? I mean, you don't have to implement all the interfaces explicitly, you just have to get your structure right and be done with it. Am I missing something?
- buckhx 10y ago
  It can be annoying to not have a generic collection. So if I build a heap I need to build it for a specific type or use interface{} as the values. That being said, it's never been a deal breaker for me and don't end up missing generics THAT much.
- whateveracct 10y ago
  Yes they are. And actually they would play nice with generics! Imagine type Ord a interface { Compare(a) int } Also it would be nice to do something like func Max[a <: Ord a](a, a) a (Syntax stolen from Scala) Right now, interfaces are too opaque. You can't take a value of interface type X and return the same type. You have to return an opaque X, which gives you no guarantees about its concrete implementation. Parametricity is an extremely well-motivated and solved language feature! I think if you add parametricity to Go functions (not even parametric types. Just type variables that play nice with all the builtins) you can write this and get guarantees about its implementation (assuming it follows the functor laws) func map[a,b](func(a) b, []a) []b
- rbehrends 10y ago
  Implement a heap that works for any type with a user-defined ordering relation (in particular, you should be able to implement both max heaps and min heaps over the same type by changing the ordering). The heap should allow for efficient operations to add an element, retrieve the minimum element, and to merge two heaps; merging should be able to make use of specialized bulk operations rather than just adding elements one by one. The data structure should be opaque, so that you can (e.g.) switch out binary heaps for Fibonacci heaps later on. The interface should be typesafe. Heaps are useful, inter alia, to define efficient priority queues. Other examples: * Implement directed graphs using arbitrary types for nodes and edges. Graph algorithms are useful in a number of application areas. * Implement a parser combinator library that works for various types of tokens, semantic values, and states. General problems with lack of parametric polymorphism (where subtyping is not enough) are: * The necessity for the client of a service to cast the result to the desired type. * Difficulty in implementing binary operations efficiently where both operands are of the same or related types (such as the heap merge above), because you can't know for certain that they are of the same type just because they conform to the same interface. * Lack of type safety guarantees when you're mixing incompatible instances (such as merging a min heap and a max heap over the same type).
VMG 10y ago
> The first was assuming forward slashes for paths in tests. So, for example, if you know that a config file should be in the “juju” subfolder and called “config.yml”, then your test might check that the file’s path is folder + “/juju/config.yml” - except that on Windows it would be folder + “\juju\config.yml”. Wait what? I thought this was solved ten years ago https://en.wikipedia.org/wiki/Path_(computing)#MS-DOS.2FMicrosoft_Windows_style https://en.wikipedia.org/wiki/Path_(computing)#MS-DOS.2FMicr...
- oppositelock 10y ago
  Go gives you the tools to handle this, but you have to do it yourself. filepath.Join(), filepath.ToSlash() and filepath.FromSlash() are used for this. I find myself very productive in Go, I've written a lot of it now, but I do also find myself writing more code than I thought I would, having had some expectations set by Python and Java. Go's philosophy seems to avoid doing something which can be done incorrectly, and instead punting it to developers. It's not always as simple as simply replacing forward and backward slashes.
  - carussell 10y ago
    > Go gives you the tools to handle this, but you have to do it yourself. filepath.Join(), filepath.ToSlash() and filepath.FromSlash() are used for this. You're missing the point—it's unlikely that you need to do anything at all; the Windows APIs handle forward slash as a path separator just fine. And on that note, for any MS employees reading now and who write READMEs and other docs for projects with publicly released code in this "New Microsoft" era, please default to using forward slash for the paths in your code snippets, instead of backslash. There's hardly any reason not to. (Unless you're using the command prompt—but you guys are a pointy clicky bunch and don't like the CLI anyway, right? Even so, you have PowerShell.) So you can drop practice of writing up separate "For Windows users" and "For Mac and Linux users" instructions that differ only in the type of path separator used. Just use plain ol' slash. Please tell your colleagues to do the same.
    - slimsag 10y ago
Nanshan 10y ago
I think the author just wan to said: choice Go is exactly a mistake.
- abiox 10y ago
  this appears to be incorrect.
- NateDad 10y ago
  <_< >_> No. Go was a good choice. The project benefited from it, both from a hiring perspective and from a technical perspective. If I were in charge, I would make the exact same choice again. Juju lives exactly in Go's sweet spot. A networked server worked on by more people than you can fit around your dining room table. Not every project would benefit from Go, but this one certainly did, IMO.
greenhouse_gas 10y ago
I happened to believe that a language should have one style. You want a functional language? Use it. You want a procedural language? Use it. You want an OOP language? Use it. They're all good, but not in one language. For example, you like FP idioms, and program everything with maps. I like for loops. You have to fix my code one day I'm on vacation. You think that it's ugly, and rewrite it as a map. I get back, bug comes up, I rewrite it back to for loops. Repeat. Go showed how to finally end indentation wars, maybe it can show how to end style wars.
- dasil003 10y ago
  Even though I've written more ruby than anything else over the last decade, I have to agree with this. Ruby is nice OO language, but it's unabashedly multi-paradigm, and that leads to a lot of mess when you get a team with varying backgrounds the result is potentially a byzantine mix of procedural, object-oriented and functional styles.
  - maxxxxx 10y ago
    When you are in a big project a multi paradigm language seems to be the only way for introducing new styles. In the stuff I am working on I can't just switch to a new language. I agree that this can create a mess though.
    - dasil003 10y ago
      Absolutely, but it's such a double-edged sword. All else being equal, I love ruby, and when it comes to shitty legacy codebases, you can do worse than ruby because of the power at your disposal to work around problems. On the other hand, it's not that hard in ruby to make something that is so thoroughly fucked that there's no reasonable option but to nuke it from space. The beauty of Java, Go and Haskell (never thought I'd use those 3 in the same sentence) is that you really have to work to fuck things up that badly.
- oxalorg 10y ago
  I completely agree, and it's one of the reasons I've liked Python. From the zen of Python: There should be one-- and preferably only one --obvious way to do it.
schmichael 10y ago
I'm 3.5 years into using Go exclusively as well, and this rings very true to me with regard to generics: > Interfaces are good enough 99% of the time. Generics would be really nice for generic data structures (heaps, trees, etc), but code generation is ok at that. Generics could allow for some nicer (and safer!) nil handling and error checking, but I think the benefits-vs-complexity are less clear than with generics for data structures. I think it's hard to quantify the massive benefit Go's simplicity is to onboarding developers -- especially for a new language where hiring experienced devs is next to impossible. The ease of onboarding is reason enough for businesses to consider Go as over an org's life a lot of time will be spent (wasted?) onboarding. Any language features which increase cognitive overhead had better offer some extremely compelling benefits to outweigh an increased learning curve. > And I don’t mean interface{}. We used interface{} rarely in Juju, and almost always it was because some sort of serialization was going on. This has been my experience as well. Whenever I hear someone complaining about interface{}, I wonder what they're doing. I think it's often people used to having generics or a dynamic language trying to follow similar patterns in Go and not considering alternative patterns. I've never used a language that didn't lack type information at the edges (where serialization occurs). Even using strongly typed serialization (eg Protobufs) in a language with strong type features and patterns (eg Java) I always see a fair amount of glue code converting from "weaker" serialization types to stronger internal representations. As long as those edges are architected to be easily testable (even fuzzable!); I don't see it as a problem. You have to convert from bytes-on-the-wire to typed variables somehow.
- rbehrends 10y ago
  > Any language features which increase cognitive overhead had better offer some extremely compelling benefits to outweigh an increased learning curve. This is a common misunderstanding that occurs (I think) because most people only know C++-, Java-, and C#-style generics, which can be conceptually complicated, because of the often tricky interactions with the type system (or, in the case of C++, the use for metaprogramming). In contrast, module-based genericity (SML, OCaml, Ada, Modula-3) is conceptually pretty simple, as it avoids complicating the type system. Its main downside is (relative) verbosity, but Go has never really eschewed verbosity.
deleted 10y ago
[deleted]
al2o3cr 10y ago
always always checking errors really makes for very few nil pointers being passed around If your developers always always remember to write error checks, they probably also would have always always remembered to write NULL checks.
bogomipz 10y ago
Does anyone know where this developer is going after Canonical or why they left? It seems like they got to work on a nice project while there. Maybe that's another blog post though. It piqued my curiosity I guess.
josteink 10y ago
Is it too late to queue the joke about it being 500k lines because the lack of generics and the resulting code-generation? Reading about projects with 100+ types of collections can certainly lead you to think so.

326 points by natefinch 10y ago

Great point about time. At work we've adapted github.com/WatchBeam/clock and it's helped a lot. Thanks for blogging about your work on juju! Despite Go already being five years old, many of the patterns around building large applications are only emerging now.

akerro 10y ago
Go was released in 2007, it's 10 years old. Rust is closer to be 5yo, it's 7yo according to wikipedia.
- bpicolo 10y ago
  Wasn't released to public at all until late 2009.
- jimktrains2 10y ago
  It only became stable in mid-2015. https://blog.rust-lang.org/2015/05/15/Rust-1.0.html https://blog.rust-lang.org/2015/05/15/Rust-1.0.html
  - NateDad 10y ago
    Comparing 1.0's is a much better comparison, except in cases where there was huge adoption before 1.0.
    - jimktrains2 10y ago
      I wouldn't say rust was hugely adopted before 1.0 (not that it's hugely adopted after, either).
      - NateDad 10y ago
        Yes, sorry, wasn't saying that either Go or Rust were an exception, just trying to forestall the inevitable "but what about X thing that was used by everyone and is still 0.9" :) For Go vs. Rust, I think a comparison of 1.0 releases is perfect.
      - stymaar 10y ago
        I think gp meant that for some platforms it can make sense to count even 0.x releases (nodejs is the best example I think), even though for most it doesn't (including Rust and Go).
- f2f 10y ago
  Go was announced to the world on November 10th, 2009.
weberc2 10y ago
This is a minor nit, but it seems to me that it would have been easier to configure the unit with a shorter timeout duration rather than mocking out the time functions. Am I mistaken?
- zalmoxes 10y ago
  It really depends. You might want to test something that requires days/weeks expiration. I would agree that the Clock interface is a bit large. In personal projects I prefer to just have something simple like timer func() time.Time
  - weberc2 10y ago
    I don't understand your argument. What's special about a days/weeks that would prevent you from shortening this value for your tests? In other words, if I want to test my timeout logic, it should be independent of the timeout value, so I should be free to use a shorter timeout value in my timeout tests.
- jerf 10y ago
  Mocking out the time functions means you don't get any race conditions. If you try to just twiddle with how long the time functions run, you still never quite know how long to set them. Bear in mind that even rather generous timeouts like a full millisecond to set a map entry in another goroutine can still fail if your system's CPU is loaded or the system is in swap, and spurious test failures are spurious test failures regardless of their cause. Often with this sort of code you're testing in one goroutine and the test code is in one or more other goroutines. If your test code can emit something down a synchronous channel, then your test code can also be sure it is staying in sync. I mean, even to test with "extra special small test timeouts" still means you're writing in extra test structure, you might as well just use one of these time mocking things. (In fact, you should generally always use time mocking libraries. In all languages, direct access to the system clock that can not be easily redirected for testing is at least a code smell and could possibly even rise to the level of "antipattern".) I have a lot of places in my Go test code where my test code is deliberately pushing things down channels. It's one of the top reasons my test code often still ends up being in the same package, so it gets private access to those channels for safe, properly-sync'ed testing. (Not that I'm really all that concerned about trying to test only the public interface; I don't have a lot of troubles with that anyhow. YMMV.) I also have some places in my code where I have channels in the private interface of some goroutine server whose sole purpose in life is to sync with the tests. This generally appears when I have some server that I am sending a message that I expect to change the state of the server, but for which there is no reply. In order to verify that the proper changes have occurred in the data structures of the server, I need to sync with the change before I do the check. Having a simple struct{} channel whose sole purpose in life is to synchronize works well enough for that.

therealmarv 10y ago

What do people think about the future of Juju?

brianwawok 10y ago
Been a few years, and it was cool in the day, but very complex. With k8 now, not sure why one would choose Juju first. They are slightly different problem domains, but not exactly.
alsadi 10y ago
For automation of deployments we prefer ansible (ex. it's agent-less) For container orchestration it's clear that kubernetes is the winner (if you have k8s deployed you don't need juju) For managing the whole thing manage iq have much to offer.
- noir_lord 10y ago
  I like Ansible and use it for deploying vagrant setups and such but I've found quite a few things broken enough that I'm wary of using it to do deploy production stuff for quite a while I had issues with apt: so much so I ended up using shell, last time I wrote a new playbook they had resolved that though so things do get fixed.

ben_pr 10y ago

I'm looking forward to my first project with GO. It appears to offer a lot with minimal complexity. > Because Go has so little magic, I think this was easier than it would have been in other languages. You don’t have the magic that other languages have that can make seemingly simple lines of code have unexpected functionality. You never have to ask “how does this work?”, because it’s just plain old Go code. That lack of magic and his comparison to C# sounds like a really good mix.

kasey_junk 10y ago
> That lack of magic I have a really hard time understanding what people mean when they say magic. In every language I've ever worked in I spend a fair bit of time saying "how does this work". Go doesn't seem any different in that regard to me.
- johnfn 10y ago
  When people say "magic", what they often mean is "code over here can effect the execution of code over there in an implicit way". Like in Ruby, I could conditionally monkey-patch a function into an object someone way over there was using, causing code to break. Other languages, like those with stronger type systems, will not allow this to happen.
  - brightball 10y ago
    Yea, monkey patching is helpful when dealing with a 3rd party library that needs to be tweaked 10 layers up the inheritance chain without having to change the object type all over the whole system. If it gets overused it causes problems but there are times when it is close to a miracle. That said, there is a reason ruby devs are so test conscious.
    - johnfn 10y ago
      Yeah - of course monkey patching has good uses :) The problem is that when you're trying to debug an issue, it's another thing that you'll have to remember - "is anyone monkey patching something in here?"

grabcocque 10y ago

This entire piece sounds like the Blub Paradox made real. http://paulgraham.com/avg.html http://paulgraham.com/avg.html It's written with knocking down a very specific set of straw men in mind, but rather carefully avoids coming anywhere close to addressing the legitimate criticisms of Go as a language. One of the things that's most irritating about Go enthusiasts is the way they try to close ranks on legitimate critique and reframe their language's warts as "simplicity". Also: 20 bonus blub points for pulling the old 'I don't need generics, therefore NOBODY does' gambit.

aaron-lebo 10y ago
edit: there are other more useful responses
- grabcocque 10y ago
  Actually, I can't deny that I'm a language snob. I also don't think that's a bad thing. I find Go's magical maps and slices, its null pointers, its verbose error handling and its lack of generics to all be in fairly poor taste. So, yeah, I'm a snob.
  - deleted 10y ago
    [deleted]
  - eeZah7Ux 10y ago
    Few people would accuse airline pilots, surgeons, construction engineers of snobbery for demanding high standards. When it comes to software, challenging this "everything goes" culture is called "snobbery". People are called "senior engineer" after 2 years of copypasting javascript from stack overflow - and become "proficient" in a language in a week. And yet we wonder why so much software is bloated, unreliable, insecure, overly expensive to develop.
    - aaron-lebo 10y ago
      There's having standards and there's snobbery. Rejecting a language simply because it doesn't fit your conception of what a language should be is snobbery. I've chosen Go for projects specifically because it produces better software (within certain contexts). A fast simple binary, a simple language (so co-workers can look and hack on the code), etc. To act like people are picking Go because they are ignorant or stupid or lazy (which is what the notion of "blub" always is) is unfair and lazy.

mylons 10y ago

And half are from dependencies? ;)

lossolo 10y ago
Read the article: As of this writing, the main repo for Juju, http://github.com/juju/juju http://github.com/juju/juju, is 3542 files, with 540,000 lines of Go code (not included in that number is 65,000 lines of comments). Counting all dependencies except the standard library, Juju is 9523 files, holding 1,963,000 lines of Go code (not including comments, which clock in at 331,000 lines).
- mylons 10y ago
  It was a joke.

mynegation 10y ago

Off topic rant: I don't know much about the details of godeps hash file but I do wish that there were a better infrastructure for merging contents of various file formats. Built into git or shipped as a separate repository. I wasted too much time merging vcxproj.filters files just because in its XML representation one item of a sequence of folder assignments occupies 3 lines (opening tag, contents, closing tag) instead of having everything on one line. Similar problems with JSON files when new items added concurrently to the end of the array.

rogpeppe1 10y ago
The godeps file is just a dependency-per-line, tab-separated values, deliberately so it's easily amenable to shell script processing. Aside: the conflicts mentioned in the article should never be a real problem because you can always resolve the conflict by just recreating the dependences.tsv file (you should never be editing it manually anyway).
- NateDad 10y ago
  That's not exactly true, Rog. If two people have changed the file, which causes a conflict, then you have resolve the conflict manually. Recreating the file with godeps would populate it with whatever commit happens to be in your gopath right now, which has no relevance to what is conflicted in the file and could be a completely different hash.
  - rogpeppe1 10y ago
    I tend to check out one branch, run godeps -u, then check out the other one and run godeps -N -u. Then you've got the newest deps from both branches. I still wouldn't resolve the conflict manually.
    - NateDad 10y ago
      TIL I learned about godeps -N.... a month too late :) Cool, though, that definitely would help with that problem.

rjammala 10y ago

Wonder how long does it take to build juju?

f2f 10y ago
dave cheney has been tracking juju build times since the regression in the compiler when it was rewritten from C to go: older article here: https://dave.cheney.net/2016/04/02/go-1-7-toolchain-improvements https://dave.cheney.net/2016/04/02/go-1-7-toolchain-improvem... spreadsheet: https://docs.google.com/spreadsheets/d/1mczKWp3DUuQvIAwZiORD29j5LRb96QZC4mZDn72gAE4/edit#gid=0 https://docs.google.com/spreadsheets/d/1mczKWp3DUuQvIAwZiORD...
- infogulch 10y ago
  Wow the latest revision is down to 140% of 1.4.3. I'm impressed.
NateDad 10y ago
42 seconds. Just tried it with go 1.8 and a clean gopath (after downloading). That's on a 3 year old quad core i7 laptop w/ SSD. That's `go install github.com/juju/juju/...` which actually builds two binaries - the client and the server.

excepttheweasel 10y ago

I think the go language has taken a position along the lines of "re usability and abstraction are overrated". I certainly think there is some truth to that, I am really enjoying working with go on smaller projects and it is this philosophy that has made understanding the code much easier. But I wonder how it really scales on a large code base like this? Some of the best projects I've worked on leverage usability more effectively to create a sort of vocabulary. They're far more concise, there is rarely more than one source of truth, they're far easier to change and improve. Does this hold true for 540,000 lines of go code?

jrs95 10y ago
Well, it'd definitely be easier to change than something that had the wrong abstractions.
NateDad 10y ago
functions and methods are the original abstraction. Interfaces allow you to abstract implementations of sets of methods. I fail to see how that is saying that abstraction and usability are overrated. They're not. They're important.
- 2_listerine_pls 10y ago
  Yeah, I guess some need to review the definition of abstraction.
  - sbov 10y ago
    Sometimes I feel like the OO world has redefined abstraction to only mean an interface.
    - pjmlp 10y ago
      It was already known in the 90's as component based programming. https://en.wikipedia.org/wiki/Component-based_software_engineering https://en.wikipedia.org/wiki/Component-based_software_engin... https://www.amazon.com/Component-Software-Object-Oriented-Programming-Szyperski/dp/B011DCDV40/ref=sr_1_6?ie=UTF8&qid=1490373711&sr=8-6&keywords=component+software+beyond+object-oriented+programming https://www.amazon.com/Component-Software-Object-Oriented-Pr...

Gys 10y ago

3542 files 540,000 lines of Go code 65,000 lines of comments So on average only 170 lines per file, including 17 lines of comments. Are this normal ratios ?

fishywang 10y ago
This sounds normal in go. In go a minimal import (package) is a while directory, not a single file, so it's normal to split packages into smaller files, each file implements a smaller set of features.
adtac 10y ago
I think so? Assuming a regular function (a method should do one thing and one thing only) is about ~20 lines, we're looking at ~6 functions (170 lines - 17 comments - about 15 lines of newlines, imports and so on = 130-odd). Six functions sounds very reasonable to me.
butabah 10y ago
Using cloc against the Go source code, I found that it has 934025 loc with 3080 files. This averages to about 300 loc per file. There are 149688 lines of comments, which averages to about 50 per file. I'd say those ratios are very similar.
michaelmcmillan 10y ago
I try to keep my files under 100 loc, but I mostly build CRUD applications.
sulam 10y ago
The comment ratio is going to vary heavily depending on the problem the project is trying to solve. If it was a library, I would expect the ratio of comments to go up to 30% or more. The LOC per file average seems slightly high for my taste but okay.

k__ 10y ago

Are generics really that big of a thing if you got structural typing? I mean, you don't have to implement all the interfaces explicitly, you just have to get your structure right and be done with it. Am I missing something?

buckhx 10y ago
It can be annoying to not have a generic collection. So if I build a heap I need to build it for a specific type or use interface{} as the values. That being said, it's never been a deal breaker for me and don't end up missing generics THAT much.
whateveracct 10y ago
Yes they are. And actually they would play nice with generics! Imagine type Ord a interface { Compare(a) int } Also it would be nice to do something like func Max[a <: Ord a](a, a) a (Syntax stolen from Scala) Right now, interfaces are too opaque. You can't take a value of interface type X and return the same type. You have to return an opaque X, which gives you no guarantees about its concrete implementation. Parametricity is an extremely well-motivated and solved language feature! I think if you add parametricity to Go functions (not even parametric types. Just type variables that play nice with all the builtins) you can write this and get guarantees about its implementation (assuming it follows the functor laws) func map[a,b](func(a) b, []a) []b
rbehrends 10y ago
Implement a heap that works for any type with a user-defined ordering relation (in particular, you should be able to implement both max heaps and min heaps over the same type by changing the ordering). The heap should allow for efficient operations to add an element, retrieve the minimum element, and to merge two heaps; merging should be able to make use of specialized bulk operations rather than just adding elements one by one. The data structure should be opaque, so that you can (e.g.) switch out binary heaps for Fibonacci heaps later on. The interface should be typesafe. Heaps are useful, inter alia, to define efficient priority queues. Other examples: * Implement directed graphs using arbitrary types for nodes and edges. Graph algorithms are useful in a number of application areas. * Implement a parser combinator library that works for various types of tokens, semantic values, and states. General problems with lack of parametric polymorphism (where subtyping is not enough) are: * The necessity for the client of a service to cast the result to the desired type. * Difficulty in implementing binary operations efficiently where both operands are of the same or related types (such as the heap merge above), because you can't know for certain that they are of the same type just because they conform to the same interface. * Lack of type safety guarantees when you're mixing incompatible instances (such as merging a min heap and a max heap over the same type).

VMG 10y ago

> The first was assuming forward slashes for paths in tests. So, for example, if you know that a config file should be in the “juju” subfolder and called “config.yml”, then your test might check that the file’s path is folder + “/juju/config.yml” - except that on Windows it would be folder + “\juju\config.yml”. Wait what? I thought this was solved ten years ago https://en.wikipedia.org/wiki/Path_(computing)#MS-DOS.2FMicrosoft_Windows_style https://en.wikipedia.org/wiki/Path_(computing)#MS-DOS.2FMicr...

oppositelock 10y ago
Go gives you the tools to handle this, but you have to do it yourself. filepath.Join(), filepath.ToSlash() and filepath.FromSlash() are used for this. I find myself very productive in Go, I've written a lot of it now, but I do also find myself writing more code than I thought I would, having had some expectations set by Python and Java. Go's philosophy seems to avoid doing something which can be done incorrectly, and instead punting it to developers. It's not always as simple as simply replacing forward and backward slashes.
- carussell 10y ago
  > Go gives you the tools to handle this, but you have to do it yourself. filepath.Join(), filepath.ToSlash() and filepath.FromSlash() are used for this. You're missing the point—it's unlikely that you need to do anything at all; the Windows APIs handle forward slash as a path separator just fine. And on that note, for any MS employees reading now and who write READMEs and other docs for projects with publicly released code in this "New Microsoft" era, please default to using forward slash for the paths in your code snippets, instead of backslash. There's hardly any reason not to. (Unless you're using the command prompt—but you guys are a pointy clicky bunch and don't like the CLI anyway, right? Even so, you have PowerShell.) So you can drop practice of writing up separate "For Windows users" and "For Mac and Linux users" instructions that differ only in the type of path separator used. Just use plain ol' slash. Please tell your colleagues to do the same.
  - slimsag 10y ago

Nanshan 10y ago

I think the author just wan to said: choice Go is exactly a mistake.

abiox 10y ago
this appears to be incorrect.
NateDad 10y ago
<_< >_> No. Go was a good choice. The project benefited from it, both from a hiring perspective and from a technical perspective. If I were in charge, I would make the exact same choice again. Juju lives exactly in Go's sweet spot. A networked server worked on by more people than you can fit around your dining room table. Not every project would benefit from Go, but this one certainly did, IMO.

greenhouse_gas 10y ago

I happened to believe that a language should have one style. You want a functional language? Use it. You want a procedural language? Use it. You want an OOP language? Use it. They're all good, but not in one language. For example, you like FP idioms, and program everything with maps. I like for loops. You have to fix my code one day I'm on vacation. You think that it's ugly, and rewrite it as a map. I get back, bug comes up, I rewrite it back to for loops. Repeat. Go showed how to finally end indentation wars, maybe it can show how to end style wars.

dasil003 10y ago
Even though I've written more ruby than anything else over the last decade, I have to agree with this. Ruby is nice OO language, but it's unabashedly multi-paradigm, and that leads to a lot of mess when you get a team with varying backgrounds the result is potentially a byzantine mix of procedural, object-oriented and functional styles.
- maxxxxx 10y ago
  When you are in a big project a multi paradigm language seems to be the only way for introducing new styles. In the stuff I am working on I can't just switch to a new language. I agree that this can create a mess though.
  - dasil003 10y ago
    Absolutely, but it's such a double-edged sword. All else being equal, I love ruby, and when it comes to shitty legacy codebases, you can do worse than ruby because of the power at your disposal to work around problems. On the other hand, it's not that hard in ruby to make something that is so thoroughly fucked that there's no reasonable option but to nuke it from space. The beauty of Java, Go and Haskell (never thought I'd use those 3 in the same sentence) is that you really have to work to fuck things up that badly.
oxalorg 10y ago
I completely agree, and it's one of the reasons I've liked Python. From the zen of Python: There should be one-- and preferably only one --obvious way to do it.

schmichael 10y ago

I'm 3.5 years into using Go exclusively as well, and this rings very true to me with regard to generics: > Interfaces are good enough 99% of the time. Generics would be really nice for generic data structures (heaps, trees, etc), but code generation is ok at that. Generics could allow for some nicer (and safer!) nil handling and error checking, but I think the benefits-vs-complexity are less clear than with generics for data structures. I think it's hard to quantify the massive benefit Go's simplicity is to onboarding developers -- especially for a new language where hiring experienced devs is next to impossible. The ease of onboarding is reason enough for businesses to consider Go as over an org's life a lot of time will be spent (wasted?) onboarding. Any language features which increase cognitive overhead had better offer some extremely compelling benefits to outweigh an increased learning curve. > And I don’t mean interface{}. We used interface{} rarely in Juju, and almost always it was because some sort of serialization was going on. This has been my experience as well. Whenever I hear someone complaining about interface{}, I wonder what they're doing. I think it's often people used to having generics or a dynamic language trying to follow similar patterns in Go and not considering alternative patterns. I've never used a language that didn't lack type information at the edges (where serialization occurs). Even using strongly typed serialization (eg Protobufs) in a language with strong type features and patterns (eg Java) I always see a fair amount of glue code converting from "weaker" serialization types to stronger internal representations. As long as those edges are architected to be easily testable (even fuzzable!); I don't see it as a problem. You have to convert from bytes-on-the-wire to typed variables somehow.

rbehrends 10y ago
> Any language features which increase cognitive overhead had better offer some extremely compelling benefits to outweigh an increased learning curve. This is a common misunderstanding that occurs (I think) because most people only know C++-, Java-, and C#-style generics, which can be conceptually complicated, because of the often tricky interactions with the type system (or, in the case of C++, the use for metaprogramming). In contrast, module-based genericity (SML, OCaml, Ada, Modula-3) is conceptually pretty simple, as it avoids complicating the type system. Its main downside is (relative) verbosity, but Go has never really eschewed verbosity.

deleted 10y ago

[deleted]

al2o3cr 10y ago

always always checking errors really makes for very few nil pointers being passed around If your developers always always remember to write error checks, they probably also would have always always remembered to write NULL checks.

bogomipz 10y ago

Does anyone know where this developer is going after Canonical or why they left? It seems like they got to work on a nice project while there. Maybe that's another blog post though. It piqued my curiosity I guess.

josteink 10y ago

Is it too late to queue the joke about it being 500k lines because the lack of generics and the resulting code-generation? Reading about projects with 100+ types of collections can certainly lead you to think so.