9 ms·

Stunned by Go

• codezero 13y ago
  This is pretty interesting, but I wonder if it's just a one-off problem with the http library. It'll be interesting to see if someone familiar with Go has a response to this, but from the blog post it doesn't appear that this contract is being habitually violated, only that it can be, which is still a problem in practice.
  • frou_dh 13y ago
    It's a bit insidious since even if that situation is rare, just knowing that it's possible introduces a new dimension of doubt next time things go awry with any code you didn't write. Enter a "Seriously, No Groping" type modifier ;)
• bsaul 13y ago
  Anyone proficient in GO would care to provide the minimal code that illustrate that extremely weird behavior ? I find that post so disturbing that i still believe i've missed something.
  • thirsteh 13y ago
    An example: http://golang.org/src/pkg/io/io.go?s=11741:11801#L336 http://golang.org/src/pkg/io/io.go?s=11741:11801#L336 Line 336. io.Copy takes a writer and a reader, but actually asserts whether they implement ReaderFrom/WriterTo, then use their methods, ReadFrom and WriteTo, if they do. (Same thing with the http package checking if the reader implements Closer, and calling Close if it does, though admittedly 'we should be able to call Close' is a much more unsafe assumption than 'if it implements ReadFrom, and it has the right type signature, we can just call that function instead'.) The reason it doesn't take values implementing the interfaces ReaderFrom and WriterTo is that io.Copy works even if you don't implement those interfaces. It's an unfortunate and regrettable predicament, but it's not something that comes up a lot. As others have mentioned in this thread, Go certainly isn't a language with referential transparency by any means--the type system provides minimal sanity checking, and the language itself is memory-safe, but it's not trying to be a "high-assurance language."
    • MetaCosm 13y ago
      This io.Copy behavior seems like an "optimization"... less explicit than I would like but sticks to the idea of an io.Copy. Having a resource you own Closed on you isn't an optimization, it is a behavior change.
      • thirsteh 13y ago
        I agree. I think I edited my comment to specify that it's a much more unsafe assumption just before you posted your comment.
    • bsaul 13y ago
      That is SO disturbing... They don't even check whether the parameter implements another interface, they do the check directly on the method by callin it... That's so ugly... Reminds me of my surprise when i realized the math module only worked on float types, and that min/max only existed for some numerical types but not others. The thing in itself probably isn't a big deal, but it really questions the ability of the language to model complex relationships often found in business processes.
      • thirsteh 13y ago
        > That is SO disturbing... They don't even check whether the parameter implements another interface, they do the check directly on the method by callin it... That's so ugly... That's incorrect. The code you see at the top: if rt, ok := dst.(ReaderFrom); ok { return rt.ReadFrom(src) } means: "If dst implements ReaderFrom, then call ReadFrom on this value of that type." You can't arbitrarily call methods that may or may not exist. You can check if a value implements an interface by doing "newVal, ok := val.(suspectedInterface)", and checking if ok is true, or by doing "newVal := val.(suspectedInterface)" and living with a runtime panic if it doesn't implement the interface, then call a method in that interface on the new value newVal, which will be of type suspectedInterface. (If val doesn't implement suspectedInterface--i.e. ok is false--then newVal will be the zero value of suspectedInterface.)
        • 13y ago
• bicknergseng 13y ago
  I feel like the author missed a great opportunity. "Stopped by Go"
  • gfodor 13y ago
    Stop, Go
    • f2f 13y ago
      or, in "explosive tabloid headline style": Go: Stop!
  • stock_toaster 13y ago
    Do not pass Go...a ReadCloser?
• tikhonj 13y ago
  Wow, this is pretty unfortunate. Also a very good example of why so many people maintain that Go is not nearly as statically typed as languages like Haskell or OCaml: casting (essentially subverting the type system) is very common, and limits most of the type system's advantages. In large part this is because Go's type system is not very expressive. I would just like to point out that OCaml did the same things as Go (i.e. structural sub-typing). But with (global) type inference. And variants (including structural subtyping for those too). And generics (parametric polymorphism). Years before Go was developed. In OCaml, this sort of thing probably would be much less likely to come up. After having used the OCaml object system a bit (but still more than most people!), I am actually very happy with it. It certainly has issues--oh boy, does it ever--but the issues are superficial. They are the result of very limited manpower in the project, not fundamental shortcomings of the design. In OCaml, the compiler can figure out what methods you used without your telling it. So it would automatically realize that the function called the close method, and infer the appropriate type. You wouldn't even have to write an interface (called a class type in OCaml) for it. If you did write an interface, but it did not expose a close method, the code would give you a type error. I suppose you could try to do some sort of runtime casting and duck-typing, but I am not sure how. It would certainly not be a common thing to do, at all! Instead, the type would reflect exactly how the object is used in the function. It would be more explicit, more predictable and safer. And, harping back on the same theme, it would be fully inferred for you. The best of both worlds, really.
  • dragonwriter 13y ago
    > Wow, this is pretty unfortunate. Also a very good example of why so many people maintain that Go is not nearly as statically typed as languages like Haskell or OCaml I'm confused why "so many people maintain that" -- not because its a position I disagree with, because its not really a controversial position such that you'd need to "maintain" it. Go is, pretty overtly C-like, but with some language-level concurrency features, plus type system features -- interfaces and methods and reflection -- to support OOP (though its definitely not a class-based OO language) and dynamic typing, plus GC, plus filing some of the sharp edges off of pointers. It doesn't even begin to pretend to be like OCaml or Haskell in terms of strict typing.
  • 13y ago
• BarkMore 13y ago
  This issue was recently discussed on the development mail list: https://groups.google.com/d/msg/golang-dev/o9i_SbGRrzI/gWvnBnORqRYJ https://groups.google.com/d/msg/golang-dev/o9i_SbGRrzI/gWvnB... The fix is to document that Close is called: https://codereview.appspot.com/11432044/ https://codereview.appspot.com/11432044/
  • tux21b 13y ago
    Yes, Roger Peppe, one of the Go contributors, had exactly the same issue, but luckily Brad and Russ, two core devs, were able to solve the mystery immediately and updated the documentation. It's now quite funny to read a blog post from someone completely different (Tim Penhey) who had encountered and solved the same issue with exactly the same usecase on the very same day, complaining about Go without even mentioning the discussion [1] or the latest changeset [2] that solved the problem. [1]: https://groups.google.com/d/topic/golang-dev/o9i_SbGRrzI/discussion https://groups.google.com/d/topic/golang-dev/o9i_SbGRrzI/dis... [2]: https://codereview.appspot.com/11432044 https://codereview.appspot.com/11432044
    • thirsteh 13y ago
      That is pretty funny, but it doesn't really change anything. It's unfortunate that it called Close() without at least giving proper notice, and most people will probably agree that in a perfect world where we could start over without breaking APIs, the implementation would be different.
      • tux21b 13y ago
        I was just criticizing the publishing skills of some people... The problem itself is unfortunate and I am personally not a huge fan of the "no API changes" philosophy participated by the Go team. I can understand the reasons behind it, but it's still hard to read several CLs per day that are abandoned or done differently (in a non-optimal way) because of this contract.
        • enneff 13y ago
• marshray 13y ago
  > The standard http library was calling Close on my io.Reader. This is not expected behaviour when the interface clearly just takes an io.Reader (which exposes one and only one method Read). So if I implemented my own object with a Reader interface and passed it to this HTTP library, could it find some other method on my object named Close() and it could call it? Is it pure duck typing happening here, or is the HTTP library looking for a commonly-defined interface with documented semantics for Close()?
  • usea 13y ago
    In Go, a type doesn't have to declare that it implements an interface; it automatically satisfies an interface if it implements the appropriate methods. Most interfaces in Go only contain one method. If a type has a Close() method, then it is a Closer. The code that tries to cast it to a Closer will succeed. So to answer your question, it doesn't matter what the semantics of your Close() method are, it will succeed solely based on the fact that your type has a method with that name.
    • melvinmt 13y ago
      > your type has a method with that name. I would say with the same function signature, not just the name.
      • usea 13y ago
        Go doesn't support overloading. Method dispatch is done by name only. http://golang.org/doc/faq#overloading http://golang.org/doc/faq#overloading
        • statusgraph 13y ago
          It doesn't work :) http://play.golang.org/p/PVS5sarmHb http://play.golang.org/p/PVS5sarmHb
        • dragonwriter 13y ago
          But interface satisfaction is by signature, not name, and that is what is being discussed, not method dispatch.
• xinn 13y ago
  This behavior is explicitly defined in the godoc for io. I don’t see why the author should be "stunned" by a function doing exactly what it says it does.
  • nwp90 13y ago
    That'll be because it's only been in the docs since they reported their issue.
    • thirsteh 13y ago
      You're both correct. It's been in the docs for io.Copy for a long time, and was only recently added to the http docs.
• georgemcbay 13y ago
  This is caused by an unfortunate bit of code in the standard library, but is hardly representative of a fundamental problem in Go as the article implies (unless you view casting and interface coercion as a fundamental problem, and I know some people do, but that's basically a different debate since Go does not pretend to be a purely statically typed language by any stretch of the imagination). I've been using Go regularly for nearly two years now and haven't run into an issue quite like this either with the http package (what he's doing that causes this to be a problem is legal and should work, but is not a common use case) nor with any other bit of the standard library.
  • thirsteh 13y ago
    It depends on what your definition of "fundamental problem" is. If being able to treat a value differently than it was received, or to perform computations that you shouldn't be able to, e.g. impure actions inside a pure function, then very few languages satisfy that property. Even Haskell isn't safe because of unsafePerformIO--you'd need Safe Haskell, at the very least. Go has never tried to be, and the authors have never tried to give people the impression that it is that kind of language. Some of the safety comes from the language e.g. doing array bounds checking, not letting you manipulate memory manually, etc., but the rest comes from discipline and idiomatic code. (Same thing as not using 'undefined', 'fromJust', 'unsafePerformIO', etc. in Haskell unless you're absolutely sure they won't do harm--they are too useful to remove, even if they are the sources of many headaches.)
• tptacek 13y ago
  The Golang team (or at least, Russ Cox) agrees with you: https://groups.google.com/d/msg/golang-dev/o9i_SbGRrzI/SSuqWmdQxpIJ https://groups.google.com/d/msg/golang-dev/o9i_SbGRrzI/SSuqW...
• f2f 13y ago
  for those not familiar with Go, here's the problem translated in C terms: "I passed a pointer to your function and it called free() on it. how dare you!"
  • thirsteh 13y ago
    or "I passed an fp to your function, and you called close() on it. How dare you!?" :)
  • nostrademons 13y ago
    More "I passed a pointer to your function and it called free() on it, without documenting that it was taking ownership of the memory". Ownership contracts are an important part of the API of any function that does stuff with resources, whether memory, file descriptions, network connections, etc. If the API implies that it takes a pointer to a resource but will only read from or write to it, and then disposes of it, that's a bug.
    • marshray 13y ago
      I think that's a good point. It's often said that garbage collection helps prevent memory leaks, and it's also said often that memory isn't the only kind of resource which could be leaked. Here is another case where weak- and dynamic-typing language features have come up in the context of resource management. I'm not saying this one example proves anything, just that it might be interesting to be on the lookout for other events of this pattern.
  • pcwalton 13y ago
    No, that's not what the problem is. Translated to C++, the problem is "you passed a pointer to A∗, you dynamic_cast<>'d it to a subtype B∗ without documenting that you did so".
    • nly 13y ago
      Not really. You can't dynamic_cast across across protected or private inheritance boundaries, even if the function attempting it is a friend. As long as your public inheritance hierarchy maintains your contract (and they should) you're fine.
• rjempson 13y ago
  I am just wondering whether you can write your own wrapper class that implements (exposes) the read method and doesn't expose a close method (or expose a close method that does nothing) and pass that in? I am not suggesting this is a good solution, just curious.
  • enneff 13y ago
    Yes, easily: var rc io.ReadCloser // a file, whatever readerOnly := struct{io.Reader}{rc} // only a Reader
    • frou_dh 13y ago
      That's cool. The io.Reader is an unnamed field, so its method is effectively pushed to the surface of the wrapping struct instead of needing to be explicitly forwarded to.
• pedromorgan 13y ago
  Why dont the Go team do golang2 now.. There are a few issues.. already mentioned in open fire chat this year 2013.. Would be nicer to actually recongise flaws now.. and offer alternative.. even faster tht python 3000 could ? Thats whats in my mind.. and fix the the documentation so the classes interlink and it a bit of a java/md/rst/doc style..
  • dragonwriter 13y ago
    > Why dont the Go team do golang2 now.. There are a few issues.. I suspect that it is because Go 1 (issues and all) is useful enough that people are making productive use of it, and they'd like to flush out more of the issues that may exist -- and give more time for different approaches to those issues to be tried in experimental forks and, where possible, libraries, preprocessers, etc. > and fix the the documentation so the classes interlink Not sure what this means, and Go doesn't have classes. (Its not just an issue of calling classes different names, it divides up functionality differently than class-based OO systems -- packages are in a sense like classes [they are the closest equivalent to the function of classes as containers of static methods], structs are a bit like classes [in the role of containing members, and in that anonymous embedding approximates inheritance], every type is a bit like classes [in that any type can have methods attached to it, so long as they are attached in the same package as the type is defined]; but all these things are also unlike classes in various ways, too.)
• MatthewPhillips 13y ago
  Go didn't fail you, OOP failed you.
  • marshray 13y ago
    There's nothing about OOP that requires weak typing.
    • MatthewPhillips 13y ago
      I was referring to the fact that this function mutates the state of one of its arguments.
      • marshray 13y ago
        But so does Read(), right? Even in Haskell I'd be calling the HTTP thingy in the context of the IO monad.
• cmccabe 13y ago
  Supporting typecasting is hardly earth-shattering. Pretty much every statically typed programming language in actual use supports typecasts. That includes Scala, Java, C++, C, C#, Pascal, Algol, and even Ada. This title should be "stunned by shitty poorly documented library function" not "stunned by Go." There are a few languages out there that don't support typecasting. I think SML was one, for example. But they made a lot of other design choices to be able to support that choice, and so far that set of choices has not gained popularity. At minimum, you would need a fairly complex type system with generics to be able to completely forbid typecasting. You also run into problems where stable APIs completely block progress in a particular area.
  • pcwalton 13y ago
    > There are a few languages out there that don't support typecasting. I think SML was one, for example. Haskell too, unless you explicitly opt in with Data.Typeable. It's arguably true in a certainly commonly-used subset of C++ as well: this type of typecast doesn't work in C++ unless you use dynamic_cast or another RTTI system (like COM). dynamic_cast only works on classes with a vtable. Many C++ projects don't use any form of RTTI. (Note that the Go code does not assert that the Reader is a ReadClose; rather it depends on RTTI to determine whether the reader is a ReadClose.)
    • nly 13y ago
      dynamic_cast downcasts in C++ only work across public inheritance. In particular a object that inheritance a Closable interface privately and a Reader interface publicly wouldn't allow a cross cast. dynamic_cast isn't a loophole in the type system, it's there to strengthen it.
      • marshray 13y ago
        I don't think I've ever seen C++ that inherited an interface privately. That said, it's extremely common for classes to lack any virtual functions and thus be unusable with dynamic_cast.
• rsc 13y ago
  Go developer here. I posted this on the blog too. """ This is a mistake, plain and simple. It was pointed out on the Go developer mailing list a few days before your post, and you can see my reply there. NewRequest should take an io.ReadCloser. Unfortunately, due to backwards compatibility, a long standing semantic mistake like this is not something we can just fix, but we've at least documented it. The tweet you quoted is wrong, and I want to explain why. If an interface value v is passed to a function f, then what does happen from time to time is that f will look for custom methods on v that are at least logically equivalent to the static interface type f is declared to expect. For example, io.Copy takes an io.Reader and an io.Writer and does the obvious thing, Reading into a buffer and then passing that buffer to Write. Of course, it would be nice to bypass the buffer when possible, and so there is a standard WriterTo interface that an io.Reader can also implement that, in contrast to Read, which copies into a buffer, says "read the data out of yourself and directly into this Writer". If io.Copy finds that method, it will use it, but the operation - reading - is the same. There is an analogous case for the Writer too. If you pass a value v to json.Marshal, Marshal will check to see if v implements json.Marshaler, meaning v knows how to marshal itself as JSON, in which case v is given that opportunity. Again, same operation you requested. In contrast, if you have an io.Reader, while it might be okay to look for other Read-like methods, it is certainly not okay to look for Close. That's a very different operation, it's surprising, and it shouldn't be there. FWIW, I'm not aware of any other instances of this kind of semantic mixup in the standard library. But again, unfortunately, we can't change it until Go 2. All we can do for now is document it and move on. """
  • marshray 13y ago
    Out of simple idle language-design-bystander curiosity, can you recall any other times where this class of bug has bitten?
    • rsc 13y ago
      No. A related bug would be if you implement Read and (say) ReadByte, but your ReadByte is buggy. Then when you get weird behavior passing yourself to something that expects a Reader (but internally needs byte-at-a-time and uses ReadByte to avoid double-buffering the input) it can take a minute or two to realize that ReadByte might be the problem and not Read. I've only seen that once or twice.
• tantalor 13y ago
  <tldr>App author found a nasty example of a common antipattern in library code which caused a bug in his app. Documentation was updated to point out dragons.
• GhotiFish 13y ago
  Even if there is agreement that this is a problem, it strikes me that it can become a problem. The standard library should be exemplary, what do you think would happen in the hands of a developer who's "not as smart as he thinks he is?" Wasn't that part of go's pitch? I never really considered that libraries would go so far as to inspect the types for methods rather than just make those methods a requirement. The fact that they are able means someone else is going to use it, and it's going to bite me. Don't like.
  • nknighthb 13y ago
    What was part of Go's pitch, exactly? And where did you see it pitched? People outside the Go team have tried to pitch Go as a lot of things, most of them having no relationship to what Go was actually intended to accomplish. The FAQ[1] may be enlightening. Nowhere does it say Go is supposed to be a straightjacket that makes bad programmers into good ones. [1] http://golang.org/doc/faq#What_is_the_purpose_of_the_project http://golang.org/doc/faq#What_is_the_purpose_of_the_project
    • GhotiFish 13y ago
      You'll have to forgive me, I'm at work right now so I can't pull up the talk Rob Pike did on Go, I think it was 2011. Around the point where he explained Go was not a language to expand the boundaries of computer science. It's for programmers who "often are not as smart as they think they are" I'll look it up when I get the chance. That might be later tonight.
      • nknighthb 13y ago
        I'm reasonably confident you're assigning too much significance to what was probably an offhand remark in a particular context. I just looked at slides from a 2010 talk[1] by Pike which was pretty explicit about what they were doing in this area: "It's a form of duck typing, but (usually) checkable at compile time. It's also another form of orthogonality." And this one from Russ Cox in 2010[2]: "Compiled, statically-typed languages (C, C++, Java) require too much typing and too much typing: * verbose, lots of repetition * too much focus on type hierarchy * types get in the way as much as they help * compiles take far too long" A common theme for Go from the beginning has been the balance between type safety and not getting in the way of the programmer. The analogy to duck typing has been a consistent observation throughout Go's public existence. This is incompatible with the sentiment you took from whatever talk you saw. Go's architects didn't set out to make a Java clone. It's more like Java's evil twin. [1] http://talks.golang.org/2010/ExpressivenessOfGo-2010.pdf http://talks.golang.org/2010/ExpressivenessOfGo-2010.pdf [2] http://talks.golang.org/2010/go_talk-20100112.html http://talks.golang.org/2010/go_talk-20100112.html
• Groxx 13y ago
  So what's the fix for this? Wrap the object in another that only defines Read? Or is there another way?