6 ms·

How Go detects struct copies with sync.noCopy

70 points by func25 2mo ago

woadwarrior01 1mo ago
> noCopy is a special marker for types that must not be copied after their first use. if it looks like a hack, walks like a hack, and quacks like a hack...
- pjmlp 1mo ago
  The whole Go design philosophy in one sentence, that is what one gets by refusing to adopt modern language practices.
  - deleted 1mo ago
    [deleted]
  - 0x696C6961 1mo ago
    If this was true, why did they add generics?
    - pjmlp 1mo ago
      The generics that they eventually added, because of programming language market pressure, Kubernetes had an whole code generation gizmo to work around it, are a clunky implementation versus what could have been done if done from the get go.
      - 0x696C6961 1mo ago
        The generics implementation is perfectly fine. The fact that you completely dodged my question is telling. But please, keep living in your alternate reality.
        pjmlp 1mo ago
        I did not dodge the question at all, you just didn't like the answer. "Market pressure for language adoption". Just like all the features they keep adding where they said that Go doesn't need them in first place, turns out those features exist in other programming languages for a reason.
        0x696C6961 1mo ago
        This idea that they just caved and added them due to pressure is a false narrative invented by detractors like yourself. Here is a timeline if you're interested: 2009 Go released. June 2010 https://github.com/golang/proposal/blob/master/design/15292/2010-06-type-functions.md https://github.com/golang/proposal/blob/master/design/15292/... Jan 2011 https://github.com/golang/proposal/blob/master/design/15292-generics.md https://github.com/golang/proposal/blob/master/design/15292-... March 2011 https://github.com/golang/proposal/blob/master/design/15292/2011-03-gen.md https://github.com/golang/proposal/blob/master/design/15292/... Oct 2013 https://github.com/golang/proposal/blob/master/design/15292/2013-10-gen.md https://github.com/golang/proposal/blob/master/design/15292/... Dec 2013 https://github.com/golang/proposal/blob/master/design/15292/2013-12-type-params.md https://github.com/golang/proposal/blob/master/design/15292/... April 2016 https://github.com/golang/go/issues/15292 https://github.com/golang/go/issues/15292 Aug 2018 https://github.com/golang/proposal/blob/master/design/go2draft-generics-overview.md https://github.com/golang/proposal/blob/master/design/go2dra... Aug 2018 https://github.com/golang/proposal/blob/master/design/go2draft-contracts.md https://github.com/golang/proposal/blob/master/design/go2dra... June 2020 https://go.dev/blog/generics-next-step https://go.dev/blog/generics-next-step Jan 2021 https://github.com/golang/go/issues/43651 https://github.com/golang/go/issues/43651 Jan 2021 https://github.com/golang/proposal/blob/master/design/43651-type-parameters.md https://github.com/golang/proposal/blob/master/design/43651-... Feb 2021 Proposal accepted. March 2022 Ships in Go 1.18. You could criticize Go for moving too slow. But I'm pretty happy with the way they landed. They definitely don't feel bolted-on.
        pjmlp 1mo ago
        Ah the usual barrage of links from Go folks, luckily we have also come prepared. > In three years of Go surveys, lack of generics has always been listed as one of the top three problems to fix in the language. https://blog.golang.org/why-generics https://blog.golang.org/why-generics > In retrospect, we were biased too much by experience with C++ without concepts and Java generics. We would have been well-served to spend more time with CLU and C++ concepts earlier. https://github.com/golang/proposal/blob/master/design/go2draft-generics-overview.md https://github.com/golang/proposal/blob/master/design/go2dra... And naturally Rob Pike's pearl, > think the transition to getting polymorphism into the language through generics is has still got some years to work through there's still things that don't quite work right https://youtu.be/yE5Tpp2BSGw?t=2134 https://youtu.be/yE5Tpp2BSGw?t=2134
- neilalexander 1mo ago
  It's not really a hack. It's a hint to a static analyser, that's all.
  - woadwarrior01 1mo ago
    Most languages would encode such behavior in a trait or a protocol instead of a zero-length struct field. It's type information and ought to be encoded in the type. From that perspective, I think it is a hack.
    - kbolino 1mo ago
      This feels like a style complaint and not one about substance. An inaccessible (because it's named _) zero-length struct field is just another kind of metadata. It also doesn't require you to pollute the method set, which would be a bigger issue. The real hack to me is that anything which simply has Lock() and Unlock() methods is considered uncopyable.
      - reorder9695 1mo ago
        Coming from Rust anyway (can't say I'm familiar with Go), nothing about having it as a trait would pollute the method set, traits don't have to have methods (e.g. pin, unpin, send, sync, etc.). Yes a zero length field can be metadata and this is a style complaint but a struct's fields are traditionally its data and the type is its metadata, I feel like it's harmful to mix these two as I at least wouldn't generally look at fields to find metadata for the struct.
        kbolino 1mo ago
        Interfaces in Go (the closest equivalent to traits in Rust) don't have to have methods either, but they are structurally typed. This is like "static duck typing" if you will. So an interface with no methods is implemented by every type in the language (indeed, this became such a useful pattern that the name "any" was reserved for it in Go 1.18). Marker interfaces can exist in Go, but here they are another kind of hack. They must define at least one method (which doesn't have to be public), though that method is never actually meant to be called.
- never_inline 1mo ago
  Every language can't be rust.
- breakingcups 1mo ago
  I'd want to blame Go's compatibility guarantee for this, but I can't because it wouldn't actually stop them from adding a proper solution for this.. Just like the magic comments, it's a sad thing to see appear in this language because it feels like magic incantations one has to know that bend over backwards to not actually extend the language to fit the use case.
  - programcookie 1mo ago
    This shipped with Go 1.7, almost ten years ago to the day.
    - breakingcups 1mo ago
      I don't think this changes anything about the point I was making.
- stevecoalbear 1mo ago
  Go's full of hacks, and holds no shame over it. Zero-initialized everything, and proceeding to 'defer' instead of RAII, generic builtin types despite lack of generics (until recently), no builtin list type, slices having capacity... This was Go's design philosophy until Rob Pike left - to do the simple thing simply and not try to be clever about it.
  - tialaramex 1mo ago
    > no builtin list type Wait, which thing do you mean by a "list type" ? A growable array type like Rust's Vec<T> or C++ std::vector<T> or the ArrayList type seen in several languages ? Or do you mean a linked list type akin to C++ std::list or std::forward_list or Rust's std::collections::LinkedList ? "List" is vague, which is appropriate if you're talking about very high level abstractions where it doesn't matter how it works and 5 gigabytes, 5 bits, 5 weeks or 5 seconds are all finite so who cares - but in the real world we usually do care.
    - Groxx 1mo ago
      Yeah, I really don't see much of a reason for Go to get a first-party linked list type. In most cases in non-list-oriented languages the performance and ergonomics are awful compared to a competent growable array type, the mechanical sympathy of "real" linked lists is generally terrible. Plus they're extremely easy to build if you truly have a good use for one, particularly with generics.
    - deleted 1mo ago
      [deleted]
    - stevecoalbear 1mo ago
      It doesn't matter which one you mean, because Go has neither of them.
  - josephg 1mo ago
    It’s not simple though. The language is simpler, sure. But you pay for language simplicity with program complexity. In go, you have to write and debug a lot more code. I don’t mind spending a few extra weeks learning a more complex language if doing so saves me months of time down the track programming and debugging. That is an excellent investment.
- saturn_vk 1mo ago
  So, like https://doc.rust-lang.org/std/marker/index.html https://doc.rust-lang.org/std/marker/index.html?
  - shakow 1mo ago
    Those are integrated in the compiler and don't need a second tool to work. Also, AFAIK, they only leverage standard language behaviour and are not a hardcoded special case.
    - Georgelemental 1mo ago
      Nearly everything in that module relies on at least some hardcoded compiler magic (what Rust calls "language items", with a `#[lang = "..."]` attribute on the definition). The only exception is `Send`—and even that is still an auto trait, which on stable Rust can only be defined by the standard library. (There are plans to also remove the lang-item status from `Unpin`.)
      - shakow 1mo ago
        Fair point, I was only looking at the structs, not the traits.
- deleted 1mo ago
  [deleted]
CamouflagedKiwi 1mo ago
This feels like it should ideally be something public in the structs package so anyone can leverage it, not just a specially blessed internal thing for the sync package.
- 0x696C6961 1mo ago
  https://github.com/golang/go/issues/70811 https://github.com/golang/go/issues/70811 Go moves slowly
- wbl 1mo ago
  You can very easily define one yourself.
  - kbolino 1mo ago
    You can exploit the mechanism described in the article yourself, but it's already changed once in the past and is not part of any compatibility guarantee. As with structs.HostLayout, a blessed structs.NoCopy in the standard library could guarantee that it works forever. I think the bigger issue remains that it doesn't actually do anything in the language (but, then again, neither does structs.HostLayout--yet).
- ape4 1mo ago
  I agree, its a nice piece of semantics to be added to a struct
- advisedwang 1mo ago
  It's not a specially blessed type. As the article says, anything that implements sync.Locker acts like this.
deleted 1mo ago
[deleted]
meerita 1mo ago
I spoke with Aliaksandr Valialkin (author of noCopy) and he gave me his reasons: - https://x.com/valyala/status/2088638160242683954 https://x.com/valyala/status/2088638160242683954 He also gave an answer of what he would change now: https://itnext.io/go-evolves-in-the-wrong-direction-7dfda8a1a620 https://itnext.io/go-evolves-in-the-wrong-direction-7dfda8a1... It seems he's not happy anymore with the new direction of Go because they're implementing things from other languages.
- fithisux 1mo ago
  He's right. Countering the performance advantage of Rust and keeping its simplicity would be higher priority.
- zarzavat 1mo ago
  > I'd remove user-defined generics from Go, and all the overcomplicated shit related to them, including iterator functions. Go advances one blub ragequit at a time.
fithisux 1mo ago
Very good article. Interesting approach working in harmony with `go vet`.

70 points by func25 2mo ago

> noCopy is a special marker for types that must not be copied after their first use. if it looks like a hack, walks like a hack, and quacks like a hack...

pjmlp 1mo ago
The whole Go design philosophy in one sentence, that is what one gets by refusing to adopt modern language practices.
- deleted 1mo ago
  [deleted]
- 0x696C6961 1mo ago
  If this was true, why did they add generics?
  - pjmlp 1mo ago
    The generics that they eventually added, because of programming language market pressure, Kubernetes had an whole code generation gizmo to work around it, are a clunky implementation versus what could have been done if done from the get go.
    - 0x696C6961 1mo ago
      The generics implementation is perfectly fine. The fact that you completely dodged my question is telling. But please, keep living in your alternate reality.
      - pjmlp 1mo ago
        I did not dodge the question at all, you just didn't like the answer. "Market pressure for language adoption". Just like all the features they keep adding where they said that Go doesn't need them in first place, turns out those features exist in other programming languages for a reason.
        0x696C6961 1mo ago
        This idea that they just caved and added them due to pressure is a false narrative invented by detractors like yourself. Here is a timeline if you're interested: 2009 Go released. June 2010 https://github.com/golang/proposal/blob/master/design/15292/2010-06-type-functions.md https://github.com/golang/proposal/blob/master/design/15292/... Jan 2011 https://github.com/golang/proposal/blob/master/design/15292-generics.md https://github.com/golang/proposal/blob/master/design/15292-... March 2011 https://github.com/golang/proposal/blob/master/design/15292/2011-03-gen.md https://github.com/golang/proposal/blob/master/design/15292/... Oct 2013 https://github.com/golang/proposal/blob/master/design/15292/2013-10-gen.md https://github.com/golang/proposal/blob/master/design/15292/... Dec 2013 https://github.com/golang/proposal/blob/master/design/15292/2013-12-type-params.md https://github.com/golang/proposal/blob/master/design/15292/... April 2016 https://github.com/golang/go/issues/15292 https://github.com/golang/go/issues/15292 Aug 2018 https://github.com/golang/proposal/blob/master/design/go2draft-generics-overview.md https://github.com/golang/proposal/blob/master/design/go2dra... Aug 2018 https://github.com/golang/proposal/blob/master/design/go2draft-contracts.md https://github.com/golang/proposal/blob/master/design/go2dra... June 2020 https://go.dev/blog/generics-next-step https://go.dev/blog/generics-next-step Jan 2021 https://github.com/golang/go/issues/43651 https://github.com/golang/go/issues/43651 Jan 2021 https://github.com/golang/proposal/blob/master/design/43651-type-parameters.md https://github.com/golang/proposal/blob/master/design/43651-... Feb 2021 Proposal accepted. March 2022 Ships in Go 1.18. You could criticize Go for moving too slow. But I'm pretty happy with the way they landed. They definitely don't feel bolted-on.
        pjmlp 1mo ago
        Ah the usual barrage of links from Go folks, luckily we have also come prepared. > In three years of Go surveys, lack of generics has always been listed as one of the top three problems to fix in the language. https://blog.golang.org/why-generics https://blog.golang.org/why-generics > In retrospect, we were biased too much by experience with C++ without concepts and Java generics. We would have been well-served to spend more time with CLU and C++ concepts earlier. https://github.com/golang/proposal/blob/master/design/go2draft-generics-overview.md https://github.com/golang/proposal/blob/master/design/go2dra... And naturally Rob Pike's pearl, > think the transition to getting polymorphism into the language through generics is has still got some years to work through there's still things that don't quite work right https://youtu.be/yE5Tpp2BSGw?t=2134 https://youtu.be/yE5Tpp2BSGw?t=2134
neilalexander 1mo ago
It's not really a hack. It's a hint to a static analyser, that's all.
- woadwarrior01 1mo ago
  Most languages would encode such behavior in a trait or a protocol instead of a zero-length struct field. It's type information and ought to be encoded in the type. From that perspective, I think it is a hack.
  - kbolino 1mo ago
    This feels like a style complaint and not one about substance. An inaccessible (because it's named _) zero-length struct field is just another kind of metadata. It also doesn't require you to pollute the method set, which would be a bigger issue. The real hack to me is that anything which simply has Lock() and Unlock() methods is considered uncopyable.
    - reorder9695 1mo ago
      Coming from Rust anyway (can't say I'm familiar with Go), nothing about having it as a trait would pollute the method set, traits don't have to have methods (e.g. pin, unpin, send, sync, etc.). Yes a zero length field can be metadata and this is a style complaint but a struct's fields are traditionally its data and the type is its metadata, I feel like it's harmful to mix these two as I at least wouldn't generally look at fields to find metadata for the struct.
      - kbolino 1mo ago
        Interfaces in Go (the closest equivalent to traits in Rust) don't have to have methods either, but they are structurally typed. This is like "static duck typing" if you will. So an interface with no methods is implemented by every type in the language (indeed, this became such a useful pattern that the name "any" was reserved for it in Go 1.18). Marker interfaces can exist in Go, but here they are another kind of hack. They must define at least one method (which doesn't have to be public), though that method is never actually meant to be called.
never_inline 1mo ago
Every language can't be rust.
breakingcups 1mo ago
I'd want to blame Go's compatibility guarantee for this, but I can't because it wouldn't actually stop them from adding a proper solution for this.. Just like the magic comments, it's a sad thing to see appear in this language because it feels like magic incantations one has to know that bend over backwards to not actually extend the language to fit the use case.
- programcookie 1mo ago
  This shipped with Go 1.7, almost ten years ago to the day.
  - breakingcups 1mo ago
    I don't think this changes anything about the point I was making.
stevecoalbear 1mo ago
Go's full of hacks, and holds no shame over it. Zero-initialized everything, and proceeding to 'defer' instead of RAII, generic builtin types despite lack of generics (until recently), no builtin list type, slices having capacity... This was Go's design philosophy until Rob Pike left - to do the simple thing simply and not try to be clever about it.
- tialaramex 1mo ago
  > no builtin list type Wait, which thing do you mean by a "list type" ? A growable array type like Rust's Vec<T> or C++ std::vector<T> or the ArrayList type seen in several languages ? Or do you mean a linked list type akin to C++ std::list or std::forward_list or Rust's std::collections::LinkedList ? "List" is vague, which is appropriate if you're talking about very high level abstractions where it doesn't matter how it works and 5 gigabytes, 5 bits, 5 weeks or 5 seconds are all finite so who cares - but in the real world we usually do care.
  - Groxx 1mo ago
    Yeah, I really don't see much of a reason for Go to get a first-party linked list type. In most cases in non-list-oriented languages the performance and ergonomics are awful compared to a competent growable array type, the mechanical sympathy of "real" linked lists is generally terrible. Plus they're extremely easy to build if you truly have a good use for one, particularly with generics.
  - deleted 1mo ago
    [deleted]
  - stevecoalbear 1mo ago
    It doesn't matter which one you mean, because Go has neither of them.
- josephg 1mo ago
  It’s not simple though. The language is simpler, sure. But you pay for language simplicity with program complexity. In go, you have to write and debug a lot more code. I don’t mind spending a few extra weeks learning a more complex language if doing so saves me months of time down the track programming and debugging. That is an excellent investment.
saturn_vk 1mo ago
So, like https://doc.rust-lang.org/std/marker/index.html https://doc.rust-lang.org/std/marker/index.html?
- shakow 1mo ago
  Those are integrated in the compiler and don't need a second tool to work. Also, AFAIK, they only leverage standard language behaviour and are not a hardcoded special case.
  - Georgelemental 1mo ago
    Nearly everything in that module relies on at least some hardcoded compiler magic (what Rust calls "language items", with a `#[lang = "..."]` attribute on the definition). The only exception is `Send`—and even that is still an auto trait, which on stable Rust can only be defined by the standard library. (There are plans to also remove the lang-item status from `Unpin`.)
    - shakow 1mo ago
      Fair point, I was only looking at the structs, not the traits.
deleted 1mo ago
[deleted]

CamouflagedKiwi 1mo ago

This feels like it should ideally be something public in the structs package so anyone can leverage it, not just a specially blessed internal thing for the sync package.

0x696C6961 1mo ago
https://github.com/golang/go/issues/70811 https://github.com/golang/go/issues/70811 Go moves slowly
wbl 1mo ago
You can very easily define one yourself.
- kbolino 1mo ago
  You can exploit the mechanism described in the article yourself, but it's already changed once in the past and is not part of any compatibility guarantee. As with structs.HostLayout, a blessed structs.NoCopy in the standard library could guarantee that it works forever. I think the bigger issue remains that it doesn't actually do anything in the language (but, then again, neither does structs.HostLayout--yet).
ape4 1mo ago
I agree, its a nice piece of semantics to be added to a struct
advisedwang 1mo ago
It's not a specially blessed type. As the article says, anything that implements sync.Locker acts like this.

deleted 1mo ago

[deleted]

meerita 1mo ago

I spoke with Aliaksandr Valialkin (author of noCopy) and he gave me his reasons: - https://x.com/valyala/status/2088638160242683954 https://x.com/valyala/status/2088638160242683954 He also gave an answer of what he would change now: https://itnext.io/go-evolves-in-the-wrong-direction-7dfda8a1a620 https://itnext.io/go-evolves-in-the-wrong-direction-7dfda8a1... It seems he's not happy anymore with the new direction of Go because they're implementing things from other languages.

fithisux 1mo ago
He's right. Countering the performance advantage of Rust and keeping its simplicity would be higher priority.
zarzavat 1mo ago
> I'd remove user-defined generics from Go, and all the overcomplicated shit related to them, including iterator functions. Go advances one blub ragequit at a time.

fithisux 1mo ago

Very good article. Interesting approach working in harmony with `go vet`.