5 ms·

Ruby's private keyword is weird

109 points by jez 3y ago

LesZedCB 3y ago
yes this makes sense because ruby is a message passing language. even accessors are messages. only self can receive private messages. and the sharing a private instance variable between separate instances example is exactly what protected does, which ruby also has.
- caseyohara 3y ago
  Exactly. And prior to Ruby 2.7, even calling a private method with an explicit `self` as the receiver was not allowed, only the implicit `self` from "within" the class body ("within" in quotes because, as the blog post points out, Ruby's concept of being inside a class body is unlike many other languages).
- jez 3y ago
  I don't think this has to be the case! I think you could build a different language with the same message passing semantics as Ruby, but with the limitations that classes are not allowed to be re-opened and method definitions must appear syntactically at the top-level of a class (and maybe a few other limitations), and then build something more similar to how `private` works in a traditional, statically-typed language. You definitely don't arrive at a language that has all the same feeling of Ruby, but you could still have the "everything is an object, and everything is message passing" character to the language.
stouset 3y ago
I’ve only glanced quickly at the article, but I think the author’s understanding of private in Ruby is wrong. class Parent attr_accessor :x private :x def initialize(x) self.x = x # └── (1) allowed end def ==(other) self.x == other.x # │ └── (2) not allowed # └── (1) allowed end end This is not doing what the author thinks it is doing. Or, it is, but I don’t think the authors understand why. `private :x` only marks the x method as private, but `x=` is a different method. They would see different behavior with private :x private :x= In general, public, private, and protected do the following. Public methods can be called with any receiver: explicit (x.foo()), self (self.foo()), or implicit (foo()). Protected methods can be invoked on self or with an implicit receiver. Private methods can only be invoked on an implicit receiver. Note that this has nothing to do with the class hierarchy. A private method can be called from a subclass as long as it has an implicit receiver. The difference between protected and private is not really practically useful, so I personally use them to signal different levels of “private-ness”. I use protected methods to implement a class’ “internal API”, which is the core thing the public API is abstracting and exposing. I use private methods for implementing ancillary details that aren’t really core to what the class does but handle some one-off unrelated data manipulation or munging. But that’s just me. I’ll also admit that something may have changed with Ruby’s rules around these keywords around 3.0 that I missed, so if that’s the case I apologize.
- jez 3y ago
  Sorry, that’s simply a typo translating from Java to Ruby. I believe the point that section is trying to make still holds after editing it to say “private :x, :x=” > Private methods can only be invoked on an implicit receiver. This changed in Ruby 2.7. You can now include the self keyword, which is important because it allows calling methods that share names with Ruby keywords. “begin” is a keyword but “self.begin” is a call to a method named begin. So if that method happens to be private, you can call it now in Ruby 2.7 without needing “send(:begin)” Other than the typo (thanks! I’ll fix that when I’m back at my computer), I think that the rest of the post underscores what you’ve already pointed out about how private works, modulo the misunderstanding about the changes in Ruby 2.7
  - stouset 3y ago
    Aha! Glad I caveated things by saying I may have missed a change in semantics. I’m surprised they did that in a minor release and not a major one. So, then, is there no actual difference between private and protected now?
    - jez 3y ago
      The difference between private and protected is a bit subtle so I’ll defer to the Ruby FAQ: https://www.ruby-lang.org/en/documentation/faq/7/ https://www.ruby-lang.org/en/documentation/faq/7/ (On this page there’s a section called “What’s the difference between private and protected?” which explains it.) It’s interesting in its own right, and it’s in my backlog to blog about because I don’t like any of the existing resources contrasting the two.
      - tsimionescu 3y ago
        From the examples in this FAQ, it seems like protected works a bit more like Java's private - you can use it only inside the class, but for any object of that class, not just self, right? (with the exception that it applies to subclasses) So Ruby's private only allows you to interact with other methods on the current object. Protected allows objects of the same class to send each other the message, but not other classes. Would this be a fair summary?
lmm 3y ago
Some of the reason for Scala having `private[this]` is that it allows an implementation that behaves (and performs) like a Java private field. Scala fields generally aren't raw fields at the JVM level (because of Scala's "uniform access principle" that `def x` should be accessible like `val x`), but private[this] fields allow the compiler to represent them as raw fields since they really can't be accessed elsewhere. The only other thing I'm aware of being allowed to go in the[] part of a Scala access modifier is a containing package (or object?), which allows you to emulate Java's "default" (AKA "package-private") visibility.
- jez 3y ago
  That’s cool! Thanks for sharing, I had never considered that aspect of private[this]. Mentioned in the article, but it also powers covariant classes. In a language like C# only interfaces can have covariant type parameters, because it wouldn’t be sound to store a value in a (C#) private field if it had a covariant type. This influences a lot of the API design in C# where basically every container a covariant/immutable interface alongside the invariant/mutable implementation. (Java gets around this with use site variance annotations which are more flexible but have their own drawbacks).
3np 3y ago
> So far, I’m only aware of Ruby’s private modifier and Scala’s protected[this] which behave like this. If you know of any other languages, please email me! I’d love to hear about them. Rerouting throgh comments: Event if it's very different, TypeScript's private has quite surprising semantics as well...
- tubthumper8 3y ago
  TypeScript's `private` keyword is like the Pirate's Code: > More like _guidelines_, than actual rules
thayne 3y ago
protected[this] was removed in scala 3[0]. And honestly, I can't think of a case where it was really useful. [0]: https://docs.scala-lang.org/scala3/reference/dropped-features/this-qualifier.html https://docs.scala-lang.org/scala3/reference/dropped-feature...
- jez 3y ago
  Thank you for sharing! It looks like the Scala project found allowing protected[this] fields as an escape hatch for variance to be unsound: https://github.com/scala/bug/issues/7093 https://github.com/scala/bug/issues/7093 Going to have to chew on that and what it means for the same feature in Sorbet… Thanks again for bringing this up!
dmurray 3y ago
The other approach to private methods is weird. It breaks the object-oriented paradigm. Why should my Dog be able to check out the internal StomachContents of another Dog, while that information isn't available to a Vet or even to the Dog's Owner? Possibly the Dog can even check out the StomachContents of a Wolf or an Elephant, depending on your language and your inheritance rules. Even if the Dog otherwise has no relation to the second animal. It's because we're seeing behind the curtain. The internal implementation of method Fido.vomit() is some equivalent of Dog::vomit(Fido) so the current object isn't actually in as privileged a position as it might appear to a reader. The language privileges all members of class Dog instead. Ruby has the same problem even more, because the first thing any Ruby library author does is to monkeypatch everyone else's objects. It solves it in the opposite way, by really trying to make the language know when an object is calling a method on itself, but again the implementation leaks and the language runtime can't always know that. Which is fine - Ruby's philosophy is to provide convenient escape hatches for this kind of thing anyway, and to limit you by convention rather than by compiler errors.
- Rodeoclash 3y ago
  It's a great example and I appreciate you writing it out, but Dog.stomacheContents... you must be programming dwarf fortress!
- masa331 3y ago
  > the first thing any Ruby library author does is to monkeypatch everyone else's objects Serious Ruby libraries don't do that very often
- wruza 3y ago
  Visibility rules are not for objects. These exist for defining class implementation / maintenance borders. Iow, for gods of all dogs and elephants to not step on each other feet. Objects themselves have nothing to do with that and OOP-world analogy was always shallow to begin with, at least in mainstream OOP.
  - nsonha 3y ago
    > OOP-world analogy was always shallow to begin with I would also add that the purpose of visibility is to encapsulate and not some sort of access control. All the objects of the same class are implemented with the same code so they are within the same encapsulation boundary.
uticus 3y ago
I’ll add to existing comments that it might also be interesting to look through conversations re Smalltalk. Ruby was influenced by Smalltalk, so interesting to note in Smalltalk it is impossible to have private methods. https://stackoverflow.com/q/7399340 https://stackoverflow.com/q/7399340 http://computer-programming-forum.com/3-smalltalk/5139b6267b2d0b78.htm http://computer-programming-forum.com/3-smalltalk/5139b6267b... …etc
furyofantares 3y ago
> The member x cannot be accessed in subclasses of Parent, like Child This is a typo and should say "can", right?
- jez 3y ago
  Yep, another bad copy paste from the previous section. Thanks! Will fix in a bit.
aardvark179 3y ago
Ruby's behaviour is weird if you're used to many other OO languages, but it shares that weirdness with other languages influenced by Smalltalk, such as the Magik languages used in the Smallworld GIS. Both ideas of what private means make sense, but I can certainly understand the confusion they cause if you expect one behaviour and discover your language does something different.
- sfn42 3y ago
  That's why I always approach new languages as a complete beginner. It's tempting to jump into the deep end thinking you know the basics, but in my experience it has a lot of pitfalls. I like to find a book or "web book" with exercises such as http://JavaScript.info http://JavaScript.info and just follow it. Read it and do all the exercises. It takes some time and some times it feels a bit like the exercises are too easy, but often times I find that they are deceivingly complex precisely because I'm misunderstanding something. It's like a win/win, either I do the exercise in a few minutes because it really was easy, or I learn something. I enjoy entering zen mode and just banging out a ton of exercises. I see a lot of people consuming video tutorials/courses without exercises and I think they're doing themselves a disservice. They get to chill and just watch a video and feel like they're learning but most of the time they really arent. I see people waste hours and hours watching courses or transcribing code from tutorials and the moment they try to do anything on their own they realize they have absolutely no idea what they're doing. It's just a huge waste of time. People are different and stuff but I don't really buy the whole "people learn differently" thing. Everyone learns best by doing, and the structured "read about a topic, then test your knowledge by completing tailored exercises for that topic" approach allows you to have a linear and thorough path from zero to fairly advanced. I've tried books without exercises and I hated them. I'm not opposed to looking up videos on specific topics you struggle with, but I think videos are best used as a supplement rather than a primary form of education.
kevincox 3y ago
Python takes an interesting approach that mimics the behaviour of statically-typed languages without static typing. If you have a member that starts with `__` then it gets rewritten. For example's sake let's say that `self.__x` in class `Parent` becomes `__Parent_x`. Also note that because the resulting name starts with `__` as well you can't easily just do this mangling yourself as writing `p.__Parent_x` in class `Other` will result in `p.__Other_Parent_x`. (But you can easily access via something like `getattr(p, "__Parent_x")`). This means that `self.__x == that.__x` will desugar to something like `self.__Parent_x == that.__Parent_x` so you will be able to access privates of other members of the same class. It also means that subclasses won't be able to access private members because `self.__x` will become `self.__Child_x`. IIUC JavaScript has something similar for `this.#x`. Although this is implemented much more robustly and doesn't rely on unique class names and I believe is actually "secure" as in no other class can gain access. But I am not up-to-date on JS reflection abilities. The code desugars to something like `const X_PROP = new Symbol()` created once-per-class and accessible only to the class itself, then the accesses become something like `this[X_PROP]`.
- jez 3y ago
  Fascinating, thank you for sharing! More about this Python feature in the docs: https://docs.python.org/3/tutorial/classes.html#private-variables https://docs.python.org/3/tutorial/classes.html#private-vari...
- tubthumper8 3y ago
  > IIUC JavaScript has something similar for `this.#x`. Although this is implemented much more robustly and doesn't rely on unique class names and I believe is actually "secure" as in no other class can gain access. But I am not up-to-date on JS reflection abilities. The code desugars to something like `const X_PROP = new Symbol()` created once-per-class and accessible only to the class itself, then the accesses become something like `this[X_PROP]`. Documentation: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Classes/Private_properties https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... TL;DR it's a language feature, there's no reflection and it's not a syntax sugar for `new Symbol()`. It's actually implemented as a _syntax_ error to access the private member outside the class, which is made possible by using a sigil `#`

109 points by jez 3y ago

yes this makes sense because ruby is a message passing language. even accessors are messages. only self can receive private messages. and the sharing a private instance variable between separate instances example is exactly what protected does, which ruby also has.

caseyohara 3y ago
Exactly. And prior to Ruby 2.7, even calling a private method with an explicit `self` as the receiver was not allowed, only the implicit `self` from "within" the class body ("within" in quotes because, as the blog post points out, Ruby's concept of being inside a class body is unlike many other languages).
jez 3y ago
I don't think this has to be the case! I think you could build a different language with the same message passing semantics as Ruby, but with the limitations that classes are not allowed to be re-opened and method definitions must appear syntactically at the top-level of a class (and maybe a few other limitations), and then build something more similar to how `private` works in a traditional, statically-typed language. You definitely don't arrive at a language that has all the same feeling of Ruby, but you could still have the "everything is an object, and everything is message passing" character to the language.

stouset 3y ago

I’ve only glanced quickly at the article, but I think the author’s understanding of private in Ruby is wrong. class Parent attr_accessor :x private :x def initialize(x) self.x = x # └── (1) allowed end def ==(other) self.x == other.x # │ └── (2) not allowed # └── (1) allowed end end This is not doing what the author thinks it is doing. Or, it is, but I don’t think the authors understand why. `private :x` only marks the x method as private, but `x=` is a different method. They would see different behavior with private :x private :x= In general, public, private, and protected do the following. Public methods can be called with any receiver: explicit (x.foo()), self (self.foo()), or implicit (foo()). Protected methods can be invoked on self or with an implicit receiver. Private methods can only be invoked on an implicit receiver. Note that this has nothing to do with the class hierarchy. A private method can be called from a subclass as long as it has an implicit receiver. The difference between protected and private is not really practically useful, so I personally use them to signal different levels of “private-ness”. I use protected methods to implement a class’ “internal API”, which is the core thing the public API is abstracting and exposing. I use private methods for implementing ancillary details that aren’t really core to what the class does but handle some one-off unrelated data manipulation or munging. But that’s just me. I’ll also admit that something may have changed with Ruby’s rules around these keywords around 3.0 that I missed, so if that’s the case I apologize.

jez 3y ago
Sorry, that’s simply a typo translating from Java to Ruby. I believe the point that section is trying to make still holds after editing it to say “private :x, :x=” > Private methods can only be invoked on an implicit receiver. This changed in Ruby 2.7. You can now include the self keyword, which is important because it allows calling methods that share names with Ruby keywords. “begin” is a keyword but “self.begin” is a call to a method named begin. So if that method happens to be private, you can call it now in Ruby 2.7 without needing “send(:begin)” Other than the typo (thanks! I’ll fix that when I’m back at my computer), I think that the rest of the post underscores what you’ve already pointed out about how private works, modulo the misunderstanding about the changes in Ruby 2.7
- stouset 3y ago
  Aha! Glad I caveated things by saying I may have missed a change in semantics. I’m surprised they did that in a minor release and not a major one. So, then, is there no actual difference between private and protected now?
  - jez 3y ago
    The difference between private and protected is a bit subtle so I’ll defer to the Ruby FAQ: https://www.ruby-lang.org/en/documentation/faq/7/ https://www.ruby-lang.org/en/documentation/faq/7/ (On this page there’s a section called “What’s the difference between private and protected?” which explains it.) It’s interesting in its own right, and it’s in my backlog to blog about because I don’t like any of the existing resources contrasting the two.
    - tsimionescu 3y ago
      From the examples in this FAQ, it seems like protected works a bit more like Java's private - you can use it only inside the class, but for any object of that class, not just self, right? (with the exception that it applies to subclasses) So Ruby's private only allows you to interact with other methods on the current object. Protected allows objects of the same class to send each other the message, but not other classes. Would this be a fair summary?

lmm 3y ago

Some of the reason for Scala having `private[this]` is that it allows an implementation that behaves (and performs) like a Java private field. Scala fields generally aren't raw fields at the JVM level (because of Scala's "uniform access principle" that `def x` should be accessible like `val x`), but private[this] fields allow the compiler to represent them as raw fields since they really can't be accessed elsewhere. The only other thing I'm aware of being allowed to go in the[] part of a Scala access modifier is a containing package (or object?), which allows you to emulate Java's "default" (AKA "package-private") visibility.

jez 3y ago
That’s cool! Thanks for sharing, I had never considered that aspect of private[this]. Mentioned in the article, but it also powers covariant classes. In a language like C# only interfaces can have covariant type parameters, because it wouldn’t be sound to store a value in a (C#) private field if it had a covariant type. This influences a lot of the API design in C# where basically every container a covariant/immutable interface alongside the invariant/mutable implementation. (Java gets around this with use site variance annotations which are more flexible but have their own drawbacks).

3np 3y ago

> So far, I’m only aware of Ruby’s private modifier and Scala’s protected[this] which behave like this. If you know of any other languages, please email me! I’d love to hear about them. Rerouting throgh comments: Event if it's very different, TypeScript's private has quite surprising semantics as well...

tubthumper8 3y ago
TypeScript's `private` keyword is like the Pirate's Code: > More like _guidelines_, than actual rules

thayne 3y ago

protected[this] was removed in scala 3[0]. And honestly, I can't think of a case where it was really useful. [0]: https://docs.scala-lang.org/scala3/reference/dropped-features/this-qualifier.html https://docs.scala-lang.org/scala3/reference/dropped-feature...

jez 3y ago
Thank you for sharing! It looks like the Scala project found allowing protected[this] fields as an escape hatch for variance to be unsound: https://github.com/scala/bug/issues/7093 https://github.com/scala/bug/issues/7093 Going to have to chew on that and what it means for the same feature in Sorbet… Thanks again for bringing this up!

dmurray 3y ago

The other approach to private methods is weird. It breaks the object-oriented paradigm. Why should my Dog be able to check out the internal StomachContents of another Dog, while that information isn't available to a Vet or even to the Dog's Owner? Possibly the Dog can even check out the StomachContents of a Wolf or an Elephant, depending on your language and your inheritance rules. Even if the Dog otherwise has no relation to the second animal. It's because we're seeing behind the curtain. The internal implementation of method Fido.vomit() is some equivalent of Dog::vomit(Fido) so the current object isn't actually in as privileged a position as it might appear to a reader. The language privileges all members of class Dog instead. Ruby has the same problem even more, because the first thing any Ruby library author does is to monkeypatch everyone else's objects. It solves it in the opposite way, by really trying to make the language know when an object is calling a method on itself, but again the implementation leaks and the language runtime can't always know that. Which is fine - Ruby's philosophy is to provide convenient escape hatches for this kind of thing anyway, and to limit you by convention rather than by compiler errors.

Rodeoclash 3y ago
It's a great example and I appreciate you writing it out, but Dog.stomacheContents... you must be programming dwarf fortress!
masa331 3y ago
> the first thing any Ruby library author does is to monkeypatch everyone else's objects Serious Ruby libraries don't do that very often
wruza 3y ago
Visibility rules are not for objects. These exist for defining class implementation / maintenance borders. Iow, for gods of all dogs and elephants to not step on each other feet. Objects themselves have nothing to do with that and OOP-world analogy was always shallow to begin with, at least in mainstream OOP.
- nsonha 3y ago
  > OOP-world analogy was always shallow to begin with I would also add that the purpose of visibility is to encapsulate and not some sort of access control. All the objects of the same class are implemented with the same code so they are within the same encapsulation boundary.

uticus 3y ago

I’ll add to existing comments that it might also be interesting to look through conversations re Smalltalk. Ruby was influenced by Smalltalk, so interesting to note in Smalltalk it is impossible to have private methods. https://stackoverflow.com/q/7399340 https://stackoverflow.com/q/7399340 http://computer-programming-forum.com/3-smalltalk/5139b6267b2d0b78.htm http://computer-programming-forum.com/3-smalltalk/5139b6267b... …etc

furyofantares 3y ago

> The member x cannot be accessed in subclasses of Parent, like Child This is a typo and should say "can", right?

jez 3y ago
Yep, another bad copy paste from the previous section. Thanks! Will fix in a bit.

aardvark179 3y ago

Ruby's behaviour is weird if you're used to many other OO languages, but it shares that weirdness with other languages influenced by Smalltalk, such as the Magik languages used in the Smallworld GIS. Both ideas of what private means make sense, but I can certainly understand the confusion they cause if you expect one behaviour and discover your language does something different.

sfn42 3y ago
That's why I always approach new languages as a complete beginner. It's tempting to jump into the deep end thinking you know the basics, but in my experience it has a lot of pitfalls. I like to find a book or "web book" with exercises such as http://JavaScript.info http://JavaScript.info and just follow it. Read it and do all the exercises. It takes some time and some times it feels a bit like the exercises are too easy, but often times I find that they are deceivingly complex precisely because I'm misunderstanding something. It's like a win/win, either I do the exercise in a few minutes because it really was easy, or I learn something. I enjoy entering zen mode and just banging out a ton of exercises. I see a lot of people consuming video tutorials/courses without exercises and I think they're doing themselves a disservice. They get to chill and just watch a video and feel like they're learning but most of the time they really arent. I see people waste hours and hours watching courses or transcribing code from tutorials and the moment they try to do anything on their own they realize they have absolutely no idea what they're doing. It's just a huge waste of time. People are different and stuff but I don't really buy the whole "people learn differently" thing. Everyone learns best by doing, and the structured "read about a topic, then test your knowledge by completing tailored exercises for that topic" approach allows you to have a linear and thorough path from zero to fairly advanced. I've tried books without exercises and I hated them. I'm not opposed to looking up videos on specific topics you struggle with, but I think videos are best used as a supplement rather than a primary form of education.

kevincox 3y ago

Python takes an interesting approach that mimics the behaviour of statically-typed languages without static typing. If you have a member that starts with `__` then it gets rewritten. For example's sake let's say that `self.__x` in class `Parent` becomes `__Parent_x`. Also note that because the resulting name starts with `__` as well you can't easily just do this mangling yourself as writing `p.__Parent_x` in class `Other` will result in `p.__Other_Parent_x`. (But you can easily access via something like `getattr(p, "__Parent_x")`). This means that `self.__x == that.__x` will desugar to something like `self.__Parent_x == that.__Parent_x` so you will be able to access privates of other members of the same class. It also means that subclasses won't be able to access private members because `self.__x` will become `self.__Child_x`. IIUC JavaScript has something similar for `this.#x`. Although this is implemented much more robustly and doesn't rely on unique class names and I believe is actually "secure" as in no other class can gain access. But I am not up-to-date on JS reflection abilities. The code desugars to something like `const X_PROP = new Symbol()` created once-per-class and accessible only to the class itself, then the accesses become something like `this[X_PROP]`.

jez 3y ago
Fascinating, thank you for sharing! More about this Python feature in the docs: https://docs.python.org/3/tutorial/classes.html#private-variables https://docs.python.org/3/tutorial/classes.html#private-vari...
tubthumper8 3y ago
> IIUC JavaScript has something similar for `this.#x`. Although this is implemented much more robustly and doesn't rely on unique class names and I believe is actually "secure" as in no other class can gain access. But I am not up-to-date on JS reflection abilities. The code desugars to something like `const X_PROP = new Symbol()` created once-per-class and accessible only to the class itself, then the accesses become something like `this[X_PROP]`. Documentation: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Classes/Private_properties https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... TL;DR it's a language feature, there's no reflection and it's not a syntax sugar for `new Symbol()`. It's actually implemented as a _syntax_ error to access the private member outside the class, which is made possible by using a sigil `#`