I don't follow the point you're trying to make. What I'm referring to is how JavaScript's object orientedness, which relies heavily on references, doesn't play

I don't follow the point you're trying to make. What I'm referring to is how JavaScript's object orientedness, which relies heavily on references, doesn't play very well with immutable values. The only way to really eschew mutability is to ditch the concept of objects representing a state and a set of methods that mutate its state (which maintain encapsulation and enforce separation of concerns). Unless you decide to make all your prototype methods return new instances of the constructor upon mutation, which would be both inefficient and prone to error.