9 ms·

Objc_msgSend's New Prototype

85 points by beefhash 7y ago

dep_b 7y ago
That’s why we’ve been warned for years to not call them directly from C I guess! Mike Ash has such amazing content always.
- saagarjha 7y ago
  I don’t see why you can’t call objc_msgSend directly if you cast it correctly.
  - dep_b 7y ago
    You _can_ call it as you could call it before as well only there's a pretty big chance your existing code will break because of this change.
    - braindeath 7y ago
      No that’s not right. The implementation is not changing. The prototype is changing to break code that was poorly formed in the first place. If you had cast to the correct type in your code as you always should have, this change shouldn’t break anything. As far as code breaking changes go, the impact should be low. How many times in a code base should you need to call msg send? And the risk of problems is low. Only if your code was silently relying on float corruption would you notice a change. Anybody making six figures should be able to analyze such a thing without too much whining.
      - dep_b 7y ago
        Well I'm not sure either who would rely on extensive calling of this function but why would you call it "whining"? Apart from the fact that earning six figures still isn't the norm for most Objective-C developers. > The prototype is changing to break code that was poorly formed in the first place. I guess calling it directly would even be poor form by itself so who knows.
        braindeath 7y ago
        > I guess calling it directly would even be poor form by itself so who knows. The point being that there are limited cases where it is justifiable. For example, a VM or language interop would likely be justified. Those types of cases mean the absolute number of call sites should be small though. In other words, the problem is somewhat self-limiting. Probably, in appropriate use, the function would be called extensively, but likely not from many sites. > Apart from the fact that earning six figures still isn't the norm for most Objective-C developers. I have no horse in this race.. I don't work in the industry for years. This was also a somewhat tongue-in-cheek remark that I think you are reading too deeply into. The real point is, software engineers are generally paid well relative to median incomes and have a cushy job. On the scale of pain, this doesn't seem like a big deal - not even a Python 2 to 3 transition. BLS average salary for software engineers is $103k, not including bonuses and stock, etc. I would think the average engineer that would justifiably use msg send directly is being paid greater than the BLS average - which includes all the enterprise LOB app slaves that don't do incredibly well.
- vbezhenar 7y ago
  How are you supposed to call Objective C code from C then?
  - 0x0 7y ago
    Call it from an extern C function in an .m file?
    - vbezhenar 7y ago
      Not very practical, if you need to use a lot of APIs. One of advantage of Objective C is that it's easy to interop with C.
      - pjmlp 7y ago
        You are seeing the other way around. That advantage is to migrate C code into Objective-C, just like C++ and TypeScript have done as well. In fact, NeXTSTEP and its derived OSes have very little pure C code, beyond BSD/XNU kernel and the POSIX APIs.
        monocasa 7y ago
        You'd be surprised. The new APFS.Framework appears to be all C for instance.
        pjmlp 7y ago
        Even then you're supposed to actually use the higher level filesystem framework APIs from Objective-C and Swift. https://developer.apple.com/documentation/foundation/file_system/about_apple_file_system https://developer.apple.com/documentation/foundation/file_sy...
        monocasa 7y ago
        I mean, yeah, you're not "supposed to" use it; it's a private framework. Also, the only reason why someone like me would know if it's in C or not is that I need one of the aspects of that library that can't be expressed in the high level APIs. My point is that there's tons of non legacy C code, in the OS outside of the kernel and posix subsystems on *OS.
- mpweiher 7y ago
  You can still call objc_msgSend() from C. Just need to cast it correctly.
saagarjha 7y ago
I wonder if the new signature has any effect on pointer authentication (or was related to those changes). Also, I think the title would be better served with the correct capitalization of the method name, objc_msgSend.
andr 7y ago
Are there any conditions where the compiler can optimize away the call to objc_msgSend? Or is it always used for any call between 2 ObjC/Swift methods?
- vidarh 7y ago
  I don't know about Objective-C specifically, but generally one way to achieve this in most dynamic languages is a speed/memory tradeoff. In my Ruby compiler project which has to deal with the same level of dynamic behaviour, I handle dynamic overriding of methods with C++-style vtables, which turns method calls into the equivalent of this C-ish pseudo-code: (* ob->vtable[some_method_offset])(args...) Since Ruby classes can have a method_missing handling undefined methods, for any class that doesn't implement a given method, the vtable contains a pointer to a thunk that tweaks the stack to push the relevant method symbol as the first argument and then does the same as above with the method offset of method_missing. Since Ruby classes can have methods overridden at any time, if class Bar inherits from class Foo, inherits from class Object, and I override a method in class Object, that again has previously been explicitly overridden in class Bar, this would happen: - Store pointer to method in Object in ptr. - Replace method pointer in Object's vtable. - Iterate over all direct sub-classes of Object (but here we only care about Foo) - Compare the same method offset against ptr. Since Foo has not overridden the method, it matches. - Replace the pointer in Foo, and iterate over all direct sub-classes of Foo (but here we only care about Bar) - Compare the method offset against ptr. Since Bar has overridden the method, it doesn't match, so leave it alone. This means that as long as method overrides doesn't happen extremely frequently, the cost of method overrides is relatively low: iterate over all the descendant classes of the class you override a method in. Method calls on the other hand are about as cheap as virtual method calls in C++, except when you hit method_missing where this approach gives you a very low extra overhead of tweaking the stack to add the symbol and jumping to the method_missing implementation. This overall approach works for most dynamic languages. The caveat is memory - if you have an application with very large class hierarchies in a language where they are all singly rooted (as in Ruby where they all ultimately inherit from SimpleObject), each vtable will cost you at last pointer_size*global_number_of_method_names. In practice so far I've not seen all that many cases where this is a problem, and it's always possible for the compiler to set a roof above which it will resort to a slow send mechanism (because you'll need to support that anyway in any language that allows dynamic send mechanics; e.g. in Ruby you can always send a message to an object by a dynamically obtained symbol so you still need the equivalent of objc_msgSend as well). A slightly cheaper approach in terms of memory was described by Michael Franz[1]. His approach was to group methods in interfaces, so instead of a vtable of method pointers, you had a vtable of pointers to interfaces with pointers to methods. You save memory as most classes would typically implement most or none of the methods of an interface; it provides potential namespacing of the methods if you want to do that, and you can cut memory further by re-using the same vtable for an interface until someone tries to override at least one method in it. The cost is one extra indirection at call-sites. [1] Protocol Extension: A Technique for Structuring Large Extensible SoftwareSystems (1994) https://pdfs.semanticscholar.org/60d3/045d924dfb912e184014aa2c004214fb8236.pdf https://pdfs.semanticscholar.org/60d3/045d924dfb912e184014aa...
dahart 7y ago
> As long as you cast it to the correct type, it always works. The new way of doing things thus encourages doing things correctly and makes it harder to do things wrong. This seems like a really interesting case where removing type information, removing something was presumably thought to be a safety guard, and requiring type casting, makes the type safety stronger by forcing the user to do it explicitly. Maybe it makes sense that dynamic calling mechanisms should require casting (and static type checking is less safe) where static calling mechanisms should use static type checking (and type casting is less safe). > C specifies that certain types get promoted to wider types when passed as a variadic argument. Integers smaller than int (such as char and short) get promoted to int, and float gets promoted to double. If your method signature includes one of these types, it's not possible for a caller to pass a parameter as that exact type if it's using a variadic prototype. Holy crap! I’ve used these things for many years without knowing this important tidbit. Does anyone know if it’s the same in C++, or whether GPUs are currently ignoring or following the spec?
- emilfihlman 7y ago
  I mean, you have seen how printf uses %d for char, short and int as a formatter?
  - dahart 7y ago
    Yeah, of course, but that doesn’t by itself lead me to the conclusion that those are all first converted to another type before being passed on the stack. They could be passed as-is and type-cast inside the printf implementation. The more interesting printf example would be %f, since according to what I learned today, it’s not a float, it’s a double. Are you saying you already knew doubles were passed in variadics instead of float, and the reason you knew is printf?
    - microtherion 7y ago
      > They could be passed as-is and type-cast inside the printf implementation. How would you know what to type-cast FROM? It's not like C compilers pass type information around, nor are C values self describing. > Are you saying you already knew doubles were passed in variadics instead of float, and the reason you knew is printf? I can't speak for the person you're answering to, but in pre-ANSI C, these promotions were ubiquitous (since there were no function prototypes yet), so programmers were very likely to have heard about them at some point.
tpush 7y ago
As an aside, am I missing something or is Apple's doc on objc_msgSend describing the parameters of the old form? The parameters section doesn't seem to match the declaration section.

85 points by beefhash 7y ago

That’s why we’ve been warned for years to not call them directly from C I guess! Mike Ash has such amazing content always.

saagarjha 7y ago
I don’t see why you can’t call objc_msgSend directly if you cast it correctly.
- dep_b 7y ago
  You _can_ call it as you could call it before as well only there's a pretty big chance your existing code will break because of this change.
  - braindeath 7y ago
    No that’s not right. The implementation is not changing. The prototype is changing to break code that was poorly formed in the first place. If you had cast to the correct type in your code as you always should have, this change shouldn’t break anything. As far as code breaking changes go, the impact should be low. How many times in a code base should you need to call msg send? And the risk of problems is low. Only if your code was silently relying on float corruption would you notice a change. Anybody making six figures should be able to analyze such a thing without too much whining.
    - dep_b 7y ago
      Well I'm not sure either who would rely on extensive calling of this function but why would you call it "whining"? Apart from the fact that earning six figures still isn't the norm for most Objective-C developers. > The prototype is changing to break code that was poorly formed in the first place. I guess calling it directly would even be poor form by itself so who knows.
      - braindeath 7y ago
        > I guess calling it directly would even be poor form by itself so who knows. The point being that there are limited cases where it is justifiable. For example, a VM or language interop would likely be justified. Those types of cases mean the absolute number of call sites should be small though. In other words, the problem is somewhat self-limiting. Probably, in appropriate use, the function would be called extensively, but likely not from many sites. > Apart from the fact that earning six figures still isn't the norm for most Objective-C developers. I have no horse in this race.. I don't work in the industry for years. This was also a somewhat tongue-in-cheek remark that I think you are reading too deeply into. The real point is, software engineers are generally paid well relative to median incomes and have a cushy job. On the scale of pain, this doesn't seem like a big deal - not even a Python 2 to 3 transition. BLS average salary for software engineers is $103k, not including bonuses and stock, etc. I would think the average engineer that would justifiably use msg send directly is being paid greater than the BLS average - which includes all the enterprise LOB app slaves that don't do incredibly well.
vbezhenar 7y ago
How are you supposed to call Objective C code from C then?
- 0x0 7y ago
  Call it from an extern C function in an .m file?
  - vbezhenar 7y ago
    Not very practical, if you need to use a lot of APIs. One of advantage of Objective C is that it's easy to interop with C.
    - pjmlp 7y ago
      You are seeing the other way around. That advantage is to migrate C code into Objective-C, just like C++ and TypeScript have done as well. In fact, NeXTSTEP and its derived OSes have very little pure C code, beyond BSD/XNU kernel and the POSIX APIs.
      - monocasa 7y ago
        You'd be surprised. The new APFS.Framework appears to be all C for instance.
        pjmlp 7y ago
        Even then you're supposed to actually use the higher level filesystem framework APIs from Objective-C and Swift. https://developer.apple.com/documentation/foundation/file_system/about_apple_file_system https://developer.apple.com/documentation/foundation/file_sy...
        monocasa 7y ago
        I mean, yeah, you're not "supposed to" use it; it's a private framework. Also, the only reason why someone like me would know if it's in C or not is that I need one of the aspects of that library that can't be expressed in the high level APIs. My point is that there's tons of non legacy C code, in the OS outside of the kernel and posix subsystems on *OS.
mpweiher 7y ago
You can still call objc_msgSend() from C. Just need to cast it correctly.

saagarjha 7y ago

I wonder if the new signature has any effect on pointer authentication (or was related to those changes). Also, I think the title would be better served with the correct capitalization of the method name, objc_msgSend.

andr 7y ago

Are there any conditions where the compiler can optimize away the call to objc_msgSend? Or is it always used for any call between 2 ObjC/Swift methods?

vidarh 7y ago
I don't know about Objective-C specifically, but generally one way to achieve this in most dynamic languages is a speed/memory tradeoff. In my Ruby compiler project which has to deal with the same level of dynamic behaviour, I handle dynamic overriding of methods with C++-style vtables, which turns method calls into the equivalent of this C-ish pseudo-code: (* ob->vtable[some_method_offset])(args...) Since Ruby classes can have a method_missing handling undefined methods, for any class that doesn't implement a given method, the vtable contains a pointer to a thunk that tweaks the stack to push the relevant method symbol as the first argument and then does the same as above with the method offset of method_missing. Since Ruby classes can have methods overridden at any time, if class Bar inherits from class Foo, inherits from class Object, and I override a method in class Object, that again has previously been explicitly overridden in class Bar, this would happen: - Store pointer to method in Object in ptr. - Replace method pointer in Object's vtable. - Iterate over all direct sub-classes of Object (but here we only care about Foo) - Compare the same method offset against ptr. Since Foo has not overridden the method, it matches. - Replace the pointer in Foo, and iterate over all direct sub-classes of Foo (but here we only care about Bar) - Compare the method offset against ptr. Since Bar has overridden the method, it doesn't match, so leave it alone. This means that as long as method overrides doesn't happen extremely frequently, the cost of method overrides is relatively low: iterate over all the descendant classes of the class you override a method in. Method calls on the other hand are about as cheap as virtual method calls in C++, except when you hit method_missing where this approach gives you a very low extra overhead of tweaking the stack to add the symbol and jumping to the method_missing implementation. This overall approach works for most dynamic languages. The caveat is memory - if you have an application with very large class hierarchies in a language where they are all singly rooted (as in Ruby where they all ultimately inherit from SimpleObject), each vtable will cost you at last pointer_size*global_number_of_method_names. In practice so far I've not seen all that many cases where this is a problem, and it's always possible for the compiler to set a roof above which it will resort to a slow send mechanism (because you'll need to support that anyway in any language that allows dynamic send mechanics; e.g. in Ruby you can always send a message to an object by a dynamically obtained symbol so you still need the equivalent of objc_msgSend as well). A slightly cheaper approach in terms of memory was described by Michael Franz[1]. His approach was to group methods in interfaces, so instead of a vtable of method pointers, you had a vtable of pointers to interfaces with pointers to methods. You save memory as most classes would typically implement most or none of the methods of an interface; it provides potential namespacing of the methods if you want to do that, and you can cut memory further by re-using the same vtable for an interface until someone tries to override at least one method in it. The cost is one extra indirection at call-sites. [1] Protocol Extension: A Technique for Structuring Large Extensible SoftwareSystems (1994) https://pdfs.semanticscholar.org/60d3/045d924dfb912e184014aa2c004214fb8236.pdf https://pdfs.semanticscholar.org/60d3/045d924dfb912e184014aa...

dahart 7y ago

> As long as you cast it to the correct type, it always works. The new way of doing things thus encourages doing things correctly and makes it harder to do things wrong. This seems like a really interesting case where removing type information, removing something was presumably thought to be a safety guard, and requiring type casting, makes the type safety stronger by forcing the user to do it explicitly. Maybe it makes sense that dynamic calling mechanisms should require casting (and static type checking is less safe) where static calling mechanisms should use static type checking (and type casting is less safe). > C specifies that certain types get promoted to wider types when passed as a variadic argument. Integers smaller than int (such as char and short) get promoted to int, and float gets promoted to double. If your method signature includes one of these types, it's not possible for a caller to pass a parameter as that exact type if it's using a variadic prototype. Holy crap! I’ve used these things for many years without knowing this important tidbit. Does anyone know if it’s the same in C++, or whether GPUs are currently ignoring or following the spec?

emilfihlman 7y ago
I mean, you have seen how printf uses %d for char, short and int as a formatter?
- dahart 7y ago
  Yeah, of course, but that doesn’t by itself lead me to the conclusion that those are all first converted to another type before being passed on the stack. They could be passed as-is and type-cast inside the printf implementation. The more interesting printf example would be %f, since according to what I learned today, it’s not a float, it’s a double. Are you saying you already knew doubles were passed in variadics instead of float, and the reason you knew is printf?
  - microtherion 7y ago
    > They could be passed as-is and type-cast inside the printf implementation. How would you know what to type-cast FROM? It's not like C compilers pass type information around, nor are C values self describing. > Are you saying you already knew doubles were passed in variadics instead of float, and the reason you knew is printf? I can't speak for the person you're answering to, but in pre-ANSI C, these promotions were ubiquitous (since there were no function prototypes yet), so programmers were very likely to have heard about them at some point.

tpush 7y ago

As an aside, am I missing something or is Apple's doc on objc_msgSend describing the parameters of the old form? The parameters section doesn't seem to match the declaration section.