5 ms·

This is similar to the work done in winapi [1], com-imp[2] and my own tangential work on porting VST3 [3] to Rust [4] I'm really glad MS is doing this. What ne

This is similar to the work done in winapi [1], com-imp[2] and my own tangential work on porting VST3 [3] to Rust [4] I'm really glad MS is doing this. What needs to be a bit clearer to me is how they maintain ABI compatibility under the hood of MSVC for COM interfaces (which uses the vtable layout of an inherited class) and how that's compatible with MinGW/GCC stacks on Windows, mostly what can break it. I got stuck porting VST3 with multiple inheritance, and it was a headache trying to reverse engineer the appropriate struct layouts for COM implementations. [1] https://github.com/retep998/winapi-rs/blob/0.3/src/macros.rs https://github.com/retep998/winapi-rs/blob/0.3/src/macros.rs [2] https://github.com/Connicpu/com-impl https://github.com/Connicpu/com-impl [3] https://github.com/steinbergmedia/vst3sdk https://github.com/steinbergmedia/vst3sdk [4] https://github.com/m-hilgendorf/cli-host https://github.com/m-hilgendorf/cli-host (sorry for the messy code, it was a weekend of hacking away trying to host a VST3 in pure Rust)

pjmlp 7y ago
COM layout is followed upon most mainstream Windows compiled languages, namely major C++ compilers, .NET, Delphi, Eiffel, Ada, so it is not MSVC++ keeping ABI compatibility under the hood on their own.
- holy_city 7y ago
  My issue isn't ABI stability but the ABI itself w.r.t vtable layout. Best I can tell it should be similar to Itanium's spec? [1]. It's been months since I did this, but iirc my problems stemmed from having multiple interfaces on top of the same implementation and the ordering/layout of those interfaces, though the IUnknown interface which is supposed to handle that. [1] https://itanium-cxx-abi.github.io/cxx-abi/abi.html#vtable https://itanium-cxx-abi.github.io/cxx-abi/abi.html#vtable
  - barrkel 7y ago
    COM is agnostic as to how you do multiple inheritance - it doesn't have the concept. It specifies the QueryInterface protocol, but you don't need to return the same instance for the result of the QI call, just one that uses the same lifetime refcount. Tear-off interfaces and delegated implementations are things in this world.
    - ptx 7y ago
      Meaning that the AddRef/Release counter has to be shared between that group of objects?
      - pjc50 7y ago
        Same object, different interfaces.
        magicalhippo 7y ago
        You can implement the other interfaces using other objects, but they should delegate the ref counting to the root/main object.
  - Const-me 7y ago
    If you can read C#, maybe this will help: https://github.com/Const-me/ComLightInterop https://github.com/Const-me/ComLightInterop Specifically, this class implementing vtable callable from C++: https://github.com/Const-me/ComLightInterop/blob/master/ComLight/ManagedObject.cs https://github.com/Const-me/ComLightInterop/blob/master/ComL... About multiple interfaces, all of them need 3 first vtable entries pointing to the 3 IUnknown methods. Also, don't forget that when client calls QueryInterface on any interface of the same object with IID_IUnknown argument, you must return same IUnknown pointer. Some parts of COM use that pointer as object's identity.
  - jcranmer 7y ago
    MSVC uses a completely different ABI from Itanium, and you shouldn't rely on the Itanium ABI to inform you what it might look like. vtable layout in the most basic situations is going to be accidentally portable because those situations boil down to "it's a struct of function pointers," and there's only so many ways you can order the fields of a struct. But even here, MSVC uses a quite different ABI: the order of the vtable entries can change if you overload a virtual method with a non-virtual method.
    - spacechild1 7y ago
      AFAIK, non-virtual methods never affect the vtable layout. But when you overload a virtual method with another virtual method, the ordering in the vtable is unspecified! Also, a public COM interface mustn't have a virtual destructor, because some compilers (e.g. recent GCC) put more than one method in the vtable. Implementation classes might define a virtual destructor, though.
barrkel 7y ago
COM is an ABI standard. The structs are defined in terms of C with Winapi (stdcall) calling convention. Very little needs reverse engineering - it's all pIntf->vTable->func(pIntf, ...). You can explain it on a whiteboard in a couple of minutes. The way MSVC does it may need reverse engineering (it may be patented btw). I could explain how Delphi implements COM interfaces, but any specific implementation is actually more complicated than the ABI, because they're trying to add implementation ergonomics on top of the basic calling convention.
- holy_city 7y ago
  Is the ABI documented anywhere? Every time I google around for it, I just get information like "COM is ABI stable and language agnostic" but not what the ABI is. I've successfully implemented implementations of single COM interfaces and get the basics, my trouble was in implementing many interfaces for the same implementation and running into copious segfaults when testing the Rust implementation through a reference app written in C++.
  - pjmlp 7y ago
    Here are some good COM books. Essential COM by Don Box https://www.amazon.com/Essential-COM-Don-Box/dp/0201634465 https://www.amazon.com/Essential-COM-Don-Box/dp/0201634465 Inside COM by Dale Rogerson https://www.amazon.com/Inside-Microsoft-Programming-Dale-Rogerson/dp/1572313498 https://www.amazon.com/Inside-Microsoft-Programming-Dale-Rog... Then if you have access to public libraries, maybe one of them has one of the several Microsoft Systems Journals issues, later MSDN Magazine, with plenty of low level COM articles. COM is from the days where good documentation was to be found in books, not on the Interwebs.
    - holy_city 7y ago
      thank you!
      - vic20forever 7y ago
        archive.org has the following pre-dotnet edition of MSDN Library - you'll find a lot of COM info in it: https://archive.org/details/MSDN_Library_October_2001_Disc_1 https://archive.org/details/MSDN_Library_October_2001_Disc_1 https://archive.org/details/MSDN_Library_October_2001_Disc_2 https://archive.org/details/MSDN_Library_October_2001_Disc_2 https://archive.org/details/MSDN_Library_October_2001_Disc_3 https://archive.org/details/MSDN_Library_October_2001_Disc_3

pjmlp 7y ago

COM layout is followed upon most mainstream Windows compiled languages, namely major C++ compilers, .NET, Delphi, Eiffel, Ada, so it is not MSVC++ keeping ABI compatibility under the hood on their own.

holy_city 7y ago
My issue isn't ABI stability but the ABI itself w.r.t vtable layout. Best I can tell it should be similar to Itanium's spec? [1]. It's been months since I did this, but iirc my problems stemmed from having multiple interfaces on top of the same implementation and the ordering/layout of those interfaces, though the IUnknown interface which is supposed to handle that. [1] https://itanium-cxx-abi.github.io/cxx-abi/abi.html#vtable https://itanium-cxx-abi.github.io/cxx-abi/abi.html#vtable
- barrkel 7y ago
  COM is agnostic as to how you do multiple inheritance - it doesn't have the concept. It specifies the QueryInterface protocol, but you don't need to return the same instance for the result of the QI call, just one that uses the same lifetime refcount. Tear-off interfaces and delegated implementations are things in this world.
  - ptx 7y ago
    Meaning that the AddRef/Release counter has to be shared between that group of objects?
    - pjc50 7y ago
      Same object, different interfaces.
      - magicalhippo 7y ago
        You can implement the other interfaces using other objects, but they should delegate the ref counting to the root/main object.
- Const-me 7y ago
  If you can read C#, maybe this will help: https://github.com/Const-me/ComLightInterop https://github.com/Const-me/ComLightInterop Specifically, this class implementing vtable callable from C++: https://github.com/Const-me/ComLightInterop/blob/master/ComLight/ManagedObject.cs https://github.com/Const-me/ComLightInterop/blob/master/ComL... About multiple interfaces, all of them need 3 first vtable entries pointing to the 3 IUnknown methods. Also, don't forget that when client calls QueryInterface on any interface of the same object with IID_IUnknown argument, you must return same IUnknown pointer. Some parts of COM use that pointer as object's identity.
- jcranmer 7y ago
  MSVC uses a completely different ABI from Itanium, and you shouldn't rely on the Itanium ABI to inform you what it might look like. vtable layout in the most basic situations is going to be accidentally portable because those situations boil down to "it's a struct of function pointers," and there's only so many ways you can order the fields of a struct. But even here, MSVC uses a quite different ABI: the order of the vtable entries can change if you overload a virtual method with a non-virtual method.
  - spacechild1 7y ago
    AFAIK, non-virtual methods never affect the vtable layout. But when you overload a virtual method with another virtual method, the ordering in the vtable is unspecified! Also, a public COM interface mustn't have a virtual destructor, because some compilers (e.g. recent GCC) put more than one method in the vtable. Implementation classes might define a virtual destructor, though.

barrkel 7y ago

COM is an ABI standard. The structs are defined in terms of C with Winapi (stdcall) calling convention. Very little needs reverse engineering - it's all pIntf->vTable->func(pIntf, ...). You can explain it on a whiteboard in a couple of minutes. The way MSVC does it may need reverse engineering (it may be patented btw). I could explain how Delphi implements COM interfaces, but any specific implementation is actually more complicated than the ABI, because they're trying to add implementation ergonomics on top of the basic calling convention.

holy_city 7y ago
Is the ABI documented anywhere? Every time I google around for it, I just get information like "COM is ABI stable and language agnostic" but not what the ABI is. I've successfully implemented implementations of single COM interfaces and get the basics, my trouble was in implementing many interfaces for the same implementation and running into copious segfaults when testing the Rust implementation through a reference app written in C++.
- pjmlp 7y ago
  Here are some good COM books. Essential COM by Don Box https://www.amazon.com/Essential-COM-Don-Box/dp/0201634465 https://www.amazon.com/Essential-COM-Don-Box/dp/0201634465 Inside COM by Dale Rogerson https://www.amazon.com/Inside-Microsoft-Programming-Dale-Rogerson/dp/1572313498 https://www.amazon.com/Inside-Microsoft-Programming-Dale-Rog... Then if you have access to public libraries, maybe one of them has one of the several Microsoft Systems Journals issues, later MSDN Magazine, with plenty of low level COM articles. COM is from the days where good documentation was to be found in books, not on the Interwebs.
  - holy_city 7y ago
    thank you!
    - vic20forever 7y ago
      archive.org has the following pre-dotnet edition of MSDN Library - you'll find a lot of COM info in it: https://archive.org/details/MSDN_Library_October_2001_Disc_1 https://archive.org/details/MSDN_Library_October_2001_Disc_1 https://archive.org/details/MSDN_Library_October_2001_Disc_2 https://archive.org/details/MSDN_Library_October_2001_Disc_2 https://archive.org/details/MSDN_Library_October_2001_Disc_3 https://archive.org/details/MSDN_Library_October_2001_Disc_3