4 ms·

> Without shared libraries, the only way to do plugins is via IPC. That's primitive superstition: - dlopen() works in statically linked programs too. - Even

by 08-15 6y ago

> Without shared libraries, the only way to do plugins is via IPC. That's primitive superstition: - dlopen() works in statically linked programs too. - Even without dlopen(), you can load code dynamically. It's not magic, especially if the plugin is linked statically. - Besides, how is IPC bad? I take fcgi over Apache's modules any day.

pjmlp 6y ago
dlopen is useless in static linked programs when the idea is to provide plugins after the fact. The idea of using plugins is exactly that various parts are able to ship them at various times during the lifetime of the applications. Patching files compiled statically is obviously not what one wants from plugins. Incidently plugins support is exactly why Go added support for dynamic linking into their toolchain.
- 08-15 6y ago
  One of us is confused; I honestly don't know which one. Where is the problem in this scenario? My application (statically) linked provides a plugin interface. It works by calling dlopen() on the plugin and then passes a record of functions to the entry points. The plugin can be provided by a third party, and top make that easier, I provide an SDK (a header file). No file is patched, and I have no idea what this has to do with Go. Enlighten me?
  - pjmlp 6y ago
    Your solution requires compiling everything from scratch in a single executable, that is not how plugins are supposed to be designed. We have been there before with solutions like graphics drivers for Borland's BGI library for their MS-DOS compilers. Where a driver interface is provided, and then each "plugin" registers themselves on application startup. Adding new plugins to an existing application in this scenario requires recompilation and updating the uses/#include being used for plugins, or patch the executable from an existing .obj file. If you are using dlopen on a third party binary instead of your own, then you are already using dynamic linking by definition, by loading third party code dynamically and revolving the proper address locations for all symbols. Go only had static compilation on the beginning as the only true way, but it failed short exactly in this scenario, to the point that eventually plugin package came to be and dynamic compilation support is now a feature of Go's toolchain as well, although many seem to still not be aware that Go compilers can also produced dynamic libraries.
    - tannhaeuser 6y ago
      That may all be true, but I guess GP was asking what that has to do with statically linking your main binary (or not), which of course you can do, yet still use shared libs as plugin loading mechanism. For example, ODBC drivers (with or without the various unixODBC/iODBC proxies) can be dlopen()ed from otherwise statically compiled main binaries just fine, can't they (even though for ODBC the typical use case is using a driver manager rather than invoking dlopen() directly)?
      - pjmlp 6y ago
        Sure, but that raises the question why bother with static linking if dlopen if being used anyway. With dlopen the application can crash in very interesting ways, if it doesn't handle loading the errors in a proper way. And there is no way to use something like dumpbin or ldd to actually find out what it is looking for, even something like strings might not help, because the path to the file being dynamically loaded in an explicit way might itself be dynamically constructed at runtime.
    - 08-15 6y ago
      > then you are already using dynamic linking by definition "Well, technically..." I proposed using the dlopen() mechanism (or a custom linker) from a program that is itself statically linked. Because that's what the argument is about: statically linking the stuff you always need. Now, technically, that's dynamic linking. And because I'm linking the plugins dynamically, I might as well link everything dynamically, right? No, absolutely not. Because the latter gives me DLL hell, and rpaths, and library maintainers who think that changing LD_LIBRARY_PATH is perfectly sensible. The former doesn't. Not equivalent. Not at all. We're not talking about BGI or overlays or Go. We're talking about statically linked binaries calling the dynamic linker. You didn't actually know that was possible, did you?
      - pjmlp 6y ago
        Actually that was just how UNIX dynamic linking started out, by using models that patched a.out, and I still have the Slackware CD with the first Linux kernel that was capable of loading proper ELF files instead. So my knowledge how these things work goes quite back in time.

by 08-15 6y ago

pjmlp 6y ago

dlopen is useless in static linked programs when the idea is to provide plugins after the fact. The idea of using plugins is exactly that various parts are able to ship them at various times during the lifetime of the applications. Patching files compiled statically is obviously not what one wants from plugins. Incidently plugins support is exactly why Go added support for dynamic linking into their toolchain.

08-15 6y ago
One of us is confused; I honestly don't know which one. Where is the problem in this scenario? My application (statically) linked provides a plugin interface. It works by calling dlopen() on the plugin and then passes a record of functions to the entry points. The plugin can be provided by a third party, and top make that easier, I provide an SDK (a header file). No file is patched, and I have no idea what this has to do with Go. Enlighten me?
- pjmlp 6y ago
  Your solution requires compiling everything from scratch in a single executable, that is not how plugins are supposed to be designed. We have been there before with solutions like graphics drivers for Borland's BGI library for their MS-DOS compilers. Where a driver interface is provided, and then each "plugin" registers themselves on application startup. Adding new plugins to an existing application in this scenario requires recompilation and updating the uses/#include being used for plugins, or patch the executable from an existing .obj file. If you are using dlopen on a third party binary instead of your own, then you are already using dynamic linking by definition, by loading third party code dynamically and revolving the proper address locations for all symbols. Go only had static compilation on the beginning as the only true way, but it failed short exactly in this scenario, to the point that eventually plugin package came to be and dynamic compilation support is now a feature of Go's toolchain as well, although many seem to still not be aware that Go compilers can also produced dynamic libraries.
  - tannhaeuser 6y ago
    That may all be true, but I guess GP was asking what that has to do with statically linking your main binary (or not), which of course you can do, yet still use shared libs as plugin loading mechanism. For example, ODBC drivers (with or without the various unixODBC/iODBC proxies) can be dlopen()ed from otherwise statically compiled main binaries just fine, can't they (even though for ODBC the typical use case is using a driver manager rather than invoking dlopen() directly)?
    - pjmlp 6y ago
      Sure, but that raises the question why bother with static linking if dlopen if being used anyway. With dlopen the application can crash in very interesting ways, if it doesn't handle loading the errors in a proper way. And there is no way to use something like dumpbin or ldd to actually find out what it is looking for, even something like strings might not help, because the path to the file being dynamically loaded in an explicit way might itself be dynamically constructed at runtime.
  - 08-15 6y ago
    > then you are already using dynamic linking by definition "Well, technically..." I proposed using the dlopen() mechanism (or a custom linker) from a program that is itself statically linked. Because that's what the argument is about: statically linking the stuff you always need. Now, technically, that's dynamic linking. And because I'm linking the plugins dynamically, I might as well link everything dynamically, right? No, absolutely not. Because the latter gives me DLL hell, and rpaths, and library maintainers who think that changing LD_LIBRARY_PATH is perfectly sensible. The former doesn't. Not equivalent. Not at all. We're not talking about BGI or overlays or Go. We're talking about statically linked binaries calling the dynamic linker. You didn't actually know that was possible, did you?
    - pjmlp 6y ago
      Actually that was just how UNIX dynamic linking started out, by using models that patched a.out, and I still have the Slackware CD with the first Linux kernel that was capable of loading proper ELF files instead. So my knowledge how these things work goes quite back in time.