3 ms·

One question: is it possible to load .so files (through dlopen/dlsym) on Linux when compiling to APE? I was working on something similar (though much smaller i

One question: is it possible to load .so files (through dlopen/dlsym) on Linux when compiling to APE? I was working on something similar (though much smaller in scope), but had to stop when I realised that `ld-linux.so` has some internal APIs that Glibc uses to setup dlopen()/dlsym(); essentially meaning that it is very hard -if not impossible - to load any shared libraries if one does not link to Glibc. What I wouldn't give for a liblinux, similar to NTDLL.

surajrmal 6y ago
It doesn't appear to be supported in their libc implementation: https://github.com/jart/cosmopolitan/blob/d7ac16a9ed56ebdc70481f4a802265ea15619894/libc/runtime/ldso.c https://github.com/jart/cosmopolitan/blob/d7ac16a9ed56ebdc70...
- surajrmal 6y ago
  Looks like you're also giving up position independent code as that also relies on the dynamic linker.
- HexDecOctBin 6y ago
  Yeah, as I feared. Some one needs to wrest the control of ld-linux.so from Glibc's hands and give it to the Kernel developers (who "will not break userspace"). Otherwise, Linux will always be a all-work-for-no-reward platform.
matheusmoreira 6y ago
> What I wouldn't give for a liblinux, similar to NTDLL. Hey I actually tried to make such a thing. https://github.com/matheusmoreira/liblinux https://github.com/matheusmoreira/liblinux It provides access to Linux system calls and process start up code that gets all the arguments, environment and auxiliary values. I have several examples of applications written in 100% freestanding C with zero dependencies other than this library: https://github.com/matheusmoreira/liblinux/tree/master/examples https://github.com/matheusmoreira/liblinux/tree/master/examp... It's a bit too low level but I actually planned to make my own ld-liblinux eventually. It currently works with static and dynamic linking. The ld-linux.so is able to link even though nothing else links against glibc. I didn't test dynamic loading though. I stopped working on it because I found better solution for system calls on the Linux kernel repository: https://github.com/torvalds/linux/blob/master/tools/include/nolibc/nolibc.h https://github.com/torvalds/linux/blob/master/tools/include/... Do you think liblinux could have a future?
- HexDecOctBin 6y ago
  > I found better solution for system calls Yeah, I have seen that file. Thought last I checked, it crashes in certain conditions [1]; I didn't really go back to it to fix it. > I actually planned to make my own ld-liblinux eventually Can you share this code? If not, could you document how exactly would you have done this? I lost all motivation once I realized that ld-linux and Glibc communicate with private APIs, but I'd love to actually work on it if it was possible to do this. > Do you think liblinux could have a future? Absolutely. The problem with Linux currently is that the loader rejects programs at the sign of slightest incompatibility before even trying to execute them, which means that there is nothing that the application code could do to work around said incompatibility. If we were able to get our code running (or, as I like to say it, make %RIP point inside the main function), we could then do whatever hacky-bullshit was required make the software work. All I need is to get our code executing, and from that point on, I'll handle system compatibility. [1]: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95411 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95411
  - matheusmoreira 6y ago
    Not sure why the stack is unaligned there. The way I solved this in liblinux is to just pass the stack pointer to a C function and compute all the other pointers there. That way the compiler deals with the stack stuff. _start: xorq %rbp,%rbp /* user code should mark the deepest stack frame * by setting the frame pointer to zero */ movq %rsp,%rdi call liblinux_start movq %rax,%rdi movq $__NR_exit,%rax syscall static void *after(void *vector) { void **pointer = (void **) vector; while (*pointer++ != 0); return pointer; } int liblinux_start(void *stack_pointer) { long count; char *arguments; char *environment; struct auxiliary *values; count = *((long *) stack_pointer); arguments = ((char **) stack_pointer) + 1; environment = arguments + count + 1; values = after(environment); /* start is the main function */ return start(count, arguments, environment, values); } > Can you share this code? If not, could you document how exactly would you have done this? Unfortunately I didn't make it that far. I was planning to study how glibc and musl do it just like how I studied their system call implementations. If I start working on this again I'm gonna look this up. I'll need to learn this linking stuff in order to support the kernel vDSO anyway.

surajrmal 6y ago

It doesn't appear to be supported in their libc implementation: https://github.com/jart/cosmopolitan/blob/d7ac16a9ed56ebdc70481f4a802265ea15619894/libc/runtime/ldso.c https://github.com/jart/cosmopolitan/blob/d7ac16a9ed56ebdc70...

surajrmal 6y ago
Looks like you're also giving up position independent code as that also relies on the dynamic linker.
HexDecOctBin 6y ago
Yeah, as I feared. Some one needs to wrest the control of ld-linux.so from Glibc's hands and give it to the Kernel developers (who "will not break userspace"). Otherwise, Linux will always be a all-work-for-no-reward platform.

matheusmoreira 6y ago

> What I wouldn't give for a liblinux, similar to NTDLL. Hey I actually tried to make such a thing. https://github.com/matheusmoreira/liblinux https://github.com/matheusmoreira/liblinux It provides access to Linux system calls and process start up code that gets all the arguments, environment and auxiliary values. I have several examples of applications written in 100% freestanding C with zero dependencies other than this library: https://github.com/matheusmoreira/liblinux/tree/master/examples https://github.com/matheusmoreira/liblinux/tree/master/examp... It's a bit too low level but I actually planned to make my own ld-liblinux eventually. It currently works with static and dynamic linking. The ld-linux.so is able to link even though nothing else links against glibc. I didn't test dynamic loading though. I stopped working on it because I found better solution for system calls on the Linux kernel repository: https://github.com/torvalds/linux/blob/master/tools/include/nolibc/nolibc.h https://github.com/torvalds/linux/blob/master/tools/include/... Do you think liblinux could have a future?

HexDecOctBin 6y ago
> I found better solution for system calls Yeah, I have seen that file. Thought last I checked, it crashes in certain conditions [1]; I didn't really go back to it to fix it. > I actually planned to make my own ld-liblinux eventually Can you share this code? If not, could you document how exactly would you have done this? I lost all motivation once I realized that ld-linux and Glibc communicate with private APIs, but I'd love to actually work on it if it was possible to do this. > Do you think liblinux could have a future? Absolutely. The problem with Linux currently is that the loader rejects programs at the sign of slightest incompatibility before even trying to execute them, which means that there is nothing that the application code could do to work around said incompatibility. If we were able to get our code running (or, as I like to say it, make %RIP point inside the main function), we could then do whatever hacky-bullshit was required make the software work. All I need is to get our code executing, and from that point on, I'll handle system compatibility. [1]: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95411 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95411
- matheusmoreira 6y ago
  Not sure why the stack is unaligned there. The way I solved this in liblinux is to just pass the stack pointer to a C function and compute all the other pointers there. That way the compiler deals with the stack stuff. _start: xorq %rbp,%rbp /* user code should mark the deepest stack frame * by setting the frame pointer to zero */ movq %rsp,%rdi call liblinux_start movq %rax,%rdi movq $__NR_exit,%rax syscall static void *after(void *vector) { void **pointer = (void **) vector; while (*pointer++ != 0); return pointer; } int liblinux_start(void *stack_pointer) { long count; char *arguments; char *environment; struct auxiliary *values; count = *((long *) stack_pointer); arguments = ((char **) stack_pointer) + 1; environment = arguments + count + 1; values = after(environment); /* start is the main function */ return start(count, arguments, environment, values); } > Can you share this code? If not, could you document how exactly would you have done this? Unfortunately I didn't make it that far. I was planning to study how glibc and musl do it just like how I studied their system call implementations. If I start working on this again I'm gonna look this up. I'll need to learn this linking stuff in order to support the kernel vDSO anyway.