4 ms·

> I am talking about a context switch to kernel code simply to access the shared memory. All interactions with SM require syscalls. Please enlighten me with th

> I am talking about a context switch to kernel code simply to access the shared memory. All interactions with SM require syscalls. Please enlighten me with the syscall you use to access a shared memory... because there are none. As far as the kernel is concerned, "memory" is just a set of pages mapped to a process at a specific address. These pages can be anonymous or named (meaning other processes can map them through that name). There is no syscall, no context switch, involved to read or write memory, named or anonymous. Hell, that's even the whole purpose of memory mapped IO. As mentioned earlier, there is also no real difference between a thread and a process as far as the kernel is concerned. A thread is just a special case of process which maps its heap to its parent. Even the word "thread" doesn't really exist for the Linux kernel. We just call these "lightweight processes", because these are just processes with a shared heap.

usrbinbash 3y ago
> Please enlighten me with the syscall you use to access a shared memory... because there are none. Oh rly? https://man7.org/linux/man-pages/man2/shmat.2.html https://man7.org/linux/man-pages/man2/shmat.2.html https://man7.org/linux/man-pages/man2/shmctl.2.html https://man7.org/linux/man-pages/man2/shmctl.2.html https://man7.org/linux/man-pages/man2/shmdt.2.html https://man7.org/linux/man-pages/man2/shmdt.2.html https://man7.org/linux/man-pages/man2/shmget.2.html https://man7.org/linux/man-pages/man2/shmget.2.html Pretty sure it says something about "System Calls Manual" at the top of all these man pages ;-) And IO on the SHM once it's attached may look "free", but it isn't. The M-mapping incurs a further overhead which simply doesn't exist for threads: A threads heap space is the same address space as those of it's siblings in the same process.
- Galanwe 3y ago
  No but seriously, are you actually trying to understand and learn, or just stuck trying to google your way out of your mistakes? 1) You linked the man pages of sys5 shared memories. Everyone switched to posix shared memories since literally 20 years. 2) Furthermore, these man pages dont even support your argument. Did you actually read them? These are just for the mapping, not for actually accessing (read/write) the shared memory. I would suggest, for your own future development, that you care less about trying to look knowledgeable on internet, and more about actually being.
  - usrbinbash 3y ago
    > You linked the man pages of sys5 shared memories. Everyone switched to posix shared memories since literally 20 years. Which is irrelevant, because there is barely any functional difference between the two. POSIX SHM uses a better API, providing a file-descriptor like object. That's all. And yes, using that API also requires syscalls. > Furthermore, these man pages dont even support your argument. Wrong, they absolutely do. "Accessing" something doesn't just involve the IO, it also involves the setup. And this requires syscalls, in SysV as it does in POSIX. > Did you actually read them? These are just for the mapping, not for actually accessing (read/write) the shared memory I am well aware of that, hence my seperate mentioning of IO later in my post. ;-) And the argument in that post stands as solid as it was before. `mmap` MAPS memory. This mapping requires an address translation overhead EVERYTIME THE MEMORY IS ACCESSED. This translation doesn't happen in userspace. Now, there is a way around that, in principle: If I mmap with ANONYMOUS and SHARED, and then `fork()`, I could have the same mapping in the child process. The problem here is: This relies on the forking after the mmap(). Any newly created objects, if I manage to map them into the child process, will again require address mapping. Again, this isn't an issue in threads. As soon as I create a new object on the heap, it is available to every thread, under the same address. But hey, what do I know. But I think the people who are going to dedicate countless hours of their lives making actual parallel processing via multithreading possible in Python know why they are doing so. As do the people who implemented abstractions around threading in basically every major programming language ;-)
    - Galanwe 3y ago
      > Which is irrelevant, because there is barely any functional difference between the two Well it is relevant, it tells that you're not familiar with the subject and most likely googled what looked related and found outdated man pages which seemed related. > Wrong, they absolutely do. "Accessing" something doesn't just involve the IO, it also involves the setup I think common sense would disagree, but if that's your line of arguing, sure. > And the argument in that post stands as solid as it was before. `mmap` MAPS memory. This mapping requires an address translation overhead EVERYTIME THE MEMORY IS ACCESSED. This translation doesn't happen in userspace. How deep are you willing to dig your hole there? Two messages ago you were arguing that accessing a shared memory required syscall and thus was slow. Now you've accepted it doesn't require syscall but argue it requires some magical dynamic translation by the kernel? I'm sorry but again that is just wrong. All the kernel does is inject the mappings when setting up TLB/MMU at context switch. The virtual address translation happens without kernel intervention, through the MMU, in _exactly_ the same manner whether this is a named or anonymous memory mapping, whether it is a shared or heap page. In fact the MMU couldn't care less what these addresses are, it has no concept of the meaning of what it translates.

usrbinbash 3y ago

> Please enlighten me with the syscall you use to access a shared memory... because there are none. Oh rly? https://man7.org/linux/man-pages/man2/shmat.2.html https://man7.org/linux/man-pages/man2/shmat.2.html https://man7.org/linux/man-pages/man2/shmctl.2.html https://man7.org/linux/man-pages/man2/shmctl.2.html https://man7.org/linux/man-pages/man2/shmdt.2.html https://man7.org/linux/man-pages/man2/shmdt.2.html https://man7.org/linux/man-pages/man2/shmget.2.html https://man7.org/linux/man-pages/man2/shmget.2.html Pretty sure it says something about "System Calls Manual" at the top of all these man pages ;-) And IO on the SHM once it's attached may look "free", but it isn't. The M-mapping incurs a further overhead which simply doesn't exist for threads: A threads heap space is the same address space as those of it's siblings in the same process.

Galanwe 3y ago
No but seriously, are you actually trying to understand and learn, or just stuck trying to google your way out of your mistakes? 1) You linked the man pages of sys5 shared memories. Everyone switched to posix shared memories since literally 20 years. 2) Furthermore, these man pages dont even support your argument. Did you actually read them? These are just for the mapping, not for actually accessing (read/write) the shared memory. I would suggest, for your own future development, that you care less about trying to look knowledgeable on internet, and more about actually being.
- usrbinbash 3y ago
  > You linked the man pages of sys5 shared memories. Everyone switched to posix shared memories since literally 20 years. Which is irrelevant, because there is barely any functional difference between the two. POSIX SHM uses a better API, providing a file-descriptor like object. That's all. And yes, using that API also requires syscalls. > Furthermore, these man pages dont even support your argument. Wrong, they absolutely do. "Accessing" something doesn't just involve the IO, it also involves the setup. And this requires syscalls, in SysV as it does in POSIX. > Did you actually read them? These are just for the mapping, not for actually accessing (read/write) the shared memory I am well aware of that, hence my seperate mentioning of IO later in my post. ;-) And the argument in that post stands as solid as it was before. `mmap` MAPS memory. This mapping requires an address translation overhead EVERYTIME THE MEMORY IS ACCESSED. This translation doesn't happen in userspace. Now, there is a way around that, in principle: If I mmap with ANONYMOUS and SHARED, and then `fork()`, I could have the same mapping in the child process. The problem here is: This relies on the forking after the mmap(). Any newly created objects, if I manage to map them into the child process, will again require address mapping. Again, this isn't an issue in threads. As soon as I create a new object on the heap, it is available to every thread, under the same address. But hey, what do I know. But I think the people who are going to dedicate countless hours of their lives making actual parallel processing via multithreading possible in Python know why they are doing so. As do the people who implemented abstractions around threading in basically every major programming language ;-)
  - Galanwe 3y ago
    > Which is irrelevant, because there is barely any functional difference between the two Well it is relevant, it tells that you're not familiar with the subject and most likely googled what looked related and found outdated man pages which seemed related. > Wrong, they absolutely do. "Accessing" something doesn't just involve the IO, it also involves the setup I think common sense would disagree, but if that's your line of arguing, sure. > And the argument in that post stands as solid as it was before. `mmap` MAPS memory. This mapping requires an address translation overhead EVERYTIME THE MEMORY IS ACCESSED. This translation doesn't happen in userspace. How deep are you willing to dig your hole there? Two messages ago you were arguing that accessing a shared memory required syscall and thus was slow. Now you've accepted it doesn't require syscall but argue it requires some magical dynamic translation by the kernel? I'm sorry but again that is just wrong. All the kernel does is inject the mappings when setting up TLB/MMU at context switch. The virtual address translation happens without kernel intervention, through the MMU, in _exactly_ the same manner whether this is a named or anonymous memory mapping, whether it is a shared or heap page. In fact the MMU couldn't care less what these addresses are, it has no concept of the meaning of what it translates.