3 ms·

"Where is the need for recompilation?" https://docs.python.org/3/c-api/typeobj.html#c.PyObject.ob_refcnt https://docs.python.org/3/c-api/typeobj.html#c.PyObjec

"Where is the need for recompilation?" https://docs.python.org/3/c-api/typeobj.html#c.PyObject.ob_refcnt https://docs.python.org/3/c-api/typeobj.html#c.PyObject.ob_r... The stable ABI lets extensions access the reference count of any object directly. I don't know why. Normally the functions Py_IncRef and Py_DecRef should be sufficient. Objects no longer have a single number as their reference count. Edit: In Python 3.2-3.9, the stable ABI included Py_INCREF, the C macro. https://docs.python.org/3/c-api/type.html#c.PyType_FromModuleAndSpec https://docs.python.org/3/c-api/type.html#c.PyType_FromModul... The stable ABI lets extensions create new Python types. These can override tp_alloc field to use custom memory allocators when instances of the type are instantiated. The custom memory allocator needs to initialise the reference count to 1. > But you're just guessing that it's slower in GIL mode, aren't you? There are new implementations with per-object locks of list.append, dict.__setitem__ etc. These are incredibly common operations in Python code. These inherently will be more complicated than the previous implementation, meaning more instructions and slower. So there needs to be a branch at the start of list.append of whether to go to the old gil implementation or the new nogil one. Adding a branch so frequently will inherently make the runtime slower. Now with a lot of work these can be optimised and the speed penalty reduced, but CPython goes on an annual release cycle. If the nogil code is held as a fork of the CPython repository without being merged in, until it reaches a performance goal, that brings other issues.

Dylan16807 3y ago
> Objects no longer have a single number as their reference count. Does that stay true once the GIL turns back on? > These can override tp_alloc field to use custom memory allocators when instances of the type are instantiated. The custom memory allocator needs to initialise the reference count to 1. I'm not following why this affects ABI compatibility, sorry. > So there needs to be a branch at the start of list.append of whether to go to the old gil implementation or the new nogil one. Adding a branch so frequently will inherently make the runtime slower. I don't know about that. CPUs handle always-taken and never-taken branches very well.
- gpderetta 3y ago
  > I don't know about that. CPUs handle always-taken and never-taken branches very well. and dynamic patching is also an option.
- lozenge 3y ago
  > Does that stay true once the GIL turns back on? In the current nogil implementation, AFAICS, it seems the GIL can't be turned back on so there is no answer yet. Theoretically, you could have a one-off operation which fixes all objects when the GIL is turned on. However, there's no way to get all objects in Python. gc.get_objects() only returns tracked objects, and there is no way to list untracked objects. It seems, three fields are exposed on every CPython object in the stable ABI, any change which affects their offsets will break the stable ABI. https://github.com/capi-workgroup/problems/issues/4#issuecomment-1540917809 https://github.com/capi-workgroup/problems/issues/4#issuecom... > I'm not following why this affects ABI compatibility, sorry. True, PyType_FromSpec can set tp_alloc to a wrapper function which papers over the difference in what the "allocfunc" should initialise the memory to. Re performance - merging the change is the only way people will actually start targeting nogil.

Dylan16807 3y ago

> Objects no longer have a single number as their reference count. Does that stay true once the GIL turns back on? > These can override tp_alloc field to use custom memory allocators when instances of the type are instantiated. The custom memory allocator needs to initialise the reference count to 1. I'm not following why this affects ABI compatibility, sorry. > So there needs to be a branch at the start of list.append of whether to go to the old gil implementation or the new nogil one. Adding a branch so frequently will inherently make the runtime slower. I don't know about that. CPUs handle always-taken and never-taken branches very well.

gpderetta 3y ago
> I don't know about that. CPUs handle always-taken and never-taken branches very well. and dynamic patching is also an option.
lozenge 3y ago
> Does that stay true once the GIL turns back on? In the current nogil implementation, AFAICS, it seems the GIL can't be turned back on so there is no answer yet. Theoretically, you could have a one-off operation which fixes all objects when the GIL is turned on. However, there's no way to get all objects in Python. gc.get_objects() only returns tracked objects, and there is no way to list untracked objects. It seems, three fields are exposed on every CPython object in the stable ABI, any change which affects their offsets will break the stable ABI. https://github.com/capi-workgroup/problems/issues/4#issuecomment-1540917809 https://github.com/capi-workgroup/problems/issues/4#issuecom... > I'm not following why this affects ABI compatibility, sorry. True, PyType_FromSpec can set tp_alloc to a wrapper function which papers over the difference in what the "allocfunc" should initialise the memory to. Re performance - merging the change is the only way people will actually start targeting nogil.