3 ms·

on this topic, it's interesting how my mindset changes when I'm writing Rust vs Python, with regards to vtables, dispatching, and allocation. Writing Rust for

on this topic, it's interesting how my mindset changes when I'm writing Rust vs Python, with regards to vtables, dispatching, and allocation. Writing Rust for a toy project: "I MUST avoid allocation, and the dyn keyword, at all cost!!" Writing the same toy project in Python: "lol just copy the string who cares"

kukkamario 3y ago
Perspective really changes when simple integer addition can take more time than allocation in a compiled language.
- ninkendo 3y ago
  Hmm, assuming you’re talking about heap allocation, could you elaborate? I’m curious. [Edit: You may have meant “when simple integer addition [in a scripting language] can take more time than allocation in a compiled language”, in which case sure, I agree. I had originally interpreted your statement as saying compiled languages can do allocation faster than they can do addition…] Allocation in most environments I’ve seen involves keeping track of a “free list”, which does some amount of pointer chasing to find a contiguous section of memory that can fulfill the request. Deallocation drops the item from the list, etc. This can result in some fragmentation, and the segments of memory managed by the free list are less likely to be in cache because they’re somewhat dispersed compared to stack memory. Some environments have multiple free lists depending on the expected size of the allocated objects, to avoid fragmentation. How could all this be faster than a single integer addition?
  - LegionMammal978 3y ago
    When it's a bigint addition, as all integer additions in Python are. Also, even when both integers are small, Python usually has to do an allocation anyway to store the result.
deleted 3y ago
[deleted]
gpderetta 3y ago
I complain about python slowness all the time, but in a way is liberating. As you have no chance to make the code faster, you just stop worrying about it.
- nickpsecurity 3y ago
  I've been learning Python recently. I found that one can make it much faster with a little work. Coming from other languages, it was also easy to write it in a way that its many accelerators, like Cython or mycpp, could use. What's the catch? For me, there were two. First, I'm having to learn to code in the "Pythonic" style so my code will fit into existing projects. An example in a class I took was moving from while loops to for i in range(len(list)). While you won't forget to increment the counter, it felt like being so clever would reduce optimizations. Another project needed whole, string copies for conciseness vs a char-by-char algorithm that was highly-efficient. While it may be my inexperience, it seems like Pythonic style inherently favors conciseness and readibility standards over performance. Second, the extensions seem tightly coupled with the CPython interpreter. I've read enough articles to know that's causing problems for people wanting to add capabilities. It may not matter, though, since most of us use Python for its ecosystem benefits. We can just switch tools when needed.
cleak 3y ago
I've actually shifted most of my personal dev to Rust now, and so this vtable hack has become less relevant. Rust makes it very easy to avoid virtual functions. If I had to redo this bit of code in Rust, a trait would boil any sort of update function down to its concrete type and give (I'd expect) great performance with all the convenience of virtual functions.

kukkamario 3y ago

Perspective really changes when simple integer addition can take more time than allocation in a compiled language.

ninkendo 3y ago
Hmm, assuming you’re talking about heap allocation, could you elaborate? I’m curious. [Edit: You may have meant “when simple integer addition [in a scripting language] can take more time than allocation in a compiled language”, in which case sure, I agree. I had originally interpreted your statement as saying compiled languages can do allocation faster than they can do addition…] Allocation in most environments I’ve seen involves keeping track of a “free list”, which does some amount of pointer chasing to find a contiguous section of memory that can fulfill the request. Deallocation drops the item from the list, etc. This can result in some fragmentation, and the segments of memory managed by the free list are less likely to be in cache because they’re somewhat dispersed compared to stack memory. Some environments have multiple free lists depending on the expected size of the allocated objects, to avoid fragmentation. How could all this be faster than a single integer addition?
- LegionMammal978 3y ago
  When it's a bigint addition, as all integer additions in Python are. Also, even when both integers are small, Python usually has to do an allocation anyway to store the result.

deleted 3y ago

[deleted]

gpderetta 3y ago

I complain about python slowness all the time, but in a way is liberating. As you have no chance to make the code faster, you just stop worrying about it.

nickpsecurity 3y ago
I've been learning Python recently. I found that one can make it much faster with a little work. Coming from other languages, it was also easy to write it in a way that its many accelerators, like Cython or mycpp, could use. What's the catch? For me, there were two. First, I'm having to learn to code in the "Pythonic" style so my code will fit into existing projects. An example in a class I took was moving from while loops to for i in range(len(list)). While you won't forget to increment the counter, it felt like being so clever would reduce optimizations. Another project needed whole, string copies for conciseness vs a char-by-char algorithm that was highly-efficient. While it may be my inexperience, it seems like Pythonic style inherently favors conciseness and readibility standards over performance. Second, the extensions seem tightly coupled with the CPython interpreter. I've read enough articles to know that's causing problems for people wanting to add capabilities. It may not matter, though, since most of us use Python for its ecosystem benefits. We can just switch tools when needed.

cleak 3y ago

I've actually shifted most of my personal dev to Rust now, and so this vtable hack has become less relevant. Rust makes it very easy to avoid virtual functions. If I had to redo this bit of code in Rust, a trait would boil any sort of update function down to its concrete type and give (I'd expect) great performance with all the convenience of virtual functions.