4 ms·

The Python/NumPy gotcha that has caused me the most pain by far is def foo(x, y): return x+y where the intention is for x and y to be NumPy arrays, b

by gspr 6y ago

The Python/NumPy gotcha that has caused me the most pain by far is def foo(x, y): return x+y where the intention is for x and y to be NumPy arrays, but the caller accidentally passes lists (or vice versa). This is especially common because a lot of NumPy functions are agnostic as to whether they operate on arrays or other iterables.

avian 6y ago
I've been bitten by this as well. In general, having two types that are mostly, but not always, interchangeable, often leads to hard to debug problems. A similar situation that comes to mind was str and unicode types in Python 2. Sometimes you end up with a stray value of a wrong type somewhere in a complicated data structure. It gets propagated down the line since most parts don't care if it's one type or the other. This makes it possible for the value to propagate far away from the original mistake that caused it to be of the wrong type in the first place. Finally when some code chokes up on it, it can be completely unrelated to the code that needs fixing. Since it's hard to debug the original cause this is often fixed by just adding the type conversion at the place that choked up. But this is just kicking the can down the street, since the remaining type inconsistency will soon pop up a problem somewhere else.
nabla9 6y ago
That's clear example of bad language design. Overloading operator syntax and breaking semantics in dynamic language is big no-no. It may not be good for statically typed languages either, but at least they can get away with it. Convention of using + for joining sequences and addition is human language level abstraction. It breaks in programming. In formal theory strings over an alphabet form a semiring, but (*) is used for concatenation and (+) for alternation. Not applicable anyway. Last, no clear syntactical cue to distinguish operators acting on elements from those acting on objects.
antpls 6y ago
I disagree. The intention in foo is not for x and y to be numpy arrays, it is to add 2 objects together. "If it walks like a duck and it quacks like a duck, then it must be a duck". That function can work on anything implementing +, this is the philosophy of Python. It enables the whole numpy, dask, TensorFlow ecosystem. If you intend to limit it to numpy arrays, you either use numpy 1.20's static types, or you add more checks at the beginning of the function, such as np.asarray(x).

by gspr 6y ago

avian 6y ago

I've been bitten by this as well. In general, having two types that are mostly, but not always, interchangeable, often leads to hard to debug problems. A similar situation that comes to mind was str and unicode types in Python 2. Sometimes you end up with a stray value of a wrong type somewhere in a complicated data structure. It gets propagated down the line since most parts don't care if it's one type or the other. This makes it possible for the value to propagate far away from the original mistake that caused it to be of the wrong type in the first place. Finally when some code chokes up on it, it can be completely unrelated to the code that needs fixing. Since it's hard to debug the original cause this is often fixed by just adding the type conversion at the place that choked up. But this is just kicking the can down the street, since the remaining type inconsistency will soon pop up a problem somewhere else.

nabla9 6y ago

That's clear example of bad language design. Overloading operator syntax and breaking semantics in dynamic language is big no-no. It may not be good for statically typed languages either, but at least they can get away with it. Convention of using + for joining sequences and addition is human language level abstraction. It breaks in programming. In formal theory strings over an alphabet form a semiring, but (*) is used for concatenation and (+) for alternation. Not applicable anyway. Last, no clear syntactical cue to distinguish operators acting on elements from those acting on objects.

antpls 6y ago

I disagree. The intention in foo is not for x and y to be numpy arrays, it is to add 2 objects together. "If it walks like a duck and it quacks like a duck, then it must be a duck". That function can work on anything implementing +, this is the philosophy of Python. It enables the whole numpy, dask, TensorFlow ecosystem. If you intend to limit it to numpy arrays, you either use numpy 1.20's static types, or you add more checks at the beginning of the function, such as np.asarray(x).