3 ms·

I've opened some issues over the years with Pandas on the transition to Arrow dtypes. They are generally very nice but even today I have many workaround patches

I've opened some issues over the years with Pandas on the transition to Arrow dtypes. They are generally very nice but even today I have many workaround patches in our codebase to avoid bugs (e.g. fillna and dropna working as expected). This PDEP doesn't discuss nullable types at all (i.e. what is currently Float32 instead of float32), which is worth mentioning (though support is pretty good now in Pandas). If any of the core developers reads this, can I suggest `float32?` instead of `Float32`? I think it's more obvious what is intended with the question mark than with capitalization.

nerdponx 2y ago
Discussion of nullability is definitely an important omission here. Part of the issue is that Pandas itself supports multiple flavors of "nullability", e.g. the old NaN hack for Numpy float dtypes and None for object dtype. It is the way it is for good historical reasons, but now that we have proper nullable dtypes, the existence of these special cases only increases the importance of a coherent design going forward.