5 ms·

I dislike some of these. Some are cool, like 'Self', but giant "swiss army knife" libraries feel odd. > Avoid boilerplate when setting instance attributes dat

by orf 6y ago

I dislike some of these. Some are cool, like 'Self', but giant "swiss army knife" libraries feel odd. > Avoid boilerplate when setting instance attributes dataclasses[1] > Type Dispatch functools.singledispatch works with type annotations[2] > A more useful __repr__ dataclasses[1] > A better pathlib.Path path_object.read_text()[3]. The pickle stuff is interesting, but I don't see the value over just pickle.load(path_object). Monkeypatching Path to add this fees odd and unpythonic. 1. https://docs.python.org/3/library/dataclasses.html https://docs.python.org/3/library/dataclasses.html 2. https://docs.python.org/3/library/functools.html#functools.singledispatch https://docs.python.org/3/library/functools.html#functools.s... 3. https://docs.python.org/3/library/pathlib.html#pathlib.Path.read_text https://docs.python.org/3/library/pathlib.html#pathlib.Path....

pbsds 6y ago
I agree on the monkeypatching of pathlib. It makes it difficult for a reader to look up what the functions do. Providing a different module, like `import nPathlib as Pathlib` would be a lot more discoverable.
- jph00 6y ago
  Your coding environment really should be providing that basic functionality for you.
  - Too 6y ago
    Do you have example of environment that can do this? I don’t think even pycharm manages. While I generally agree with your sentiment, dynamic monkey patching and heavy meta programming like this is usually not very friendly for any implementation of auto complete.
ebg13 6y ago
> functools.singledispatch works with type annotations[2] Only for the first argument. "Note that the dispatch happens on the type of the first argument, create your function accordingly". Making delegated kwargs transparent is 100% cool though and addresses a huge problem with delegated interfaces. I'd use that all the time. I agree with you about the swiss army thing. I wish it weren't bundled with all this other stuff.
- dragonwriter 6y ago
  Multimethod library is mature, handles multiple dispatch, works with type annotations, and appears to have been stable and relatively complete more than a half decade before fastcore was released.
formerly_proven 6y ago
Using pickle for "Path.save/load" feels like a shortcut to me that is inappropriate for many circumstances, since loading a pickle in Python literally means running code from that pickle. It's basically eval(Path.read()). It's not the kind of trust relationship that I would typically expect from a load/save pair. > Wait! What's going on here? We just imported pathlib.Path - why are we getting this new functionality? If you are writing a library and this is part of your expose, think long and hard. > Path.ls Highly gimmicky, doesn't support globs. Arguments are "n_max, file_type and file_exts", in that order. n_max seems largely pointless for real programs and like a workaround for a poor REPL for interactive use. The difference between the last two arguments is that the first one also accepts mimetypes (does this call "file" on every file it sees?). Arguably the signature of a "Path.ls" should rather look like "Path.ls(glob='<star>', <star>, files=True, dirs=True)" and it should probably not return a bunch of strings.
- jph00 6y ago
  Look closer. It doesn't return strings.
  - formerly_proven 6y ago
    Distinction without difference, since Path is just sugar around a string. Directory listing functions should generally return something that mimics dirent/WIN32_FIND_DATA to avoid double work. For example, filtering for file type can typically be done without an extra stat call per file.
jph00 6y ago
I'm the author of fastcore. dataclasses is not very extensible or flexible. It's fine for starting with, but it's not at all a replacement for store_attr etc. At fast.ai we used dataclasses extensively for a couple of years, but it just kept biting us. So eventually we switched to these different approaches, which have worked much better for us. singledispatch is nice enough too, but multiple dispatch with typedispatch is fat more flexible and powerful. Just because something is in the stdlib, doesn't mean it's the perfect solution to every problem, and that everyone should try to hack their code to fit into those existing solutions.
- nemetroid 6y ago
  > dataclasses is not very extensible or flexible. It's fine for starting with, but it's not at all a replacement for store_attr etc. > At fast.ai we used dataclasses extensively for a couple of years, but it just kept biting us. What were the issues you had?
  - Recursing 6y ago
    https://www.attrs.org/en/stable/why.html#data-classes https://www.attrs.org/en/stable/why.html#data-classes The most used python library for these things is attrs, I think it mostly provides support for slotted classes (more memory efficient), "converters" (normalization), validation and serialization
  - jph00 6y ago
    The key issue is that it requires inheritance, but doesn't actually support subclassing very well. Also it relies on python's type system, which is rather limited. It's a very large module for what it does, and understanding all the details of how it works in practice is a big challenge.
ghj 6y ago
I didn't know about functools.singledispatch/singledispatchmethod, it looks really nice! For typing hinting, I've been annotating with @overload which is just god awful ugly. And you still need to switch based on instance type for the actual implementation. I will try this way instead! [1] https://www.python.org/dev/peps/pep-0484/#function-method-overloading https://www.python.org/dev/peps/pep-0484/#function-method-ov...

by orf 6y ago

pbsds 6y ago

I agree on the monkeypatching of pathlib. It makes it difficult for a reader to look up what the functions do. Providing a different module, like `import nPathlib as Pathlib` would be a lot more discoverable.

jph00 6y ago
Your coding environment really should be providing that basic functionality for you.
- Too 6y ago
  Do you have example of environment that can do this? I don’t think even pycharm manages. While I generally agree with your sentiment, dynamic monkey patching and heavy meta programming like this is usually not very friendly for any implementation of auto complete.

ebg13 6y ago

> functools.singledispatch works with type annotations[2] Only for the first argument. "Note that the dispatch happens on the type of the first argument, create your function accordingly". Making delegated kwargs transparent is 100% cool though and addresses a huge problem with delegated interfaces. I'd use that all the time. I agree with you about the swiss army thing. I wish it weren't bundled with all this other stuff.

dragonwriter 6y ago
Multimethod library is mature, handles multiple dispatch, works with type annotations, and appears to have been stable and relatively complete more than a half decade before fastcore was released.

formerly_proven 6y ago

Using pickle for "Path.save/load" feels like a shortcut to me that is inappropriate for many circumstances, since loading a pickle in Python literally means running code from that pickle. It's basically eval(Path.read()). It's not the kind of trust relationship that I would typically expect from a load/save pair. > Wait! What's going on here? We just imported pathlib.Path - why are we getting this new functionality? If you are writing a library and this is part of your expose, think long and hard. > Path.ls Highly gimmicky, doesn't support globs. Arguments are "n_max, file_type and file_exts", in that order. n_max seems largely pointless for real programs and like a workaround for a poor REPL for interactive use. The difference between the last two arguments is that the first one also accepts mimetypes (does this call "file" on every file it sees?). Arguably the signature of a "Path.ls" should rather look like "Path.ls(glob='<star>', <star>, files=True, dirs=True)" and it should probably not return a bunch of strings.

jph00 6y ago
Look closer. It doesn't return strings.
- formerly_proven 6y ago
  Distinction without difference, since Path is just sugar around a string. Directory listing functions should generally return something that mimics dirent/WIN32_FIND_DATA to avoid double work. For example, filtering for file type can typically be done without an extra stat call per file.

jph00 6y ago

I'm the author of fastcore. dataclasses is not very extensible or flexible. It's fine for starting with, but it's not at all a replacement for store_attr etc. At fast.ai we used dataclasses extensively for a couple of years, but it just kept biting us. So eventually we switched to these different approaches, which have worked much better for us. singledispatch is nice enough too, but multiple dispatch with typedispatch is fat more flexible and powerful. Just because something is in the stdlib, doesn't mean it's the perfect solution to every problem, and that everyone should try to hack their code to fit into those existing solutions.

nemetroid 6y ago
> dataclasses is not very extensible or flexible. It's fine for starting with, but it's not at all a replacement for store_attr etc. > At fast.ai we used dataclasses extensively for a couple of years, but it just kept biting us. What were the issues you had?
- Recursing 6y ago
  https://www.attrs.org/en/stable/why.html#data-classes https://www.attrs.org/en/stable/why.html#data-classes The most used python library for these things is attrs, I think it mostly provides support for slotted classes (more memory efficient), "converters" (normalization), validation and serialization
- jph00 6y ago
  The key issue is that it requires inheritance, but doesn't actually support subclassing very well. Also it relies on python's type system, which is rather limited. It's a very large module for what it does, and understanding all the details of how it works in practice is a big challenge.

ghj 6y ago

I didn't know about functools.singledispatch/singledispatchmethod, it looks really nice! For typing hinting, I've been annotating with @overload which is just god awful ugly. And you still need to switch based on instance type for the actual implementation. I will try this way instead! [1] https://www.python.org/dev/peps/pep-0484/#function-method-overloading https://www.python.org/dev/peps/pep-0484/#function-method-ov...