5 ms·
Boltons: A set of BSD-licensed, pure-Python utilities
- deleted 8y ago[deleted]
- jszymborski 8y agoSome of these are really fantastic. Particularly the atomic file saving and cache utils, but I can see myself using a lot of these handy util modules. A lot of this stuff is great to implement and test yourself in large codebases, but when you're prototyping or writing a quick notebook, this stuff ranges to very handy to very powerful.
- j88439h84 8y agoToolz is one of my favorite python libraries. I use it all the time. It provides even more of this useful functionality. It also has a sped-up version called Cytoolz written with Cython. https://toolz.readthedocs.io https://toolz.readthedocs.io https://github.com/pytoolz/cytoolz https://github.com/pytoolz/cytoolz
- ktpsns 8y agoToolz is really a game changer since it allows you to write elegant functional code (lazy evaluations, currying, pipes, parallelism) in Python which feels native. As python has the "batteries included" philosophy, such useful gems should be included into the standard library...
- TylerE 8y agoI will kind of disagree. While having stuff in stdlib can be convenient, it all tends to be where active development goes to die. Compare httplib with something like Requests.
- maxerickson 8y agoThe comparison would be with urllib.request: https://docs.python.org/3/library/urllib.request.html https://docs.python.org/3/library/urllib.request.html Which right there recommends Requests, but for simple tasks it's kind of unfortunate that people bring in a library dependency.
- pmiller2 8y agoCan anyone comment on the efficiency or lack thereof of toolz?
- merlincorey 8y agoIt uses `itertools` internally very heavily, which means if you generally are using generators and such, you maintain those benefits, as well as get to work with infinite sequences, etc. If you do find a `toolz` function to be too slow, there's always the `cytoolz` versions.
- stochastic_monk 8y agocytoolz’s frequencies was around 10x as fast as using a vanilla dict and counting elements manually, which was in turn twice as fast as collections.Counter when I measured them a few years ago. It’s solid work.
- webmaven 8y agoHow does cytoolz performance compare with toolz under PyPy?
- candiodari 8y ago> Chunked and windowed iteration, in iterutils I need this. Makes so many functional expressions so simple.
- pmoriarty 8y agoCould you give some examples?
- btgeekboy 8y agoI wrote some similar code one time that generated SQL which was manually reviewed. It generated a massive WHERE IN clause. Oracle only accepts 1000 values. So I fed it a list of thousands of values, and it chunked them into rows of N objects, and into larger chunks of 1000. Having some code to do that automatically would be quite nice.
- Bootvis 8y agoWhy not insert the values in WHERE IN to a temporary helper table and join on that? It seems somewhat easier and might perform better.
- orf 8y agoOracle has a limit on the number of parameters in a query, not just in the `WHERE IN` clause. So you'd need to insert them into the helper table in batches of 1000.
- Bootvis 8y agoGood point, but isn’t that done automatically by whatever tool your using? And if the table your selecting from is big enough I would expect that this setup would be faster but it’s hard to say. It would also more easily guarantee that your results will be consistent.
- gugagore 8y ago
- VectorLock 8y agohttps://github.com/mahmoud/boltons/blob/master/boltons/easterutils.py https://github.com/mahmoud/boltons/blob/master/boltons/easte...
- maxpert 8y agoGob's program!
- mhashemi 8y agoLibrary author and lazy README updater here to report that technically as of October 09, 2018, boltons is 79 types and 146 functions, for a grand total of 225 utilities. While I'm here, here are the ones I use most: - OrderedMultiDict: https://boltons.readthedocs.io/en/latest/dictutils.html#boltons.dictutils.OrderedMultiDict https://boltons.readthedocs.io/en/latest/dictutils.html#bolt... - esp the .inverted() and .sorted() methods - Exponential backoff and jitter generator: https://boltons.readthedocs.io/en/latest/iterutils.html#boltons.iterutils.backoff https://boltons.readthedocs.io/en/latest/iterutils.html#bolt... - remap (recursive map): https://boltons.readthedocs.io/en/latest/iterutils.html#nested https://boltons.readthedocs.io/en/latest/iterutils.html#nest... (recipes here: https://sedimental.org/remap.html https://sedimental.org/remap.html) - Atomic File saving: https://boltons.readthedocs.io/en/latest/fileutils.html#boltons.fileutils.atomic_save https://boltons.readthedocs.io/en/latest/fileutils.html#bolt... - Traceback utilties, structured tracebacks: https://boltons.readthedocs.io/en/latest/tbutils.html https://boltons.readthedocs.io/en/latest/tbutils.html Also, my favorite bolton that's not a bolton, glom: https://glom.readthedocs.io/en/latest/ https://glom.readthedocs.io/en/latest/ If you have a bolton you'd like to submit, I'm pretty merge-happy. Here are the criteria: https://boltons.readthedocs.io/en/latest/architecture.html https://boltons.readthedocs.io/en/latest/architecture.html
- jmadler 8y agoWhy not contribute these upstream into the standard library?
- marmaduke 8y agoThe standard library has a higher barrier to entry than PyPI.
- mhashemi 8y agoBecause most of these utilities were created to build things, and contributing to the stdlib has never been on one of those critical paths? But really, @marmaduke's got it right. The standard library has a MUCH higher barrier to entry than PyPI. Have you seen python-ideas [1]? I truly do not have time to have it out with some of the more vocal elements of that group. But, down the list: - OMD might be a good fit for the collections module, I've chatted about it a bit with Raymond Hettinger on and off. He wasn't totally against it, so that's something! - Exponential backoff is too opinionated for itertools, and generally a lot more high-level and conceptually modern than the rest of Python's built-in networking facilities. - remap: I really like it, even though I'm typically not a big fan of functional programming in Python. But the Python devs are even less appreciative than I, with GvR disliking lambdas, and Py3 dropping reduce() from the builtins. Plus after years of watching Python core dev, it doesn't seem like the people with the time to work on core Python have much time left over to work with IRL complex APIs and other sources of dynamic nested data? - Atomic file saving probably should go in the stdlib, but probably not relying on ctypes (on Windows) and I'm not going to pull out my old Windows laptop again just to start writing C. - Traceback utilities: Probably the one I wish I had time to push for most. traceback's string-only approach feels really dated in the structured-logging age. I'd expect a lot of FUD around messing with error handling. Maybe now that boltons is pretty popular it's worth giving it a go just to see what lies beyond the typical listserv response: "put it on PyPI and if it's popular, we'll see" :) [1]: https://mail.python.org/pipermail/python-ideas/ https://mail.python.org/pipermail/python-ideas/
- kristaps 8y agoWhile convenient, some of the utils can be slowish due to input validation which might not be strictly necessary, etc. - check the implementation if using in a performance critical task. Apart from that, very useful lib, highly recommended.
- deleted 8y ago[deleted]
- _pmf_ 8y agoBe careful when googling ...
- fermigier 8y agoI'm a big fan of @mhashemi's work in Python in general, even if I don't use Boltons personally. The discussion regarding inclusion in the stdlib is sensible, and I fully agree with @mhashemi's comments. See similar discussions, for instance, regarding dataclasses vs. attrs (https://github.com/ericvsmith/dataclasses/issues/19 https://github.com/ericvsmith/dataclasses/issues/19). Sometimes, however, it's a bit annoying when something that you think should be widely available isn't included in the stdlib. The cons of course are: - Fragmentation (several project pursuing similar objectives) - Not being able to use a powerful construct to simplify code in the stdlib itself (the stdlib can only depend on the stdlib). Regarding, for instance, functional programming, I maintain a list of interesting projects in this domain in Python: https://github.com/sfermigier/awesome-functional-python/blob/master/README.md#libraries https://github.com/sfermigier/awesome-functional-python/blob... There are at least half a dozen libraries that strive to achieve similar goals, and would probably benefit from being included in the stdlib (at least partially). It's hard to pick a winner among the list (like other in this thread, I went with toolz). I believe that a way to solve (at least partially) this conundrum could be to work on some common specifications, like the JS community did with Fantasyland (https://github.com/fantasyland/fantasy-land https://github.com/fantasyland/fantasy-land), and then let projects implements the specification the way they want. WDYT ?
- WCityMike 8y agoFirst thought was a mental picture of multiple Michael Boltons.