9 ms·
I really love the wave of new tools that are gaining traction in the Python world: tqdm, rich (and soon textual), fastpi, typer, pydantic, shiv, toga, doit, dis
by BiteCode_dev 5y ago
I really love the wave of new tools that are gaining traction in the Python world: tqdm, rich (and soon textual), fastpi, typer, pydantic, shiv, toga, doit, diskcache...
With the better error messages in 3.10 and 11, plus the focus on speed, it's a fantastic era for the language and it's a ton of fun.
I didn't expect to find back the feeling of "too-good-to-be-true" I had when starting with 2.4, and yet.
In fact, being a dev is kinda awesome right now, no matter where you look. JS and PHP are getting more ergonomics, you get a load of new low level languages to sharpen your hardware, Java is modern now, C# runs on Unix, the Rust and Go communities are busy with shipping fantastic tools (ripgrep, fdfind, docker, cue, etc), windows has a decent terminal and WSL, my mother is actually using Linux and Apple came up with the M1. IDEs and browsers are incredible, and they do eat a lot of space, but I have a 32 Go RAM + 1TO SSD laptop that's as slim as a sheet of paper.
Not to mention there is a lot of money to be made.
I know it's trendy right now to say everything is bad in IT, but I disagree, overall, we have it SO good.
- ropable 5y agoI'll add BeeWare (https://beeware.org/ https://beeware.org/) as a pretty nice nice to deploy Python applications cross-platform easily. It's more like a suite of tools, though.
- trutannus 5y agoTQDM isn't all that new, it's one of the first things I started to play with when learning to write code. That said, it is an amazing tool.
- dheera 5y agoYeah, I've been using it for a while, but with the barrage of memes and politics on the internet nobody finds out about the good stuff until years later.
- BiteCode_dev 5y agotqdm is indeed 8 years old, but it's been popping up everywhere for only a couple of years now. Hence the "gaining traction".
- underdeserver 5y agoI used an early version of this in 2009.
- yairchu 5y agoIve used tqdm 20 years ago
- BiteCode_dev 5y agoI've used tqdm 30 years ago! Although the first release on pypi was in 2013: https://pypi.org/project/tqdm/#history https://pypi.org/project/tqdm/#history
- yairchu 5y agoWay before @noamraph put it up on in pypi it was internally available in our company. To my knowledge he didn't yet make it 30 years ago, but if he did then probably at least not for Python!
- underdeserver 5y agoWise of you...
- butwhywhyoh 5y agoOP didn't take issue with "gaining traction" and clearly was responding to "new tools". Tqdm isn't a new tool.
- DSingularity 5y agoIt’s been making some tqdm lately
- genidoi 5y agoNew features get added and these posts are a nice reminder to check the docs on a tool I literally never need to do that for because it's so habitual to wrap any long-running iterable inside of a `tqdm()`. TIL tqdm now has a dead simple telegram hook https://tqdm.github.io/docs/contrib.telegram/ https://tqdm.github.io/docs/contrib.telegram/
- rripken 5y agoIt never occurred to me that chat posts could be quickly edited to make them dynamic. Maybe other chat-bots do that too. Its definitely a cool add-on.
- iooi 5y agotqdm has been around for ages now, it's not exactly new.
- BiteCode_dev 5y agotqdm is indeed 8 years old, but it's been popping up everywhere for only a couple of years now. Hence the "gaining traction".
- waterproof 5y agoYou said "wave of new tools that are gaining traction" maybe you meant to say "wave of tools newly gaining traction".
- BiteCode_dev 5y agoA new wave "of tools gaining traction"
- butwhywhyoh 5y agoHow would you separate this new wave from any old waves? I've never been aware of "waves of traction". Please just admit you made a mistake instead of responding to everyone trying to defend your use of the word "new" on an old tool. :D
- deleted 5y ago[deleted]
- BiteCode_dev 5y agoI though I wrote it correctly, but I'm French, so I'm going to assume I haven't.
- rbdixon 5y agodoit [0] is a superb toolkit for building task-oriented tools with dependencies. It isn't too complex to get started and gives you a lot to work with. I've used it for an internal tool used by everyone in the company for 5+ years and it has never given me a headache. [0]: https://pydoit.org https://pydoit.org
- BiteCode_dev 5y agoAnd surprisingly underrated. I mean, it's declarative, works on Windows, easy things are (very) easy, and because you can mix and match bash and python actions, hard things are suspiciously easy too. Given how complicated the alternatives are (maeven, ninja, make, gulp...), you'd think it would have taken the world for years. Yet I've only started to see people in the Python core dev team use it this year. It's only getting traction now.
- genidoi 5y ago"hard things are suspiciously easy too." Care to share an example?
- BiteCode_dev 5y agoHere is a task that groups all static files from a web project into a directory. It has to make sure a bunch of directories exist, run a node js command to build some files from the proper dir, then run a python command to regroup them + all static files from all django apps into one dir. Simple. But then I had a problem I had to hack around. This required me to change an entry to a generated TOML file on the fly at every build. doit just lets me add a 5 lines python function that does whatever I want, and insert it between my bash tasks, and I'm done. def task_bundle_static_files(): def update_manifest(): conf = Path("var/static/manifest.toml") data = toml.loads(conf.read_text()) data["./src/main.js"] = data["index.html"] conf.write_text(toml.dumps(data)) return { "actions": [ "mkdir -p ./var/static/", "rm -fr ./var/static/* ", "cd frontend/; npm run build", "python manage.py collectstatic --noinput", update_manifest, "cp -r ./var/static/* ", ] }
- systemvoltage 5y agoDespite of all the hate Python gets (pkg system), it’s still my go to language. Most fun. Most rewarding. Getting things done fast. I can go home with Python at 5pm. And have a good time with my family.
- BiteCode_dev 5y agoRight? Doing advent of code in Python is almost cheating. It feels like playing a video game.
- mroche 5y agoI did AoC 2021 through Day 7 in Python, before switching to Go (for personal learning, can't remember if times listed below were with "go run" or from running after "go build"). D7 actually in both, to try and solve a problem I was having with my Python performance. But, it was actually a big learning moment for me when it came to structuring my logic. My Python code in Part 2 (some minor adjustments from Part 1) took about 40 seconds to run. Terrible, but usable to get a proper answer. I was able to bring it down to ~25 seconds with some optimization by adding a calculated lookup dictionary per loop. Now, the same exact logic in Go ran in about 0.8 seconds. However, I wasn't satisfied with this and realized that if I moved the dictionary to be global rather than per-loop, I would be able to realize significant gains in performance by completing eliminating redundant calculations. This change dropped the runtime from 25 seconds to 0.35 seconds (of course, applying the same logic in Go brought it down from 0.8s to 0.05s). Due to the nature of performance you can get out of the Python interpreter, it can actually lead you down paths of learning better optimization strategies that you may initially write off in other languages (depending on the use case) because they perform inherently better. It made me think a bit more about what I was doing and how I could improve it since (in this particular case), the impact of not doing so was pretty drastic.
- matsemann 5y agoI don't understand your solution to Day7. What kind of dictionary, and how come even a naive solution would be so slow? My Kotlin solution runs in about a second. And it was even so stupid that I didn't calculate the sum of the arithmetic series directly, but through a loop. Can't fathom something being slower. line.split(",").map { it.toInt() }.let { crabs -> (0..crabs.maxOf { it }).minOf { pos -> crabs.sumOf { (0..abs(it - pos)).sum() // slower than calculating arithmetic sum, but quicker to write } } } My proper Kotlin solution runs in less than a ms, though. I do think that "it's good that python is slow because it forces you to optimize" is a weird take, though.
- karmasimida 5y agotqdm is like around for 5 years or something now
- dogline 5y agoI had to look these up (just me doing Google searches): - tqdm: progress bars (https://tqdm.github.io/ https://tqdm.github.io/) - rich: text formatting (https://github.com/willmcgugan/rich https://github.com/willmcgugan/rich) - textual: TUI, using rich (https://github.com/willmcgugan/textual https://github.com/willmcgugan/textual) - fastpi: Rest APIs (https://fastapi.tiangolo.com/ https://fastapi.tiangolo.com/) - typer: CLI Library, uses Click (https://typer.tiangolo.com/ https://typer.tiangolo.com/) - pydantic: Custom data types (https://pydantic-docs.helpmanual.io/ https://pydantic-docs.helpmanual.io/) - shiv: Create Python zipapps (https://shiv.readthedocs.io/en/latest/ https://shiv.readthedocs.io/en/latest/) - toga: GUI Toolkit (https://toga.readthedocs.io/en/latest/ https://toga.readthedocs.io/en/latest/) - doit: Task runner (https://pydoit.org/ https://pydoit.org/) - diskcache: A disk cache (https://github.com/grantjenks/python-diskcache/ https://github.com/grantjenks/python-diskcache/)
- choppaface 5y agoThanks for the links! FWIW python has a built-in disk cache—- shelve ( https://docs.python.org/3/library/shelve.html https://docs.python.org/3/library/shelve.html )
- BiteCode_dev 5y agoDiskcache has the amazing benefit of transparent sharding, so you can use it in multithreading and multiprocessing programs.
- zuj 5y agoDamn, so many goodies to play with. Thanks for the list.
- rcarmo 5y agoYou can find most of those here: https://taoofmac.com/space/dev/python https://taoofmac.com/space/dev/python. Those that aren't should be there momentarily :)
- zuj 5y agoJust wanted to add Austin: Python frame stack sampler for CPython written in pure C (https://github.com/P403n1x87/austin https://github.com/P403n1x87/austin)
- axg11 5y agoYour positivity and enthusiasm is infectious. Thanks for being an optimist.
- deleted 5y ago[deleted]
- hpen 5y agoI love Python for many things: scripts, data science, prototypes, etc. But I would never use it to build a large backend system. I just can't handle all the warts. Yes it has been used by large companies, but not without their problems.
- bshipp 5y agoYou use Python to rapidly build a small backend system and then, if any warts appear that need to be smoothed over, you carve that function off to a more robust solution. Python is great for programming at the speed of inspiration.
- hpen 5y agoTo me Python is fast just like not writing tests is fast.
- syngrog66 5y agofor prototypes and MVPs time-to-learning and time-to-market are king, and programmer time is the most expensive thing. in that niche Python is helpful and tests just slow you down tests are super helpful and wise in any sort of long term or safety-critical software, obviously so, tests (and Python) are not inherently a win or a loss. they just give you different trade-offs times to write testless Python and times to write test-heavy Go, Rust etc.
- hpen 5y agoGenerally I believe in engineering tradeoffs but I think some tests are always important. Even with MVPs. Writing test might double your time to market, but they will allow you to move faster and make less regressions once in production. The early days are some of the least stable features wise and having tests in place to verify things when massively changing a codebase is nice. Also, it can greatly help add confidence to newcomers who don't know the codebase.
- 5y ago
- fnord77 5y agotoo bad python is the least energy efficient programming language, by a lot. It's like rolling coal of programming.
- bohrious 5y agoIt's energy inefficient like a school bus, not inefficient for the sake of it, but because it's a decent way to shuttle passengers of all experience levels through all weather.
- d0mine 5y agoDo you seriously think that 20 cars are more economic than a single bus? Python performance problems are exaggerated for many use-cases (hot paths are not written in Python e.g., matching regexes happens in C code(re,regex), parsing xml too (elementtree, lxml), sqlite, numpy,scipy (C, Fortran), etc. Cython makes it trivial to drop to C level where necessary)
- zuj 5y agoPython might not be energy efficient in the hardware it is definitely efficient in the wetware.
- make3 5y agofire is also really nice for easily building cli
- MichaelMoser123 5y agohttps://github.com/keleshev/schema https://github.com/keleshev/schema a library for validating python data structures by schema definitions, gave me a wow moment. (i have a slightly less original one here: https://github.com/MoserMichael/kwchecker https://github.com/MoserMichael/kwchecker )
- ehsankia 5y agoLove the idea of Schema, but not a fan of that syntax, doesn't seem Pythonic to me, seems more like something you'd see out of Javascript word. Like looking at that very first example, I have no clue what "len" means in that context. Is it implicitly checking that it's not an empty string? Then on the next line, how come `int` has `Use()` around it, but on the previous line `str` didn't? I guess that int is being used as a converter, line on the next line with str.lower, but the str was being used as a type check? Not a fan of that API.
- smus 5y agoI'd chip in dramatiq as a nice python tasks/message passing library
- jansky 5y agorich is great - exactly this library i used to solve same problem as tqdm does and it includes estimation (ETA) too! Rich is just amazing library