11 ms·

This looks interesting! A tool in the same vein that I already use is pyp (`pypyp` on PyPI). This project, pypipe, has built-in record splitting and CSV suppo

This looks interesting! A tool in the same vein that I already use is pyp (`pypyp` on PyPI). This project, pypipe, has built-in record splitting and CSV support that pyp doesn't. CSV is lower-friction. pyp automatically determines what modules to import. It is very convenient and would be nice to have here. A version of `pypprint` (https://github.com/hauntsaninja/pyp/blob/9408446a41bfdc60e44c08e1b395596b5daed033/README.md#pyp-will-attempt-to-intelligently-print-dicts-and-iterables https://github.com/hauntsaninja/pyp/blob/9408446a41bfdc60e44...) may be useful, too. The most famous Python command-line tool like this is probably Mario (https://github.com/python-mario/mario https://github.com/python-mario/mario), which isn't maintained. pyp's readme compares Mario and some other alternatives: https://github.com/hauntsaninja/pyp/blob/9408446a41bfdc60e44c08e1b395596b5daed033/README.md#related-projects https://github.com/hauntsaninja/pyp/blob/9408446a41bfdc60e44.... pypipe is different from most for also having a feature like `--explain` in pyp (code generation).

make3 3y ago
https://github.com/alecthomas/pawk https://github.com/alecthomas/pawk is also great, & auto imports any module that you use
- bugen 3y ago
  pawk is awesome. What concise code! I absolutely adore the simplicity of pawk.
deleted 3y ago
[deleted]
bugen 3y ago
It's incredible to discover so many projects similar to pypipe! In particular, pyp seems truly magical. If I had known about pyp before, I might not have developed pypipe. However, the lack of prior knowledge about pyp has given pypipe its own unique identity. Learning from these earlier projects, I can see there's still room for growth in pypipe. Thank you for the valuable insights, especially regarding the potential implementation of the auto-import feature.
- networked 3y ago
  You're welcome! > However, the lack of prior knowledge about pyp has given pypipe its own unique identity. This is why I think it is not always bad to reinvent things. Reinventions can have their own advantages, often through a somewhat different focus. There can be value in creating (a prototype of) your own solution before you see how others have solved your problem. (Or even where others draw the line around your problem, which may be different from where you do and override it.) Reinventing and sharing something is also a reliable way to learn about prior art. :-) However, you usually don't want to be the one doing the reinventing. :-) It is not the best use of one's time and resources. A better starting point is to know the state of the art well and have some disagreements with it. Let me share a personal cautionary tale. It doesn't really apply to pypipe but may still be useful or interesting. I once took it too far with static site generators. In order to make it a learning experience, I deliberately avoided studying existing ones in depth. I wrote mine in a niche programming language with what I thought was a fresh perspective. (It kind of was. A big part of it was leaning heavily on an in-memory SQLite database.) I repeated the mistakes of early content management systems, like heavy indirection and too much code in templates. Soon my static site generator had users! And I was just realizing how flawed it was. Oh no. The users were few, but they used the generator for real things, like an event. Some knew about my wheel-reinventing approach and weren't dissuaded by it. I found myself stuck with a subpar design to polish for the 1.0 release. When it was finished, I had to do a fairly difficult and only partial reworking for version 2.0. To encourage moving to the new version and not frustrate those with complex projects, I wrote a migration tool. The most serious projects based on the generator stayed on version 1.x. So, I advise against going this far. It is probably best to do the research and learn about the state of the art first. I try to do it now. Again, this is much less of a concern with pypipe. It doesn't create the same kind of user lock-in as a static site generator. At most, you will have to ask your users to upgrade their shell scripts when something changes.

make3 3y ago

https://github.com/alecthomas/pawk https://github.com/alecthomas/pawk is also great, & auto imports any module that you use

bugen 3y ago
pawk is awesome. What concise code! I absolutely adore the simplicity of pawk.

deleted 3y ago

[deleted]

bugen 3y ago

It's incredible to discover so many projects similar to pypipe! In particular, pyp seems truly magical. If I had known about pyp before, I might not have developed pypipe. However, the lack of prior knowledge about pyp has given pypipe its own unique identity. Learning from these earlier projects, I can see there's still room for growth in pypipe. Thank you for the valuable insights, especially regarding the potential implementation of the auto-import feature.

networked 3y ago
You're welcome! > However, the lack of prior knowledge about pyp has given pypipe its own unique identity. This is why I think it is not always bad to reinvent things. Reinventions can have their own advantages, often through a somewhat different focus. There can be value in creating (a prototype of) your own solution before you see how others have solved your problem. (Or even where others draw the line around your problem, which may be different from where you do and override it.) Reinventing and sharing something is also a reliable way to learn about prior art. :-) However, you usually don't want to be the one doing the reinventing. :-) It is not the best use of one's time and resources. A better starting point is to know the state of the art well and have some disagreements with it. Let me share a personal cautionary tale. It doesn't really apply to pypipe but may still be useful or interesting. I once took it too far with static site generators. In order to make it a learning experience, I deliberately avoided studying existing ones in depth. I wrote mine in a niche programming language with what I thought was a fresh perspective. (It kind of was. A big part of it was leaning heavily on an in-memory SQLite database.) I repeated the mistakes of early content management systems, like heavy indirection and too much code in templates. Soon my static site generator had users! And I was just realizing how flawed it was. Oh no. The users were few, but they used the generator for real things, like an event. Some knew about my wheel-reinventing approach and weren't dissuaded by it. I found myself stuck with a subpar design to polish for the 1.0 release. When it was finished, I had to do a fairly difficult and only partial reworking for version 2.0. To encourage moving to the new version and not frustrate those with complex projects, I wrote a migration tool. The most serious projects based on the generator stayed on version 1.x. So, I advise against going this far. It is probably best to do the research and learn about the state of the art first. I try to do it now. Again, this is much less of a concern with pypipe. It doesn't create the same kind of user lock-in as a static site generator. At most, you will have to ask your users to upgrade their shell scripts when something changes.