8 ms·

I'm mostly with you (except for your last sentence). I've picked up a lot of languages over the years, and can fall back into most of them pretty fast, but sed

I'm mostly with you (except for your last sentence). I've picked up a lot of languages over the years, and can fall back into most of them pretty fast, but sed and awk especially are hard. On the occasions that they really are the best tool for the job, I get actual books out and set aside a window of a few hours to plod through it. I think the problem is that they fall into an uncanny valley between "simple tool" and "verbose programming language"; they are really terse languages wearing a simple tool's sheepskin.

sitkack 5y ago
If I measure time to solve a problem and not line count, using a language like Python or Ruby might be 2x the lines , for a line count approaching zero anyways, not sure that matters. Performance is comparable and I don't need to relearn anything and someone else who doesn't know awk/sed doesn't need to learn it. And when one is using awk/sed that script is often invoked from a shell, bash for instance. And since the awk/sed can't hold its own as a general purpose programming language, now I am writing bash code, a personal sin. Awk/sed made sense for the time and place they are from, but things have changed.
- thaumaturgy 5y ago
  Yes, this is a Standard HN Reply: the "Specialist's Rebuttal". It's usually in some form of, "___ is obsolete, ___ is better in every way, everyone should just be using ___ now." Okay, acknowledged. Python and Ruby are superior in every regard. Would you mind letting us chat about sed and awk now for a minute?
  - yesenadam 5y ago
    I replied to someone's "There is no reason for anyone to program in C nowadays" recently on here with "But I like programming in C". They hadn't considered that reason in their mental survey of Every Possible Reason. Also.. I use Awk every day, for all kinds of things. I forget the order of parameters in split, sub etc, which takes a few seconds to look up with man awk. I got into sed a while ago but never use it. Awk, together with sort and uniq sometimes, is all I need. And it is interactive. You have an input. Print that out and look what needs to be done. Write the first transforming step in Awk. See what that did. Add the next step. See what that did. Extremely interactive. After a few minutes I have a one-liner (often 2 or 3 wrapped lines) that does exactly what I want, and easy to adapt to other similar things. I have very large bash history enabled so they stay Ctrl-R-accessible.
    - sitkack 5y ago
      Heh, Not over here doing arm chair yucking someone's yum, to you and everyone else in this shrub of the conversation. If you guys want to do SCA or model trains with ancient programming languages, go all in. I play around with Forth and APL as well. If the premise is the title of the post "day to day usage ..." then total task efficiency is important. Are you moving the goal posts and putting words in my mouth? If I have to enumerate EPR then we also suffer from the HN Standard Reply where caveats and defensive pirouettes are wedged between all the words. I think you and @thaumaturgy should chill a bit and put some dovetails on that C compiler. Oh and @thaumaturgy, I didn't say and don't think Python/Ruby are superior, my comment was addressing suitability for a task in the larger context. They were used as a concrete example. Please be a little more charitable or threads like these fall into a "someone is wrong on the internet". I just realized that the people that probably need the most assistance arguing (in the classical sense) on the internet are the ones that make it. Ironic. Or, the barbarians will and have moved in. *replying to your Also ... Your interactive workflow sounds like (and this is only an indicative example) my use of IPython the console based advanced python repl. # now here goes the philosophical rant portion of the reply But at the same time, the host isn't bash it is Python, so I can also use the full power of the language without having to think about how to serialize text between tools I am composing. Because composition is more important that each individual tool and the shell is lousy; awk/sed are shell enablers. If you are only working with line oriented text and some wellish defined delimiters, go for it. We now operate over structured formats, we should have tools that compose well with those formats so that we an think about domain problems and not self-made problems. Tools that have supplanted awk/sed are like jq and gron with new ones popping up all the time. This is where the Little Language innovation is going on. Something like jq is powerful enough both in expressivity and compute performance to part of my goto bag of tools. https://github.com/stedolan/jq https://github.com/stedolan/jq https://github.com/TomNomNom/gron https://github.com/TomNomNom/gron (it really makes json RIPgreppable) https://github.com/BurntSushi/ripgrep https://github.com/BurntSushi/ripgrep I hope the peanut gallery enjoys our exchange because we aren't even talking about the same things. Your building model sailboats and I am talking about problem solving efficiency so we can focus on domain problems. I wish Erik was here to school us both.
      - deleted 5y ago
  - kerblang 5y ago
    Wait I need to toss an extra bomb over the wall and point out the superiority of swapping "sed" out for "perl -pe" and getting everything sed had plus superior regex without needing to learn much of anything new. (I might be wrong about "everything" but that's what bomb-tossing is all about)
    - sitkack 5y ago
      Welcome to the discussion. I like it, I haven't perled in a while but does that mean you can evolve from a one liner to a full program using all of perl using this technique?
      - kerblang 5y ago
        I suppose it's a "pathway to perl" in the sense of a mild gateway drug. Perl is definitely one-liner-friendly. Every time I get into Perl I have to relearn it all over again (which is apparently not an isolated phenomenon), so I don't have great suggestions for digging further. I just habitually reach for perl -pe about 3/4 of the time.
  - dang 5y ago
    "Be kind. Don't be snarky." "Please don't sneer, including at the rest of the community." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- baby 5y ago
  I really wish all these bash scripts would be replaced with python scripts.
  - stjohnswarts 5y ago
    Yeah I redid some scripts on my team for my own usage because our team bash guru maintained them and didn't want anyone to change them. So I did my own versions in python. Sure they were mostly 2x+ as long but they were very easy to read and modify and test. The bash guru finds out and brings it up at a team meeting trying to shame me, but it ended up with my boss asking the team, and they voted on using my scripts in lieu of the older bash scripts as I had written a few unit tests around them and the rest of the team understood them without digging out unix tomes.
    - sitkack 5y ago
      I have done something similar, what I really found worked well was to wrap the bash scripts in python to instrument them and if they just operate on data, you can even spawn another command and run them both at the time confirming that the output matches. Great way to have confidence in a port and provide for a fallback path. Eventually the new code will take over the old code and the old code will wither and die.
      - stjohnswarts 5y ago
        It seems like when a bash script gets over say 50 lines or so I start doubting whether something should be a bash script :) . I saw an IRC client written entirely in bash once and it was quite a resource of esoteric bash scripting haha.
- sergiosgc 5y ago
  > Awk/sed made sense for the time and place they are from, but things have changed. Don't do that. Do not assume something is of no use because you don't know how to use it. Double that recommendation for tools that have widespread use. There is a reason they're used everywhere. In the case of awk/sed/perl: Nothing beats them for composition in one-liner text processing pipelines.
  - pnutjam 5y ago
    plus, they exist in the vast majority of unix like systems. You don't have to install anything. They are just there and available to use. Even using them in a simplistic manner is light-years ahead of trying to do stuff manually.
  - sitkack 5y ago
    I have moved on to structured data, awk and sed handle line oriented text formats. Don't assume I don't know how to use it. :) One line verses ten, I'll take the more readable thing over the APL.
- bear8642 5y ago
  >Performance is comparable Can't find links now, but fairly certain were stories in past about how switching to AWK from other tools increased performance
stjohnswarts 5y ago
I gave up on them except for the very most common uses of them for slicing/dicing text, after the most essential basics I just turn to python and suck it up. Learning to pipe things in/out in python is a great skill to have in these circumstances. I think perl is the king of this but I'm not going back to that... never again :)

sitkack 5y ago

If I measure time to solve a problem and not line count, using a language like Python or Ruby might be 2x the lines , for a line count approaching zero anyways, not sure that matters. Performance is comparable and I don't need to relearn anything and someone else who doesn't know awk/sed doesn't need to learn it. And when one is using awk/sed that script is often invoked from a shell, bash for instance. And since the awk/sed can't hold its own as a general purpose programming language, now I am writing bash code, a personal sin. Awk/sed made sense for the time and place they are from, but things have changed.

thaumaturgy 5y ago
Yes, this is a Standard HN Reply: the "Specialist's Rebuttal". It's usually in some form of, "___ is obsolete, ___ is better in every way, everyone should just be using ___ now." Okay, acknowledged. Python and Ruby are superior in every regard. Would you mind letting us chat about sed and awk now for a minute?
- yesenadam 5y ago
  I replied to someone's "There is no reason for anyone to program in C nowadays" recently on here with "But I like programming in C". They hadn't considered that reason in their mental survey of Every Possible Reason. Also.. I use Awk every day, for all kinds of things. I forget the order of parameters in split, sub etc, which takes a few seconds to look up with man awk. I got into sed a while ago but never use it. Awk, together with sort and uniq sometimes, is all I need. And it is interactive. You have an input. Print that out and look what needs to be done. Write the first transforming step in Awk. See what that did. Add the next step. See what that did. Extremely interactive. After a few minutes I have a one-liner (often 2 or 3 wrapped lines) that does exactly what I want, and easy to adapt to other similar things. I have very large bash history enabled so they stay Ctrl-R-accessible.
  - sitkack 5y ago
    Heh, Not over here doing arm chair yucking someone's yum, to you and everyone else in this shrub of the conversation. If you guys want to do SCA or model trains with ancient programming languages, go all in. I play around with Forth and APL as well. If the premise is the title of the post "day to day usage ..." then total task efficiency is important. Are you moving the goal posts and putting words in my mouth? If I have to enumerate EPR then we also suffer from the HN Standard Reply where caveats and defensive pirouettes are wedged between all the words. I think you and @thaumaturgy should chill a bit and put some dovetails on that C compiler. Oh and @thaumaturgy, I didn't say and don't think Python/Ruby are superior, my comment was addressing suitability for a task in the larger context. They were used as a concrete example. Please be a little more charitable or threads like these fall into a "someone is wrong on the internet". I just realized that the people that probably need the most assistance arguing (in the classical sense) on the internet are the ones that make it. Ironic. Or, the barbarians will and have moved in. *replying to your Also ... Your interactive workflow sounds like (and this is only an indicative example) my use of IPython the console based advanced python repl. # now here goes the philosophical rant portion of the reply But at the same time, the host isn't bash it is Python, so I can also use the full power of the language without having to think about how to serialize text between tools I am composing. Because composition is more important that each individual tool and the shell is lousy; awk/sed are shell enablers. If you are only working with line oriented text and some wellish defined delimiters, go for it. We now operate over structured formats, we should have tools that compose well with those formats so that we an think about domain problems and not self-made problems. Tools that have supplanted awk/sed are like jq and gron with new ones popping up all the time. This is where the Little Language innovation is going on. Something like jq is powerful enough both in expressivity and compute performance to part of my goto bag of tools. https://github.com/stedolan/jq https://github.com/stedolan/jq https://github.com/TomNomNom/gron https://github.com/TomNomNom/gron (it really makes json RIPgreppable) https://github.com/BurntSushi/ripgrep https://github.com/BurntSushi/ripgrep I hope the peanut gallery enjoys our exchange because we aren't even talking about the same things. Your building model sailboats and I am talking about problem solving efficiency so we can focus on domain problems. I wish Erik was here to school us both.
    - deleted 5y ago
- kerblang 5y ago
  Wait I need to toss an extra bomb over the wall and point out the superiority of swapping "sed" out for "perl -pe" and getting everything sed had plus superior regex without needing to learn much of anything new. (I might be wrong about "everything" but that's what bomb-tossing is all about)
  - sitkack 5y ago
    Welcome to the discussion. I like it, I haven't perled in a while but does that mean you can evolve from a one liner to a full program using all of perl using this technique?
    - kerblang 5y ago
      I suppose it's a "pathway to perl" in the sense of a mild gateway drug. Perl is definitely one-liner-friendly. Every time I get into Perl I have to relearn it all over again (which is apparently not an isolated phenomenon), so I don't have great suggestions for digging further. I just habitually reach for perl -pe about 3/4 of the time.
- dang 5y ago
  "Be kind. Don't be snarky." "Please don't sneer, including at the rest of the community." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
baby 5y ago
I really wish all these bash scripts would be replaced with python scripts.
- stjohnswarts 5y ago
  Yeah I redid some scripts on my team for my own usage because our team bash guru maintained them and didn't want anyone to change them. So I did my own versions in python. Sure they were mostly 2x+ as long but they were very easy to read and modify and test. The bash guru finds out and brings it up at a team meeting trying to shame me, but it ended up with my boss asking the team, and they voted on using my scripts in lieu of the older bash scripts as I had written a few unit tests around them and the rest of the team understood them without digging out unix tomes.
  - sitkack 5y ago
    I have done something similar, what I really found worked well was to wrap the bash scripts in python to instrument them and if they just operate on data, you can even spawn another command and run them both at the time confirming that the output matches. Great way to have confidence in a port and provide for a fallback path. Eventually the new code will take over the old code and the old code will wither and die.
    - stjohnswarts 5y ago
      It seems like when a bash script gets over say 50 lines or so I start doubting whether something should be a bash script :) . I saw an IRC client written entirely in bash once and it was quite a resource of esoteric bash scripting haha.
sergiosgc 5y ago
> Awk/sed made sense for the time and place they are from, but things have changed. Don't do that. Do not assume something is of no use because you don't know how to use it. Double that recommendation for tools that have widespread use. There is a reason they're used everywhere. In the case of awk/sed/perl: Nothing beats them for composition in one-liner text processing pipelines.
- pnutjam 5y ago
  plus, they exist in the vast majority of unix like systems. You don't have to install anything. They are just there and available to use. Even using them in a simplistic manner is light-years ahead of trying to do stuff manually.
- sitkack 5y ago
  I have moved on to structured data, awk and sed handle line oriented text formats. Don't assume I don't know how to use it. :) One line verses ten, I'll take the more readable thing over the APL.
bear8642 5y ago
>Performance is comparable Can't find links now, but fairly certain were stories in past about how switching to AWK from other tools increased performance

stjohnswarts 5y ago

I gave up on them except for the very most common uses of them for slicing/dicing text, after the most essential basics I just turn to python and suck it up. Learning to pipe things in/out in python is a great skill to have in these circumstances. I think perl is the king of this but I'm not going back to that... never again :)