5 ms·

Show HN: Desed – Debugger for Sed

115 points by soptikha2 6y ago

soptikha2 6y ago
Hi, author here. I’ve written a debugger for sed in Rust. This was not only to learn rust, but to actually have a solid debugger for sed. I’ve started learning sed recently and decided to start writing various algorithms in it. And as sed doesn’t have numbers and can just filter/transform text, this is a challenge even for something like comparing two numbers. I’ve seen people do amazing things with it. I’d be glad for any comments regarding code quality or the debugger itself.
- chubot 6y ago
  What does the GNU sed debugger interface look like? Which docs did you use, and were they sufficient? Does the interface resemble that of other GNU tools like bash and make? I've always wondered about these projects. Everyone says that bash and make are hard to debug, but I've never heard of anyone using them. It seems to be more common to use batch mode tracing rather than debugging. What did you use for the TUI? e.g. Curses? http://bashdb.sourceforge.net/ http://bashdb.sourceforge.net/ http://bashdb.sourceforge.net/remake/ http://bashdb.sourceforge.net/remake/ http://gmd.sourceforge.net/ http://gmd.sourceforge.net/
  - soptikha2 6y ago
    I'll add technical section to readme, but for short. I've used tui-rs[1] with crossterm[2] backend (which handles the actual terminal interaction). The docs may seem intimidating, but it's surprisingly easy to use. I've considered using curses, but tui-rs seemed that it will be easier to handle. I can certainly recommend it for making tui applications. I didn't actually use any documentation at all, it wasn't in the man page and I didn't feel like hunting for it, as it was pretty easy to understand just by looking at the output. Sed debug works by running the code specified just as normal, but it annotates what it does to stdout. So it might look like this: https://gist.github.com/SoptikHa2/a50e90bd1b34c7238944c20d1a0a80e3 https://gist.github.com/SoptikHa2/a50e90bd1b34c7238944c20d1a... Sed actually has pretty well-designed interface IMO, it's pretty minimal and has no nasty surprises. Edit: Here[3] is something I've written about it. [1]: https://github.com/fdehau/tui-rs https://github.com/fdehau/tui-rs [2]: https://github.com/crossterm-rs/crossterm https://github.com/crossterm-rs/crossterm [3]: https://soptik.tech/articles/building-desed-the-sed-debugger.html https://soptik.tech/articles/building-desed-the-sed-debugger...
    - chubot 6y ago
      OK thanks, that's helpful! It looks like a lot of the newer terminal rendering library use "diff" style of rendering like web frameworks, rather than something more stateful like curses? Is that why you thought it would be easier? You just update the entire page on every action, and it takes care of the details of coming up with the minimal terminal codes? ---- This is only tangentially related, but for those interested in programming in obscure languages, I recently discovered ble.sh which is a 30K+ line TUI for bash written in pure bash! It does basically what the fish shell does, so it has a terminal rendering library, utf-8 decoder, terminal decoder, bash parser with an AST, etc. all in bash. https://github.com/akinomyoga/ble.sh https://github.com/akinomyoga/ble.sh Some details from the author on this wiki page: https://github.com/oilshell/oil/wiki/How-Interactive-Shells-Work https://github.com/oilshell/oil/wiki/How-Interactive-Shells-... Other big shell programs: https://github.com/oilshell/oil/wiki/The-Biggest-Shell-Programs-in-the-World https://github.com/oilshell/oil/wiki/The-Biggest-Shell-Progr... Latest Oil release makes progress on running these: http://www.oilshell.org/blog/2020/04/release-0.8.pre4.html http://www.oilshell.org/blog/2020/04/release-0.8.pre4.html
      - soptikha2 6y ago
        Yes, my priority was making the UI done as fast as possible, as I wanted to focus on making the debugger and didn't want to think too much about UI. I looked at some examples of both of them and decided that tui-rs looks easier to use, it takes care of everything for me. Note that I might be wrong, I didn't spend even half an hour looking at the frameworks. Another reason, albeit really small, was that tui-rs was written specifically for rust, and curses framework I looked into were essentially just C bindings, making it a bit weird to use. To be fair, this disadvantage is by far outweighed by the fact that curses is far more popular than tui-rs, and there is so much more tutorials and documentation to be found about it. But I wanted to focus on writing the debugger, so I went with what seemed to be more convenient option and paid the price of lower control and performance - for example I have troubles implementing syntax highlighting because tui-rs tries to be clever with mine ansi escape codes. --- Thanks for the links, especially the ble.sh looks amazing.
- lhoursquentin 6y ago
  Really cool stuff, I've been working on being able to debug sed scripts as well by translating it to pseudo readable C[1], compiling the result and then using gdb to step in the script, but this TUI is much more readable! [1] https://github.com/lhoursquentin/sed-bin https://github.com/lhoursquentin/sed-bin
- dmix 6y ago
  I like the tui UI, what was it like mapping it layout in the terminal? I've always wanted to try building a CLI interface like that.
  - soptikha2 6y ago
    Surprisingly easy! I was scared of building TUI like this because it seemed so complex, but it's actually not that hard. Or at least, with tui-rs[1], which is surprisingly easy to pick up. Once you read two or three examples, you'll gain pretty good overview of how to build applications with it. I can definitely recommend it, it does all the heavy lifting and leaves surprisingly little for you to care about. Just don't forget to unset terminal raw mode before your application exits, or else you'll start breaking people's terminals :-) [1]: https://github.com/fdehau/tui-rs https://github.com/fdehau/tui-rs
  - billjings 6y ago
    A friend of mine built this, and I feel the need to promote it: https://github.com/dankamongmen/notcurses https://github.com/dankamongmen/notcurses There are Rust and C++ bindings, and it performs some truly ridiculous TUI tricks.
smt88 6y ago
If you need a debugger from sed, aren't you now using the wrong tool for the job? We have lots of mature, debuggable languages to process text...
- soptikha2 6y ago
  I certainly do. This is meant just as a toy, and way to learn Rust, which is what I've written this in. If I need to actually achieve something, I use python or haskell. However sed is perfect if you have spare time and want to write basic programs in unnecessarily complex way for fun.
  - jessaustin 6y ago
    A long time ago I wrote a reformatting utility for a client in sed, because my manager didn't want to have to support the utility and Anderson claimed it couldn't be done in any language. The Androids whined and then they rewrote it in C++, which they had to support.
  - u801e 6y ago
    > However sed is perfect if you have spare time and want to write basic programs in unnecessarily complex way for fun. There are certain things that are difficult to do in sed, but are much easier to do in ed. Though ed doesn't have the branch command and labels, having essentially random access throughout the file makes up for it.
- esmi 6y ago
  > We have lots of mature, debuggable languages to process text... You mean like sed? Sed was written in the 70s and has been in mission critical stuff for decades on many platforms. What are you recommending that is more mature than that?
  - archgoon 6y ago
    Sed does not have a debugger that is 50 years old. The author was not saying that sed is a bad tool; he's saying it doesn't have a debugger. If there is a desire to have a debugger for the tool, that has existed for 50 years without having one, perhaps the tool is not being used in the way it has been used for the last 50 years. This would seem to make the argument that the tool has been tried and tested less relevant.

115 points by soptikha2 6y ago

Hi, author here. I’ve written a debugger for sed in Rust. This was not only to learn rust, but to actually have a solid debugger for sed. I’ve started learning sed recently and decided to start writing various algorithms in it. And as sed doesn’t have numbers and can just filter/transform text, this is a challenge even for something like comparing two numbers. I’ve seen people do amazing things with it. I’d be glad for any comments regarding code quality or the debugger itself.

chubot 6y ago
What does the GNU sed debugger interface look like? Which docs did you use, and were they sufficient? Does the interface resemble that of other GNU tools like bash and make? I've always wondered about these projects. Everyone says that bash and make are hard to debug, but I've never heard of anyone using them. It seems to be more common to use batch mode tracing rather than debugging. What did you use for the TUI? e.g. Curses? http://bashdb.sourceforge.net/ http://bashdb.sourceforge.net/ http://bashdb.sourceforge.net/remake/ http://bashdb.sourceforge.net/remake/ http://gmd.sourceforge.net/ http://gmd.sourceforge.net/
- soptikha2 6y ago
  I'll add technical section to readme, but for short. I've used tui-rs[1] with crossterm[2] backend (which handles the actual terminal interaction). The docs may seem intimidating, but it's surprisingly easy to use. I've considered using curses, but tui-rs seemed that it will be easier to handle. I can certainly recommend it for making tui applications. I didn't actually use any documentation at all, it wasn't in the man page and I didn't feel like hunting for it, as it was pretty easy to understand just by looking at the output. Sed debug works by running the code specified just as normal, but it annotates what it does to stdout. So it might look like this: https://gist.github.com/SoptikHa2/a50e90bd1b34c7238944c20d1a0a80e3 https://gist.github.com/SoptikHa2/a50e90bd1b34c7238944c20d1a... Sed actually has pretty well-designed interface IMO, it's pretty minimal and has no nasty surprises. Edit: Here[3] is something I've written about it. [1]: https://github.com/fdehau/tui-rs https://github.com/fdehau/tui-rs [2]: https://github.com/crossterm-rs/crossterm https://github.com/crossterm-rs/crossterm [3]: https://soptik.tech/articles/building-desed-the-sed-debugger.html https://soptik.tech/articles/building-desed-the-sed-debugger...
  - chubot 6y ago
    OK thanks, that's helpful! It looks like a lot of the newer terminal rendering library use "diff" style of rendering like web frameworks, rather than something more stateful like curses? Is that why you thought it would be easier? You just update the entire page on every action, and it takes care of the details of coming up with the minimal terminal codes? ---- This is only tangentially related, but for those interested in programming in obscure languages, I recently discovered ble.sh which is a 30K+ line TUI for bash written in pure bash! It does basically what the fish shell does, so it has a terminal rendering library, utf-8 decoder, terminal decoder, bash parser with an AST, etc. all in bash. https://github.com/akinomyoga/ble.sh https://github.com/akinomyoga/ble.sh Some details from the author on this wiki page: https://github.com/oilshell/oil/wiki/How-Interactive-Shells-Work https://github.com/oilshell/oil/wiki/How-Interactive-Shells-... Other big shell programs: https://github.com/oilshell/oil/wiki/The-Biggest-Shell-Programs-in-the-World https://github.com/oilshell/oil/wiki/The-Biggest-Shell-Progr... Latest Oil release makes progress on running these: http://www.oilshell.org/blog/2020/04/release-0.8.pre4.html http://www.oilshell.org/blog/2020/04/release-0.8.pre4.html
    - soptikha2 6y ago
      Yes, my priority was making the UI done as fast as possible, as I wanted to focus on making the debugger and didn't want to think too much about UI. I looked at some examples of both of them and decided that tui-rs looks easier to use, it takes care of everything for me. Note that I might be wrong, I didn't spend even half an hour looking at the frameworks. Another reason, albeit really small, was that tui-rs was written specifically for rust, and curses framework I looked into were essentially just C bindings, making it a bit weird to use. To be fair, this disadvantage is by far outweighed by the fact that curses is far more popular than tui-rs, and there is so much more tutorials and documentation to be found about it. But I wanted to focus on writing the debugger, so I went with what seemed to be more convenient option and paid the price of lower control and performance - for example I have troubles implementing syntax highlighting because tui-rs tries to be clever with mine ansi escape codes. --- Thanks for the links, especially the ble.sh looks amazing.
lhoursquentin 6y ago
Really cool stuff, I've been working on being able to debug sed scripts as well by translating it to pseudo readable C[1], compiling the result and then using gdb to step in the script, but this TUI is much more readable! [1] https://github.com/lhoursquentin/sed-bin https://github.com/lhoursquentin/sed-bin
dmix 6y ago
I like the tui UI, what was it like mapping it layout in the terminal? I've always wanted to try building a CLI interface like that.
- soptikha2 6y ago
  Surprisingly easy! I was scared of building TUI like this because it seemed so complex, but it's actually not that hard. Or at least, with tui-rs[1], which is surprisingly easy to pick up. Once you read two or three examples, you'll gain pretty good overview of how to build applications with it. I can definitely recommend it, it does all the heavy lifting and leaves surprisingly little for you to care about. Just don't forget to unset terminal raw mode before your application exits, or else you'll start breaking people's terminals :-) [1]: https://github.com/fdehau/tui-rs https://github.com/fdehau/tui-rs
- billjings 6y ago
  A friend of mine built this, and I feel the need to promote it: https://github.com/dankamongmen/notcurses https://github.com/dankamongmen/notcurses There are Rust and C++ bindings, and it performs some truly ridiculous TUI tricks.

smt88 6y ago

If you need a debugger from sed, aren't you now using the wrong tool for the job? We have lots of mature, debuggable languages to process text...

soptikha2 6y ago
I certainly do. This is meant just as a toy, and way to learn Rust, which is what I've written this in. If I need to actually achieve something, I use python or haskell. However sed is perfect if you have spare time and want to write basic programs in unnecessarily complex way for fun.
- jessaustin 6y ago
  A long time ago I wrote a reformatting utility for a client in sed, because my manager didn't want to have to support the utility and Anderson claimed it couldn't be done in any language. The Androids whined and then they rewrote it in C++, which they had to support.
- u801e 6y ago
  > However sed is perfect if you have spare time and want to write basic programs in unnecessarily complex way for fun. There are certain things that are difficult to do in sed, but are much easier to do in ed. Though ed doesn't have the branch command and labels, having essentially random access throughout the file makes up for it.
esmi 6y ago
> We have lots of mature, debuggable languages to process text... You mean like sed? Sed was written in the 70s and has been in mission critical stuff for decades on many platforms. What are you recommending that is more mature than that?
- archgoon 6y ago
  Sed does not have a debugger that is 50 years old. The author was not saying that sed is a bad tool; he's saying it doesn't have a debugger. If there is a desire to have a debugger for the tool, that has existed for 50 years without having one, perhaps the tool is not being used in the way it has been used for the last 50 years. This would seem to make the argument that the tool has been tried and tested less relevant.