15 ms·
Show HN: I wrote a program to convert lines of text into trees
- birchb 6y agoAuthor here: Often I have to digest log files and lists of Azure resource names. I prefer to work with hierarchies of things, so I wrote this simple filter. Turns out to quite handy, especially when combined with awk and its friends.
- jedberg 6y agoVery cool tool! Any chance that you're going to add it to Homebrew?
- eevilspock 6y agohttps://github.com/birchb1024/frangipanni/issues/2 https://github.com/birchb1024/frangipanni/issues/2
- birchb 6y agoMac binary https://github.com/birchb1024/frangipanni/releases/download/v0.4.1/frangipanni_darwin_amd64.zip https://github.com/birchb1024/frangipanni/releases/download/...
- birchb 6y agoI refer you to the answer I gave earlier.
- jarmitage 6y agoGreat, very handy! Just FYI `readlink -f` doesn't work on macOS May I suggest the name txtree?
- birchb 6y agoSorry macos people, I don't own an Apple. (Not strictly true, I have an iMac rescued from a dumpster, but it's running Ubuntu now). Apparently greadlink is available https://stackoverflow.com/a/4031502 https://stackoverflow.com/a/4031502 for you.
- clankyclanker 6y agoI love this as a post-processor for log-files. We actually have something like this at work. Unfortunately, it's applied to log-files by default. It's terrible: it breaks grep, and I haven't been able to figure a way around that. Have you found any solutions for grepping on matching lines? Presumably, one could use awk, but awk takes a lot of cycles.
- birchb 6y agoGrepping the _input_ is, like, Standard. Knowhatimean bruv?
- inadequatespace 6y agoAh yes, I was just thinking that this could be useful for `aws s3 ls` and similar. (Really I just want `tree` on `s3`. There's `s3-tree` but that uses the entire bucket rather than a prefix)
- kevmoo1 6y agoSuper cool, yo! Very unix – do one simple thing well.
- jedberg 6y agoIt's like super enhanced tree command! Very cool!
- lnenad 6y agoThis looks awesome, great job, thanks for sharing.
- AnonHP 6y agoThis is really very nice and useful! I read through the examples and thought about something that would be a good addition and then see that “-skip” was just implemented! I can skip (pun intended) using an additional layer of cut or awk because of this. The detailed examples are great too, showing different use cases and features. Thank you very much for creating this tool and sharing it.
- densekernel 6y agoWhat a beautiful idea. Like the application of quick analysis of ls or logs. Although now we are typically outputting JSON and collecting for Kibana.
- birchb 6y agodid you try the '-format json' option?
- petethepig 6y agoMaybe this is not super relevant, but my favorite hack is that any tree-like structure like this can be browsed with ncdu. Here's a gist for breaking down redis traffic by command for example: https://gist.github.com/petethepig/0f33c910fb2edad8969a5775e23bb99f https://gist.github.com/petethepig/0f33c910fb2edad8969a5775e...
- loevborg 6y agoWhoa, I didn't know that! This is super useful as a general-purpose tree viewer! For the record, the relevant commands are ncdu -o /tmp/files.json and ncdu -f /tmp/files.json
- rakoo 6y agoNeat! Is there some documentation on the format ?
- mellosouls 6y agohttps://dev.yorhel.nl/ncdu/jsonfmt https://dev.yorhel.nl/ncdu/jsonfmt
- shipit 6y agoExceptional! I am a heavy user for `find <> | xargs grep` -- this makes my life so much sweeter. Thank you @birchb!
- breck 6y agoAwesome. You really nailed it. In my experience the output for spreadsheets turns out to be key so good job highlighting that (https://github.com/birchb1024/frangipanni#output-for-spreadsheets https://github.com/birchb1024/frangipanni#output-for-spreads...). One suggestion: it may help to generalize the newline as the node separator. You may already be doing this (my go is rusty) but instead of https://github.com/birchb1024/frangipanni/blob/7543b4ee15ae7f0491e1212a43b7fc61acc7e00a/frangipanni.go#L158 https://github.com/birchb1024/frangipanni/blob/7543b4ee15ae7... be able to override the "newline" as the node separator, like you've done with the "spacer" param. What do you do when the same line is encountered?
- mklein994 6y agoNeat. I'll add this to my toolbox. Somewhat unrelated: I discovered some time ago that the column command (from util-linux) can print trees of hierarchical data (up to 2 levels deep). From the man page: $ echo -e '1 0 A\n2 1 AA\n3 1 AB\n4 2 AAA\n5 2 AAB' | column --tree-id 1 --tree-parent 2 --tree 3 1 0 A 2 1 |-AA 4 2 | |-AAA 5 2 | `-AAB 3 1 `-AB column(1): https://github.com/karelzak/util-linux/blob/master/text-utils/column.1.adoc#examples https://github.com/karelzak/util-linux/blob/master/text-util...
- birchb 6y agoI did not know that. But then it's BSD, right? http://harmful.cat-v.org/cat-v/ http://harmful.cat-v.org/cat-v/.
- kevinmgranger 6y agoI've been thinking about making something like this for a while, this is great! I always thought it was weird that "do one thing and one thing well" stopped short of dealing with tree representations on the commandline.
- globular-toast 6y agoWhat do you mean? The only reason this didn't exist is nobody did it yet. It doesn't contradict "do one thing and do it well".
- macintux 6y agoFor me, the problem isn’t that it doesn’t do one thing well, which is clearly does, it’s that there are no other tools to process trees. So it fits well with the UNIX ethos of a small tool doing one thing well, but not so much the concept of pipelines, which is closely related. Obviously not this tool’s fault.
- eevilspock 6y agoI does take advantage of pipelines, as long as it is on the end. You can use awk, sed, cut, sort, grep and a hose of other tools to massage the data before it gets put into graph form. To become the front-end of a pipe requires that its or some other hierarchical format become an effective standard shared by a number of tools. It can happen.
- rurban 6y agoWhoa, he wrote a parser. This only happens every other decade or so.</sarcasm>
- macintux 6y ago> Be kind. Don't be snarky. Have curious conversation; don't cross-examine. Please don't fulminate. Please don't sneer, including at the rest of the community. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- dang 6y agoYou've been breaking the site guidelines badly. If you keep doing this we will have to ban you. Please review https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and stick to the rules from now on.
- motohagiography 6y agoThis looks like what I always wanted for normalizing data sets and grepping logs but couldn't articulate. Thank you!
- softwaredoug 6y agoTIL from the README, you can insert `img` tags into a markdown file, and github will honor the align tag, etc ``` <img src="frangipanni.jpg" alt="A Tree" width="200" align="right"> ```
- amelius 6y agoLooks nice. Perhaps a next step could be an ncurses program that allows you to fold/unfold the trees at arbitrary places, and select entries to reveal their full path (useful for copy+paste).
- kazinator 6y ago$ find /etc/network | ./frangi.tl etc: network: if-post-down.d: wireless-tools wpasupplicant avahi-daemon if-down.d: resolvconf wpasupplicant avahi-autoipd interfaces.d interfaces if-pre-up.d: wireless-tools wpasupplicant ethtool if-up.d: ntpdate wpasupplicant 000resolvconf openssh-server ethtool avahi-autoipd slrn avahi-daemon $ find /etc/network | ./frangi-cheat.tl /etc/network /if-post-down.d /wireless-tools /wpasupplicant /avahi-daemon /if-down.d /resolvconf /wpasupplicant /avahi-autoipd /interfaces.d /interfaces /if-pre-up.d /wireless-tools /wpasupplicant /ethtool /if-up.d /ntpdate /wpasupplicant /000resolvconf /openssh-server /ethtool /avahi-autoipd /slrn /avahi-daemon $ cat frangi-cheat.tl #!/usr/bin/env txr (let (old-path) (whilet ((line (get-line))) (whenlet ((path (tok #/[^\/]*/ line)) (canon `@{path "/"}`) (pos (mismatch path old-path))) (let ((cpos (max 0 (+ pos -1 [sum [path 0..pos] len])))) (put-line `@{"" cpos}@{canon [cpos..:]}`)) (set old-path path)))) $ cat frangi.tl #!/usr/bin/env txr (defstruct (node name) list-builder name (:method equal (me) me.name) (:method ensure-child (me child-name) (let ((children me.(get))) (or (find child-name me.(get)) (let ((new-child (new (node child-name)))) me.(add new-child) new-child)))) (:method print (me stream : pretty-p) (let* ((old-im (set-indent-mode stream indent-code)) (old-id (get-indent stream)) (children me.(get)) (is-bottom (none children .(get)))) (unwind-protect (cond (children (put-line `@{me.name}:` stream) (set-indent stream (+ old-id 4)) [mapdo (op print @1 stream) children] (when is-bottom (put-char #\newline stream))) (t (put-string `@{me.name} `) stream)) (set-indent-mode stream old-im) (set-indent stream old-id))))) (let ((supernode (new (node :root)))) (whilet ((line (get-line))) (let ((path (tok #/[^\/]+/ line)) (node supernode)) (each ((comp path)) (set node node.(ensure-child comp))))) (each ((top-child supernode.(get))) (pprinl top-child)))
- deleted 6y ago[deleted]
- deleted 6y ago[deleted]
- roydivision 6y agoI find I think about things a lot in tree structures, documentation, todo lists, technical information. I’m really keen to give this a go in my work. Thanks!
- dvirsky 6y agoThis is a really wonderful idea! Thanks, OP.
- eevilspock 6y agoI love what it does for log files. Abbreviated example from the README: May 10 03:17:06 localhost systemd: Removed slice User Slice of root. May 10 03:17:06 localhost systemd: Stopping User Slice of root. becomes: May 10 03:17:06 localhost systemd : Removed slice User Slice of root : Stopping User Slice of root
- quietbritishjim 6y agoDefinitely very interesting! But I think I'd often end up stuck in the middle of a file with no context about the time of the current line, and struggling to scroll up to capture the parent without shooting past it. Maybe that could be resolved if it were combined with some sort of GUI / TUI that collapsed subtrees with default (maybe with child count and total anscentor count) e.g. folding by indentation in Vim / Emacs
- mdeck_ 6y agoIn retrospect, it’s not clear to me why I somehow expected this project to be something more literally arboreal. I even got my hopes up further when I clicked the link and saw that photo of plumerias. Yet, this project is cool enough that I’m not even disappointed.
- birchb 6y agoUm you can get actual trees with graphviz, or my favourite, YeD. I will add an example of that. https://github.com/birchb1024/frangipanni/issues/7 https://github.com/birchb1024/frangipanni/issues/7
- timonoko 6y agoSeen this about 50 years ago. It was method of saving expensive Teletype ink ribbons. Instead of printing same repeating messages on mainframe log, it printed only those parts that were different from previous line. And of course empty line was "ditto". Except this wasted paper and was later replaced with "...".
- oandrew 6y agoThere is also tree --fromfile e.g. echo a/b1 a/b2/c | xargs -n1 | tree --fromfile . └── a ├── b1 └── b2 └── c 2 directories, 2 files