5 ms·
Lnav – An advanced log file viewer for the small-scale
- hericium 4y agoDo we need lnav thread every 3 weeks?
- xdfgh1112 4y agoFirst I'd heard of it. I have wanted a tool like this for a while.
- asah 4y agoSame, and I'm working on an IoT project that precisely needs this (superhigh$ per device, don't need scale or efficiency)
- mhitza 4y agoYes /s Not everyone checks out daily HN to see all the reposts. Pulling numbers out of the air, I'd say that 5-10% of daily content on HN is a repost, should get used to it. Just hide it an carry on is what I would suggest.
- joshka 4y agolnav is pretty great. I wish it had a library that could be pulled out into an app instead of a terminal. There are a few things that make more sense with a GUI like standard keyboard shortcuts, filtering, saving exports, etc.
- hericium 4y agoNot saying there's something wrong with lnav - just that spamming HN every 3 weeks isn't great. There were submissions with links to lnav.org main site: - 21 days ago - 22 days ago - 46 days ago - 68 days ago It's a spam campaign from my pov.
- __float 4y agoDo you browse new? None of these have had more than a couple points since https://news.ycombinator.com/item?id=9294622 https://news.ycombinator.com/item?id=9294622 in 2015. It seems very reasonable that most visitors simply never saw any of those submissions.
- chalst 4y agoFive submissions in 70 days is rather less than a spam campaign in my view.
- qbasic_forever 4y agoKeep a log of how often it occurs and we can review it... in lnav
- LinuxBender 4y agoThe package is available in Alpine so I tried it out. It seems to work best if the terminal is using en_US.UTF-8. My default is C but I have a function to set UTF-8. I was curious if it could handle stdin and it does. It saves the output in ~/.config/lnav/ something to probably add to my clean up script. It is convenient that it can prepend timestamps to logs. I could see this being nice for people that do QA on build logs all day or that have to debug access logs often. If used with sensitive data one should probably add a shell function and log rotate scripts to clean up ~/.config/lnav/ using the shred command.
- c0wb0yc0d3r 4y agoI was under the impression that the shred command was pretty hard on SSDs. Did that change?
- LinuxBender 4y agoIt would be as hard as writing the file {n} number of times again. So if one created a 1MB log file and then did a 3 pass wipe that would write 3MB. Number of passes would really just depend on the system being worked on. If PCI I would expect 7 passes and I am sure that would just be factored into the cost of the servers for that environment. Shred also isn't perfect as it has no concept of the file systems journal and does not clean that data but I think it is still good practice for sensitive data, in addition to filesystem or file level encryption on systems with highly sensitive data. With sensitive data one must look at the value of the data vs the value of the SSD. If I go out of my way to extend the life of my SSD, do I risk losing $5billion? That is how I factor in whether or not extra wear-and-tear on a SSD makes sense.
- electroly 4y agoShred doesn't work at all on SSDs because wear leveling will spray the writes all over the drive, rather than overwriting the blocks you intended to. Your random bytes will end up in new blocks, not overwriting the original blocks. Trying to shred individual files is a totally pointless exercise on SSDs. It isn't doing what you want it to do.
- lathiat 4y agoDoes anyone have a trick for connecting to remote hosts as a non-root user but having it elevate to root to read the logs?
- prmoustache 4y agoFor systemd/journald managed log, best is to add non privileged user to the systemd-journal group. For logs managed by syslog daemons, most implementations allows you to set the owner and group of the log files. You could decide on a specific log group and add the desired dedicated user to this group. In the end it is usually better to ship the logs to a dedicated machine/space/database.
- tobylvp 4y agoFor Windows users, there is LogViewPlus. (I am the author). https://www.logviewplus.com/ https://www.logviewplus.com/ It has a similar feature set (tail, syntax highlighting, SQL reporting) with a focus on accessing files remotely.
- tstack 4y agoI'm the author of lnav and had never heard of logviewplus before, thanks for mentioning it. It looks like we've independently arrived at a very similar feature set, that's pretty neat. I will now shamelessly steal some of your ideas :)
- tobylvp 4y agoIt looks like you have done a great job with lnav - it's impressive. I know how much work has gone into LogViewPlus and you must have made a similar commitment. I thought I was the only one crazy enough to want to solve this problem. :-) I would be happy to chat to a like mind if you are interested. You can contact me here: https://www.logviewplus.com/contact.aspx https://www.logviewplus.com/contact.aspx Steal away! :-)
- winter_blue 4y agoIs LogViewPlus heavily tied to Windows? How hard would a port to Linux be? I love that so many Windows tools get a nice-looking GUI, versus a heavy CUI lean to *nix tools.
- tobylvp 4y agoAs things currently stand - hard. LogViewPlus is written in .Net using WinForms, so it is heavily tied to Windows. However, Microsoft is evolving. The bulk of the code can now be made to be cross-platform easily, but the front-end remains a problem. I would love to do a Linux port as soon as there is a viable cross-platform GUI technology with strong 3rd party vendor support. The controls in LogViewPlus are very rich.
- kcbanner 4y ago
- zulu-inuoe 4y agoSay I'm building a logging library, is there any sort of generally agreed-upon standard per what a log file should look like? I know there's several formats that lnav supports, but I'm not familiar with them. I've always had trouble before figuring out what a good compromise between log format and flexibility looks like. Especially wrt newlines in the log message itself (eg if I want to log a stack trace) Any thoughts for a format that would generally work ootb in lnav or other log viewers?
- 10987654321 4y agoNot sure about lnav but most log aggregation systems support json and logfmt-formatted logs, and there are many standard logging libraries that supports emitting those formats Of those json is better if you want to be able to do more advanced stuff (nest dictionaries, use lists, ...) and logfmt is better if you want to have human-readable logs without external tools as well, an example line can look like msg="Request finished" tag=request_finish status=200 user=brandur@mutelight.org user_id=1234 app=mutelight app_id=1234 Some more info here https://www.brandur.org/logfmt https://www.brandur.org/logfmt
- jteppinette 4y agoAgree with logfmt. I wrote the logfmter python library: https://github.com/jteppinette/python-logfmter https://github.com/jteppinette/python-logfmter. You can quickly have all of your logs (including 3rd party) converted to this style.
- tstack 4y agolnav does support JSON-lines and logfmt logs. For JSON-lines, it will pretty-print the log messages to make them human readable. For logfmt, I seem to remember the spec not being very clear on quoting semantics (maybe I'm wrong). Anyhow, I would suggest using JSON since it has pretty broad support at this point.
- 10987654321 4y agoHmmm, it doesn't say in https://lnav.org/features#automatic-log-format-detection https://lnav.org/features#automatic-log-format-detection but i see that at least json (and xml) is mentioned under the pretty-printing header. You would know what is supported what with you being the author, just saying that the docs aren't super clear from a quick glance :). And yes, I second the suggestion to focus on JSON. The main benefit of logfmt is that it's simpler for a human to parse directly but in general you probably shouldn't aim for that so..
- mihaigalos 4y agoNice work! I've written something similar based on regexes: https://github.com/mihaigalos/pipeview https://github.com/mihaigalos/pipeview
- _bax 4y agoI'm using every day on multiple erver and application. It a MUST
- yellow_lead 4y agoIt's cool you can SSH to demo it. More CLI tools should have that.
- tstack 4y agoThe SSH demo was inspired by git.charm.sh
- hoyd 4y agoRelated to this, is there a tool like this, but that can show some kind of simple analytics, for example page views over a period of time from an access log produced by python html server.
- guessmyname 4y ago> […] a tool like this, but that can show some kind of simple analytics Yes, you can use → https://goaccess.io/ https://goaccess.io/
- tobylvp 4y agoI mentioned this below, but you could try LogViewPlus. It has a built in SQL based reporting feature you can use. You can see it in action here: https://www.logviewplus.com/docs/create_a_custom_report.html https://www.logviewplus.com/docs/create_a_custom_report.html
- kiliancs 4y agoYou can run queries on the log files and write them to files. I have used these files to feed pre-built spreadsheets with charts, etc.
- tstack 4y agoAs mentioned in a sibling comment, https://goaccess.io https://goaccess.io is probably best for simple analytics on log files. If you want to write more complicated queries, lnav exposes log data through SQLite vtables[1]. So, you can do a SQL query and get a simple bar chart visualization. [1] - https://docs.lnav.org/en/latest/sqlext.html#sqlite-interface https://docs.lnav.org/en/latest/sqlext.html#sqlite-interface
- haspre 4y agoWorks fine on my pi.. thx With caddy server
- Thaxll 4y agoI mean if the goal is to read logs the default color scheme looks pretty bad, why there are so many colors in the example? For example the http 403/404 is red but it does not stand out. Also why a 404 would be red in the first place.
- tstack 4y ago> why there are so many colors in the example? Identifiers, like process-names/pids/IPs, are semantically highlighted by default to make it easier to visually match up values that are the same. And, it's this way because I like it this way, so that's the default. > For example the http 403/404 is red but it does not stand out. To me, the red stands out. But, I would also have a much wider window, so the red would stand out even more with the rest of the text. > Also why a 404 would be red in the first place. Because 4XX are error codes. Of course, lnav is pretty customizable at this point, so you can adjust it to your liking. The theme can be changed to something less colorful and the log format definitions can be patched to change their behavior.
- calmoo 4y agoI used this extensively when working in a cloud support role, incredible tool.
- shae 4y agoThis tool saved me weeks of work once. You gotta try it.