6 ms·
GoAccess – Visual Web Log Analyzer
- joaodlf 10y agoThis looks quite useful, and I think I'll be using it. I'm impressed by it being written in C... Especially because it can also run on the browser! Also, the default storage being Hash Tables. You don't see many modern projects like this! Actually feel like diving into the code.
- M4v3R 10y agoI used it and I can tell you that it works exactly as advertised :).
- jalfresi 10y agoCurrently using this on one of our clients sites and I love it! More tools like this I say!
- Exuma 10y agoHow is the performance when dealing with large log files? I get about 300 million pageviews a month
- SnaKeZ 10y agoSee #Performance in: https://goaccess.io/faq https://goaccess.io/faq > A dataset of about 52M hits (12GB size) is parsed in 20 mins (in-memory), 60 mins (on-disk storage).
- Neil44 10y agoMy logs are about 500MB a week and it takes less than a minute to process on my 2 core / sata server.
- nodesocket 10y agoThis looks fantastic. Any idea if it supports custom NGINX log formats? log_format main '<$host> $remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"';
- mcescalante 10y agoYes, it supports custom log formatting but you'll likely have to write a bit of parsing code in the config, see docs here: https://goaccess.io/man#custom-log https://goaccess.io/man#custom-log There are also a lot of log formatting q&a in the github issues (both closed & open) if you're using a popular format.
- rjain15 10y agoDoes it similar query and search capabilities like Splunk or Kibana. Is it a full stack log system or just a UI?
- tyingq 10y agoIt appears to specifically parse web server logs. Unique visitors, counts of 404 pages, etc. Similar (in purpose) to "Awstats" or "Webalizer" that you find on almost every shared hosting account.
- joshyi 10y agoIt seems is still in the works... https://github.com/allinurl/goaccess/issues/117 https://github.com/allinurl/goaccess/issues/117
- tzaman 10y agoCan I use this in Docker/Kubernetes?
- daenney 10y agoYes. Just mount the logs into the container.
- tzaman 10y agoActually, the problem is that Docker logs to stdout rather than to logfile(s)
- daenney 10y agoThat's what Docker's logging drivers are for. It does so by default, but it doesn't have to. https://docs.docker.com/engine/admin/logging/overview/ https://docs.docker.com/engine/admin/logging/overview/
- billconan 10y agoCan I use this to visualize custom data? for example, can I push stock prices to this platform and visualize them in real time?
- blitzd 10y agoIt's a web log analysis tool, not a general data viz platform. This may be more what you're looking for though: http://www.cyclotron.io http://www.cyclotron.io
- namelezz 10y agoRemind me of this cool project called netdata[0], also written in C. [0] - https://github.com/firehol/netdata https://github.com/firehol/netdata
- trynewideas 10y agoI've used both. GoAccess tends to be more useful to me when I have a set of log files and want to track down something specific to the web server config. That's particularly handy when working on a server I'm not already familiar with, or when I have logs but no access to the server, since I don't have to run GoAccess on the server itself. It's also the only IIS log analyzer I bother using. netdata is more useful for tracking live stats across live servers that I'm actively maintaining (especially multiple servers), and pulls a broader set of information from more sources, which is handier for tracking down intermittent networking issues. They can both do a little bit of each other's job, which is nice. I tend to use GoAccess more often but think they're both pretty good tools for visualizing server data.
- moreorless 10y agoI am having the hardest time getting goaccess to work with the IIS 8.0 W3C logs. It keeps on complaining about the date / time format. What is your secret sauce?
- joshyi 10y agoTry uncommenting the relevant date/time/log formats from your goaccess.conf. Then run goaccess -f access.log -p /path/goaccess.conf Or https://gist.github.com/soheilpro/a34957550b1bd7d42be2 https://gist.github.com/soheilpro/a34957550b1bd7d42be2
- flas9sd 10y agojust the other day - in lieu with AS ipblock info and 'rgxg cidr' - helped me see at a single glance (3xx to 2xx relative count) some redirect deficiencies of a platforms http image fetchers. As you can pipe to it, it complements the other text utilities filtering logs very well.
- mtarnovan 10y agoQuite useful. Would be nice if it could be fed from piped input too.
- blitzd 10y agoIt works just fine with piped input actually, I use it regularly with zcat.
- trynewideas 10y agoIndeed, most of the GoAccess docs around pipes provide examples for using them to backfill missing expected features, like filtering by date range, file/MIME type, status code, etc.
- pierrec 10y agoI use `find` to choose specific months and pipe the log files into goaccess (if you have lots of logs, that saves time instead of piping everything). Assuming the logs are in a bunch of gzipped files: find my_logs/ -name "*.gz" | grep '2016/10|2016/11' | xargs -I file zcat file | goaccess
- angry-hacker 10y agoShould I have log files if I host a static (very high traffic) site? Everything is static. I've turned off the logging, no idea what performance gain it has, if any, but is there a reason I should write then in file? I don't see any need.
- Something1234 10y agoProbably should. The logs have some pretty interesting details in them. http://attrition.org/postal/logs/2006/logs.001.html http://attrition.org/postal/logs/2006/logs.001.html
- voltagex_ 10y agoNSFW. Should've guessed due to it being attrition.org.
- sonicrocketman 10y agoI've been using GoAccess for a year now as a traffic analysis tool on my blog. It's been great. Easy to use, and great for real-time analysis using the command-line interface.