5 ms·
Ask HN: Kibana like data explorer UI for PostgreSQL?
I am exploring an idea of using Postgres + Timescale for storing very small volume of log data and having a "search based" web UI like Kibana's to let me and other users explore the log data.
Anyone here hacked around with such idea? have any open source projects to recommend?
Notes:
ELK will work, but it is an overkill for my needs, plus, I want to build this for fun with PostgreSQL.
I'm not good at Javascript and UI development in general (systems background), and would prefer to hack around some existing project rather than building something from scratch.
- brett-za 8y agoMaybe something like Metabase (https://metabase.com/ https://metabase.com/) would work for you?
- reacharavindh 8y agoThanks the suggestion. I played with Metabase a bit today. It is great for custom dashboards, searchable query links etc. but, I could not find a interface element that would let me "full text search" a column in Postgres table.. Do you have ideas?
- vajrapani666 8y agoYou can use custom queries in metabase and parameterize them, for any parameter you can expose the parameter with a text field available when you embed the "question" anywhere. https://hevodata.com/blog/postgresql-full-text-search-setup/ https://hevodata.com/blog/postgresql-full-text-search-setup/
- reacharavindh 8y agoThe link was very helpful to learn about making use of FTS in Postgres. Thanks! I still could not figure out how to expose a filter as a user modifiable dynamic search box/text field with Metabase. I could not find such a facility in their documentation, nor in their forums. I'm giving up on Metabase for this need.
- bosky101 8y ago+1 for metabase. had to turn off autoscanning though because it starting slowing down other queries when scanning a large table frequently. we also use pgweb linked elsewhere here for engineers. btw, we built a sql statement to spreadsheet + charts interface. DM if interested. intuitively - i think others may find it useful.
- MightySCollins 8y agoWe started with redash and moved to Metabase. It's really quite powerful and and teams great too.
- seektable 8y agoTry our free BI tool https://www.seektable.com https://www.seektable.com It has PostgreSql connector; with 'flat table' report type you can easily filter/sort records without need to write SQL, and pivot tables/charts are good for summary reports (under the hood GROUP BY queries are generated) BTW, SeekTable can connect to ElasticSearch so it may be used with this data source too.
- reacharavindh 8y agoThanks for the suggestion. But, I'm looking for an open source tool that I can tinker with, contribute to and modify to my needs if needed now or in the future. Seektable looks cool for enterprises that wants a hands-off reporting tool. Good luck!
- ahoka 8y agoYou can try Grafana: http://docs.grafana.org/features/datasources/postgres/ http://docs.grafana.org/features/datasources/postgres/
- reacharavindh 8y agoI'm looking for a search interface for searching through logs. Is there any way to achieve such an interface with Grafana? I could only see a query that results in a table view. But, I want to be able to enter the search term dynamically.
- jfkw 8y agoGrafana recently announced Loki: https://grafana.com/blog/2018/12/12/loki-prometheus-inspired-open-source-logging-for-cloud-natives/ https://grafana.com/blog/2018/12/12/loki-prometheus-inspired...
- dewey 8y agohttps://github.com/PostgREST/postgrest https://github.com/PostgREST/postgrest and a very simple ui with an input field and an output list?
- cjlm 8y agohttps://www.github.com/rickbergfalk/sqlpad https://www.github.com/rickbergfalk/sqlpad might be worth a look.
- LogicX 8y agoWe use http://redash.io http://redash.io to look at our timescale data. Works our quite well for charting, and searching... they let you put variables into your sql query, then they become input fields, making search easy.
- reacharavindh 8y agoI think this is exactly what I was looking for. I'm out now trying to figure out how to get redash running on my Mac to see if it is what I want. :-( Sadly, their documentation shows how to quickly setup redash on AWS, GCP, and Docker. But, no instruction about some guy just trying to get it running on a Mac without Docker. https://redash.io/help/open-source/setup https://redash.io/help/open-source/setup I will figure it out eventually, I think.
- arikfr 8y agoWe’re a small team and have to prioritize. At some point maintaining the Mac setup was too much, and Docker covers most use cases. But setup for Postgres should be relatively simple as the extra dependencies required are minimal. If you want you’re welcome to post a question on our forum (discuss.redash.io) and I’ll follow up with pointers on how to set it up on your Mac. Update: just noticed that our developer setup guide without Docker is still online: https://redash.io/help/open-source/dev-guide/setup https://redash.io/help/open-source/dev-guide/setup.
- reacharavindh 8y agoThanks for the link to dev-guide. I missed it because I was looking for instructions for standalone installation and skipped "developer guide" thinking it might be about dev docs about source code org and stuff. Sorry if my response sounded like a snarky comment or a rant. That was not my intention. I can only thank you for an open source project and do not feel entitled for anything. I ended up using Docker compose to get started with redash on my Mac. But, I cringe on the inside every time I am forced to use Docker (a daemon with higher privileges) to run stuff from the Internet, and it runs Postgres, Redis, a linux kernel, and so much more for essentially running a Flask application. Now that I know redash fits my needs, I will spend the time needed to read the docker compose file and trace those steps and assemble the pieces necessary to run redash directly on my Linux server without Docker. Thanks for redash!
- jason_slack 8y agoI’m using timescale with 400gb of data. I don’t use web tools but let me throw out DBeaver if you need a solid database management tool: https://dbeaver.io https://dbeaver.io
- reacharavindh 8y agoJust curious. What do you do with your 400GB of TS data on Timescale? As in how do you query it? Are they only metrics or do you have textual data in there as well?
- jason_slack 8y agoI bring in stock data. I query it. I make tables to store models for future reference or historical purposes. I have tables for order tracking, metrics, statistics, etc. it’s all automated.
- sosedoff 8y agoIf you need a lightweight web UI for Postgres - have a look at https://sosedoff.github.io/pgweb/ https://sosedoff.github.io/pgweb/
- reacharavindh 8y agoThanks for this recommendation. I'm using pgweb as a general purpose SQL web interface to Postgres. But, I cannot easily adapt pgweb to bring out a search interface that only searches a certain column of table in postgres.
- sosedoff 8y agoNot sure what kind of issues you're having with search, but it's pretty straightforward to browse table data with a filter on a single column (be it numerical or text data). Im the author - shoot me an email (in profile) if you have any questions.
- pytyper2 8y agoIs there a SQL server management studio like application for postgres?
- 123repeater 8y agoI like datagrip
- h1d 8y agoTablePlus