10 ms·
Show HN: SQL Explorer – Open-source reporting tool that Just Works
I have been working on SQL Explorer, an open source, Django-based reporting and query tool for (gulp!) almost ten years. It's a tool that fits just right for me and many others, and I love and use almost every day. Write SQL, share results, do some analysis, get insight. No surprises.
A live demo instance is here (no login or anything required):
https://demo.sqlexplorer.io/ https://demo.sqlexplorer.io/
And here's a fairly unprofessional, but very enthusiastic, video tour:
https://sql-explorer.s3.amazonaws.com/Sql+Explorer+5.mp4 https://sql-explorer.s3.amazonaws.com/Sql+Explorer+5.mp4
The UI is constrained enough that there's very little to learn, while there is still a surprising amount of functionality and flexibility to address a lot of use cases.
Some of the stuff I'm excited about in the latest version:
- Intuitive and obvious integration to ChatGPT / the AI API of your choice. Doesn't purport to be 'magic'. Good prompting + relevant table scheme & data automatically injected into the prompt.
- Create a new connection by uploading a CSV or SQLite DB as a new connection, and it's instantly queryable. CSVs are parsed, types inferred, and a SQLite DB gets created (persisted to s3).
- New and improved SQL editor with strong autocomplete (based on your schema), and some fancy keyboard shortcuts.
Some of the old stuff that is still great:
- Pivot tables in-browser, so you don't have to open results in Excel for basic analysis. Unique URLs make everything shareable.
- Expose queries (optionally) as JSON endpoints. Great for prototyping APIs and scripts.
- All of the stuff you'd expect in a reporting tool (email reports, logging, favorites, exporting, etc.)
Hope you enjoy!
- bosky101 2y agoI spent 10 mins on your docs, website. Excellent effort overall. But I didn't know until I read your comment here about the uploading CSV, instant parsing that non technical people may find very interesting. This is something pgweb for example doesn't have. Your docs are also missing a complete sample env. See that you've integrated pivottable. Nice touch! If you can figure out minimal barcharts , you may even have an opensearch/log community interested. Another killer idea is uploading CSV/json and getting faceted search. No one does this! But maybe distracting to your roadmap. Keep up the excellent work! Good luck!
- numlocked 2y agoThanks so much! Can you expand on the idea -- what do you mean by faceted search? What would that look like?
- setr 2y agoI think GP is referring to something like excel’s filters, where the UI exposes filter options dynamically based on the data actually available after all existing filters have been applied.
- iAm25626 2y agohttps://docs.datasette.io/en/stable/facets.html https://docs.datasette.io/en/stable/facets.html (almost like a group by)
- crashabr 2y agoHave you looked at Datasette? It has faceted search by default. https://github.com/datasette https://github.com/datasette
- randlet 2y agoJust wanted to say thank you for all your effort on SQLExplorer, I incorporated it into our open source radiotherapy quality assurance project (https://qatrackplus.com https://qatrackplus.com) years ago and it's been a great addition and is used in hospitals around the world :)
- numlocked 2y agoThat's amazing! Thanks so much for sharing. Happy to chat if there are specific features or functionality that would be useful. Always looking for more feedback.
- rlawson 2y agoVery useful tool! I contributed a few features as a repayment for the ton of value I have gotten from it
- numlocked 2y agoAmazing! Thanks so much!
- joelhaasnoot 2y agoHey look, its Redash! https://github.com/getredash/redash https://github.com/getredash/redash
- rlawson 2y agoRedash is great and all but if you already have a Django app this is 10 min to have up and running inside that same app
- RyanHamilton 2y agoThat's a very harsh comment to give someone that has poured hours of their life into something trying to help others unpaid. You could at least politely ask what makes it different than redash.
- numlocked 2y agoPerhaps a bit impolite, but no offense taken. It's a very crowded space and there are a ton of good tools! I work on SQL Explorer simply because I get to make the thing that works best for me. Redash is very focused on visualization. SQL Explorer is not. It is going more in the direction of in-browser analysis.
- joelhaasnoot 2y agoDon't worry I was downvoted to oblivion for it! It wasn't meant snarky but I get that that's what it reads like.
- ds_opseeker 2y agoJust discovered this and got the test project up and running... but wondering how to enable CSV import? The "upload csv file" box does not show up in the test project.
- numlocked 2y agoYou got it! You'll need three values set in settings.py: https://github.com/explorerhq/django-sql-explorer/blob/64170743b46d310684982dfe745d65643c08e844/explorer/app_settings.py#L175 https://github.com/explorerhq/django-sql-explorer/blob/64170... def user_uploads_enabled(): return (EXPLORER_USER_UPLOADS_ENABLED and EXPLORER_DB_CONNECTIONS_ENABLED and S3_BUCKET is not None) It should be in the docs, but I'll make sure it's more prominent!
- tobilg 2y agoNice tool! I built https://sql-workbench.com/ https://sql-workbench.com/ which runs completely in-browser via DuckDB WASM, and enables querying of remote CSV, JSON, Parquet and Arrow data sources, as well as uploaded local files. Charts are supported as well, see the accompanying blog post https://tobilg.com/using-duckdb-wasm-for-in-browser-data-engineering https://tobilg.com/using-duckdb-wasm-for-in-browser-data-eng...
- antonvs 2y agoI like the tech stack. Does it provide anything like the query catalog in the OP, i.e. the table labeled "All Queries" on the OP demo page?
- tobilg 2y agoThere‘s a query history tab, not sure if this what you want? Otherwise, I‘m very open to feedback, just create an issue in the linked GH Repo. Thanks!
- antonvs 2y agoThe idea is that having created a query, chart etc., you can save it for other users to run.
- tobilg 2y agoYes, you can share a query, and also the visualization. See https://tobilg.com/using-duckdb-wasm-for-in-browser-data-engineering#heading-sharing-of-data-pipelines-amp-visualizations https://tobilg.com/using-duckdb-wasm-for-in-browser-data-eng...
- Omnipresent 2y agoIs this on GitHub to take a look at the code?
- tobilg 2y ago
- andix 2y agoThis is awesome, I hope I get a chance to use it once. One thought: I think the effort should be put into the UI for the non-technical end users, instead of query builders/developer experience. I would be even fine with a tool doesn't even have a query tool and just executes SQL files from a folder/git repo. The important part would be for me to provide a perfect experience for the end users. Developers usually have a lot of tools at hand to create queries, no need for another one.
- numlocked 2y agoYep - that makes sense. The Query pane can be collapsed, effectively hiding the SQL from the end user. This is indeed how a number of people use Explorer. But it could certainly be more optimized, in the direction you suggested. I'll think about how this might be improved!
- andix 2y agoIt was just a thought based on a quick look at the screenshots. I didn't use it yet. I would use a tool like that as a low-code platform to quickly make data accessible. Might be a different use case than most users are looking for.
- whalesalad 2y agoBeen thinking Metabase could benefit greatly from AI integration. This kinda does that!
- mritchie712 2y agoIf you're looking for AI in BI, I have something for you: https://www.definite.app/ https://www.definite.app/
- whalesalad 2y agonot open source, lowest cost plan is 1k per month. lol. (for up to 5 million rows ... dealbreaker right off the bat)
- nirav72 2y agoI've been self-hosting this https://github.com/dbgate/dbgate https://github.com/dbgate/dbgate for a few years. But I like some of the features in SQL Explorer interesting - like Pivot tables and exposing queries as JSON endpoints.
- psnehanshu 2y agoAwesome project. But a somewhat irrelevant suggestion. OP could have shared the video via YouTube for better user experience (adaptive bitrate streaming) and also not had to worry about paying for S3.
- majkinetor 2y agoI like the simplicity, and yet there is a lot of stuff to do. I know there are bunch of tools that do this (superset, redash, dbeaver web etc.) but there is a great value in the feature and UX choices of any particular tool. Keep it up m8.
- numlocked 2y agoCheers! Yes - lots of good options, and you said it well.
- lelo_tp 2y agoloved the demo video! we need more enthusiasm like that, really shows how much you love the work you've put in :)
- brunoqc 2y ago> * All content that resides under the "explorer/ee/" directory of this repository is licensed under the license defined in "explorer/ee/LICENSE". Not really open source. If you care about that.
- numlocked 2y agoIt’s absolutely open source - and completely free for commercial use. That license simply encumbers that specific functionality from resale. I don’t want anyone selling a SaaS version for profit (if someone wants to do that, they can contact me and we can talk about it).
- quectophoton 2y agoNothing wrong with making source-available or open core software. But saying it's open source when its license does not follow OSI's definition is confusing at best, or misleading at worst. OSI doesn't have a monopoly on the term, but that's the generally accepted definition (at least IME).
- 8n4vidtmkvmk 2y agoFwiw I thought open source meant the source was available. "FOSS" or "MIT" mean do whatever you want with it
- croes 2y agohttps://opensource.org/osd https://opensource.org/osd
- josalhor 2y agoDo you have a minimal docker image so we can start the service with a single command?
- numlocked 2y agoNope - but would love to do it. At the moment you can clone the repo and run start.sh which should work but obviously is not bulletproof like a docker image. Feedback and PRs welcome!
- maille 2y agoIntersting, can you generate fancy printable reports from this tool?
- numlocked 2y agoHmm, depends what you mean by 'fancy'. There is a 'full screen' view of any given report: https://demo.sqlexplorer.io/50/?fullscreen=1&rows=1000&querylog_id=5080 https://demo.sqlexplorer.io/50/?fullscreen=1&rows=1000&query... And it can do PDF export (with a plugin). Would love to hear what you have in mind?
- randyburden 2y agoGreat job! And congrats on actually getting something out there. I can really see this being useful for some organizations. I also envisioned this same type of tool around 10 years ago and it is still on my ever growing list of ideas to implement. I took the idea further to support not only SQL but other languages such as HTML, JavaScript, Python, C#, etc. You could then support returning different types of media based on the URL extension such as .html to return a webpage, .json to return a JSON API, .csv to return a CSV file, etc. As time marched on, many of these same ideas came to fruition in things like AWS Lambda, Jupiter Notebooks, Microsoft Monaco Editor, etc.
- greenchair 2y agoI'm curious about how the project name was chosen considering there are existing tools with the same name.
- b0ner_t0ner 2y agoGreat effort! Feels like you should be charging for this, maybe a Pro Plan with SSO (user edits integrated into your history logs) and 1 year support for SMEs to self-host.
- dhbradshaw 2y agoThis is really cool!
- nagstler 2y agoThanks for open sourcing, I’ll definitely give this a try! Any reporting tool is only as good as the data available to it. I built Multiwoven, a Reverse ETL with SQL capabilities, to sync data from any data warehouse to destinations like this one. https://github.com/Multiwoven/multiwoven https://github.com/Multiwoven/multiwoven
- loa_observer 2y agonice work, it would be better to add graphic walker for interactive data visualization (also include pivot table): https://github.com/Kanaries/graphic-walker https://github.com/Kanaries/graphic-walker
- JoeyBananas 2y agoI'm genuinely impressed by this. It doesn't surprise me that it took 10 years to make.