12 ms·

Show HN: Visual SQL

512 points by thingsilearned 7y ago

thingsilearned 7y ago
Dave, founder of Chartio here. We're so excited to launch what we call Visual SQL today. It's been a lot of work based on customer feedback and extensive prototyping and user testing. If you'd rather skip the story attached here you can also check out our product walkthrough video or give it a spin yourself here: https://chartio.com/product/visual-sql/ https://chartio.com/product/visual-sql/
- russell_h 7y ago
  Congratulation on the launch! I sit in a monthly "metrics review" meeting where we mostly speculate about why the dashboards are broken, and what the data team will have to do to fix it. If our TPM was able to self-service this through an intuitive interface it would be a massive productivity win.
  - BenS 7y ago
    Same here! Seeing how useful tableau (a far less powerful tool) has been for our company, it's clear that there is a big market for empowering everyone to dig for data-driven insights.
- rapind 7y ago
  It's not clear to me, but this looks like a SaaS offering and not a client I can run in my own environment? If that's the case, I'm sure there's a market, but definitely not something I could use.
  - thingsilearned 7y ago
    Yes, right now it's part of Chartio. I'd love to eventually open it outside of that, but there are no immediate plans.
    - rapind 7y ago
      Good to know. Thanks. Looks pretty sweet.
- yani 7y ago
  I love the product. I am sure most people don't want to allow direct connections to their databases. I haven't used the software so it might put a lot of load on my database that the server is not able to handle. It will be best if I can import a SQL dump and then use the software on a copy of my data. The alternative right now is to run a separate server just so I can try out the software.
wayneftw 7y ago
All of my other SQL tools are running right on my machine. I hope there will at least a self-host version of this (or something just like it) available. It would be amazing to have something like this embedded in Azure Data Studio.
- thingsilearned 7y ago
  Ah, unfortunately there isn't. We're 100% cloud hosted - though we call ourselves a Hybrid cloud solution because through our reverse SSH tunnel connections (where your servers SSH into ours) we also work well with on-prem data. If security is your main reason for wanting self-hosting you may be interested to know we're also SOC2, HIPAA, and GDPR compliant.
  - ulkesh 7y ago
    That's great you're compliant with those! For HIPAA, do you enter into a BAA with prospective customers who deal in HIPAA data (PHI)? EDIT>> The implication being that you may not be HIPAA compliant if you don't enter into a BAA. IANAL, so I defer to the experts, but I figured I'd ask the original question anyway. If you are above board on all of this, I will definitely take this tool to my boss.
    - ecpreston 7y ago
      yep, Chartio does enter BAAs with customers to deal with HIPAA data! I worked with several healthcare customers and lawyers here to make that happen. Happy to connect and share more.
      - ulkesh 7y ago
        Awesome, thanks for the info! I will gladly forward this up the chain to my boss, the first chance I get. I think this tool would be great for us, just had to be certain about that. I would love a self-hosting option, but I understand that your core business model might not allow for such an option.
navs 7y ago
Awesome app. I'm keen to give this a go. I'm currently running my data visualisations using python and plotly. Flexible but tedious sometimes.
mhink 7y ago
Over the years, we’ve found that even power users who know they’ll eventually go into SQL mode prefer to start in the visual mode, as the grouping, date formatting, and joins are all done automatically. This is huge, right here. I can use and understand SQL well enough, but the syntax annoys me constantly. Avoiding that would be a huge boon.
- thingsilearned 7y ago
  Yeah - changing date formats, double typing things you're grouping and ordering by, remembering the oddities of each dialect, typing out full join paths - it's a nice experience to have that done automatically. Right now the SQL we write is very proper - with quotes around all of the column names and table names listed before each column name. It's not what a human would write. I'd love to one day make it a little more human so that the switching into SQL mode will feel even better. It'll be a fun project.
  - infinite8s 7y ago
    What do you think of Looker's idea of turning SQL into a higher level analytic DSL (which can be written/implemented by data engineers)? This DSL then makes it easy for end users to do a more complicated analytics. One issue I've had with pure SQL for analytic purposes is that a user can't build a library of higher-order functions that can be parameterized (imagine a window function to do a cumulative returns across various time periods, applied to different underlying source columns), so its difficult to build up a common business domain language without taking the typical approach of constructing SQL through glueing string fragments together typical of most programming language SQL libraries.
vvern 7y ago
How does this product relate to https://www.sigmacomputing.com/ https://www.sigmacomputing.com/? It superficially feels very similar.
- corentin88 7y ago
  How does your hairdresser relate to yet another hairdresser? It’s a business. People have ideas and implement them in different ways. That’s just business.
- thingsilearned 7y ago
  Sigma and Chartio have similar missions but a very different product approach and functionality. Sigma also tries to bring the influence of spreadsheets into the query building/data exploration phase. We do this though as a very flexible visual language, where you can create a pipeline of actions including merging queries from multiple different datasets and then doing some post query computation.
ulkesh 7y ago
Can we self-host, please? Is there a standalone app? We deal in PHI (HIPAA) and we cannot have a tool like this have any access to our data store unless it's private and secure.
- yingw787 7y ago
  I think there are health-specific AWS clouds; are they strictly client-side, or is there a managed instance you can deploy from AWS marketplace? https://aws.amazon.com/health/healthcare-compliance/?nc=sn&loc=3&dn=2 https://aws.amazon.com/health/healthcare-compliance/?nc=sn&l...
  - ulkesh 7y ago
    We already have a live product using servers from a host that specializes in HIPAA. I'm not asking about how to host HIPAA-related data. I'm asking specifically about being able to use this tool. I'm not going to hand over credentials to our database to an untrusted party (meaning, not our company). So the hope is that we can either self-host this tool, or use it as a desktop application.
- klohto 7y ago
  They are HIPAA compliant
  - thingsilearned 7y ago
    Yes we are! No plans to go on-prem yet but security is obviously a giant focus and necessity https://chartio.com/product/security/ https://chartio.com/product/security/
    - ulkesh 7y ago
      Does that include signing a BAA? (I asked this in another thread on this post, sorry for double-posting, but I'm curious the answer).
duxup 7y ago
That is super cool. But as a sort of tangent / related thing: As a visual learner I don't feel like I usually have problems visualizing SQL (that doesn't mean I wouldn't have use for this product, quite the opposite). On the other hand I personally have been playing with Firebase and OH MAN do I have problems visualizing what is going on with document type databases, I start writing code and think of the DB and my brain starts sputter like there's sand in the gears ;)
- thingsilearned 7y ago
  To be clear - this isn't about making visualizations from SQL results (though that does happen). This was us making a visual query language. It's a very flexible interface that writes SQL for you - and has been proven to be intuitive enough for 80% of business users, so they can write SQL now too. The interface right now only works for structured data sources and my brain also starts to sputter when I think about making a visual language for anything not structured :).
methyl 7y ago
The pricing is a bit too high – if I'm reading correctly it starts at $200/month. It's hard to justify such a spending for a startup where utility of the tool is unknown and 14 days trial is not enough to evaluate that. It makes sense to charge that much power users, but to get involved I'd love to see a cheaper plan.
- thingsilearned 7y ago
  I appreciate the feedback. We are discussing a few different pricing models, and if your team size is below the 5 seat minimum you can reach out and we're quite flexible there.
- dwild 7y ago
  I'm not them, but a price too high can be a way to make sure to get only a few client at first. They clearly like to test stuff, and it's much easier to do it with a few client, than thousands of them. We also often forget the indirect cost of supporting a client, at my current job our software is great for small companies, but we found out that under 50 employees, it was just not worth it to support them. You can also always lower a price, nobody will complains for a lower price ;), but making it higher without justification is much harder. So all that to say, it can makes sense to start with a price way too high,
davidjhall 7y ago
>Please enter your business email address. This form does not accept addresses from yahoo.com. But my business address, right now, is @ yahoo.com Or, I don't want to give you my real business address yet because I don't want spam at that account. Or...
- minikomi 7y ago
  ... you might work at yahoo?
  - bmn__ 7y ago
    https://news.ycombinator.com/item?id=18395589 https://news.ycombinator.com/item?id=18395589 Then you would have a mail account @yahoo-inc.com
ta1234567890 7y ago
Looks like a modern version of Microsoft Access :)
- phkahler 7y ago
  I didn't see the visual joins that you can do in MS Access, though I just looked at the first page. If it's anywhere near that useful this product is way overdue.
  - thingsilearned 7y ago
    Yup - if you grab columns from multiple different connected tables (or schemas) the join will happen automatically for you and can be adjusted to different paths. This is actually a really hard problem, especially since 70% of datasources these days don't have foreign key information and we have to detect that automatically. We also have the ability to with our visual language join data from multiple different queries/datasources. You can see how that happens here https://chartio.com/docs/visual-sql/merge-queries/ https://chartio.com/docs/visual-sql/merge-queries/
- dgudkov 7y ago
  More like Power Query.
xrd 7y ago
Reminds me a lot of DabbleDB, acquired by Twitter long ago.
- ptman 7y ago
  airtable seems to be dabbledb du jour
oarabbus_ 7y ago
I would honestly much rather have a powerful IDE where I write my own sql and my query output is visualized than any of this UI stuff.
- byproxy 7y ago
  I think LINQPad does something similar? https://www.linqpad.net/ https://www.linqpad.net/
- thingsilearned 7y ago
  Yeah we're somewhat targeting a different audience here but I also think this will end up being a much better experience even for the power users than an IDE. I believe most data exploration just shouldn't be SQL/text based. It's just faster to do it visually if there aren't extra steps added and the interface is good/flexible enough. Even as developers we spend most of our time in a GUI environment vs the command line. It's not just that it's easier/better for business users - it's also just better for everyone.
  - nojvek 7y ago
    Really like what you’ve done Dave. I’m working on a very similar idea but a different niche (turning databases into a cms + simple charts) at https://boomadmin.com https://boomadmin.com Totally the future is hybrid. Quickly press some buttons to get to a starting SQL and then go complex from there on if needed. My principle is that “simple things should be simple, complex things are possible”
simjue 7y ago
Congratulations, looks like a really nice product! As many people here ask for a self-hosted solution, Apache Drill and SuperSet might be an alternative.
- thingsilearned 7y ago
  Yes we do require a work email to signup as our target audiences are businesses. We've discussed launching a public version which I'd love to have sometime soon.
trts 7y ago
I've seen a lot of products similar to this, and it looks like one of the nicer ones. But the use case in the video and screenshots make me immediately think it wouldn't be viable in any of the organizations I've written SQL for in the past 10 years. At least 95% of the challenge in querying data doesn't come from the difficulty of writing SQL statements IMO, but from the complexity/brokenness of the data itself. Especially with the move to unstructured data streams, most data warehouses seem to have become pretty bad when it comes to extracting truth out of the information within. Many business users know some SQL but blanch at creating coherent reports from what's available to them. What BI people are paid for seems to be having knowledge about the problems particular to a domain. If your data is relatively clean and follows a good model, this would be a great way to help someone join, group, and aggregate their data without knowing SQL. I think a lot of people would use Tableau for this if it didn't also have a steep learning curve / high expense. However as soon as you get into data where you're writing case statements, coalescing null fields, matching on different data types, decoding, partitioning over streaming data etc., it doesn't help someone without knowledge of the caveats within the data sources themselves. Show me someone who doesn't know SQL using this to produce insight out of compromised data and I will be impressed.
- thingsilearned 7y ago
  I fully agree with your experience here. We have a super hard mission at Chartio - it's not just about the interface but also how the data is setup. The interface, being as flexible as it is and also enabling full schema (instead of dataset) browsing is a pretty big part though in also allowing a more agile version of data modeling. It had that very much in mind and we've written a book (soon to be published with Wiley) on proper modern data governance techniques. https://dataschool.com/data-governance/ https://dataschool.com/data-governance/ Our next phase is to help people get to that cleaner source of truth much more quickly than traditional dimensional modeling approaches. Tools like Visual SQL and DBT (https://www.getdbt.com https://www.getdbt.com) are really changing the complexities here.
  - jeanloolz 7y ago
    I would love to be involved with that book. I'm a data engineer myself and I have built SQLBucket, a python library I have been told is similar to getdbt.com (although I'm not familiar with DBT, the similarities have been mentioned to me on various occasions). Shameless plug: http://github.com/socialpoint-labs/sqlbucket http://github.com/socialpoint-labs/sqlbucket
solarkraft 7y ago
The page stopped my media playback (which happens when another piece of media is played). Why?
- thingsilearned 7y ago
  No idea. Thanks for reporting. We'll look into it
  - shrew 7y ago
    I think it's because your looped demos are autoplaying videos. Modern browsers can control media playback, and presumably these autoplaying videos are hijacking the currently played media for some people? As an aside, I was actually looking for a comment about the opposite issue: I have a browser which blocks autoplaying videos, it took me a while to realise that what I thought were static images, were in fact videos without playback controls (and a little bit longer to work out how to re-enable playback controls!). Either way, once I got them playing, it looked like a slick experience, congrats on the launch!
punnerud 7y ago
Is there a self hosting solution an engineer can test for free? I love Metabase, but this solutions seems worth a try. The comparison with AWS/Google Cloud on being SaaS is not viable because there is almost always (?) an engineer testing out the software at the workplace, that’s why it have to be free to test and OnPrem.
tmpz22 7y ago
It's always a tug of war between "power-user features" and "simplified-interfaces for common users". It seems like Chartio is using visual interfaces to bridge this very gap and it seems like a very polished outcome. I think the elephant in the room though is that this is a technical solution to a non-technical problem. You can't teach all users to be highly proficient in your app through a perfect interface - because the perfect interface does not exist. You teach them through traditional training methods and continuous training investments and infrastructure. Going through their landing pages it seems like Chartio is offering support and training packages, so there's definitely nothing wrong with them supplementing that with better visualization tools. In general however I just wish more companies would invest in the training layer earlier and not try to bail themselves out with the perfect redesign - cough reddit cough.
- thingsilearned 7y ago
  Totally - it's not a solution to everything. We can though do as much as possible making the UI intuitive, flexible and fast. Those gains are not just for business users but for everyone. Every product needs some education (excel is still of the most popular online courses) and we knew ours would be no exception which is why we've also last year launched DataSchool - our free online community driven courses on data https://dataschool.com https://dataschool.com
mstade 7y ago
Looks nice, but like others here we need something that can be self hosted or even better – run offline. Somewhat related: does anyone know of any good components/libraries that can be embedded in a web page to let users interactively build SQL queries of reasonable complexity? By that I mean something that not only lets you pick columns and filter and maybe group things, but also do joins on other tables as well. Edit: just to clarify, it doesn't have to be be visual only. In fact, something that lets you edit a query interactively either by visual components or actual SQL code is perfect, especially if it's bidirectional so that edits in the code also show in the UI and vice versa.
- danpelota 7y ago
  I don't know about embedding, but I've been happy with the Metabase UI for building these kinds of queries: https://www.metabase.com/docs/latest/users-guide/custom-questions.html https://www.metabase.com/docs/latest/users-guide/custom-ques...
- hermitcrab 7y ago
  If you want an offline, no code, visual data transformation tool, please check out our new product: https://www.easydatatransform.com/ https://www.easydatatransform.com/
  - mstade 7y ago
    Thank you, I'll take a look! Just want to clarify – it doesn't have to be no code. In fact, a bidirectional workflow where you can edit the query either by writing SQL or clicking buttons is ideal.
- texodus 7y ago
  https://perspective.finos.org/ https://perspective.finos.org/ supports simple SQL-like operations, group by, filters and sort e.g.
- nojvek 7y ago
  Thanks for this comment. I am working on something very close to this vision at https://boomadmin.com https://boomadmin.com The idea is to have an open source (but commercial use paid license) that people can self host and totally have it offline or have it hosted on cloud with a single click. I quit my job last month to work on it fulltime to work on an MVP. If you’re interested please add your email for an early beta. I would love your feedback.
graphememes 7y ago
I'm not going to lie, this really hurts my brain to try and wrap around, it's a lot of cognitive overload in one place.
- thingsilearned 7y ago
  Ha, sorry about that! Are you talking about the interface, the idea, or the content?!
  - graphememes 7y ago
    The interface, it's very dense
aerovistae 7y ago
It would reallllly be nice if you set up a web demo. I looked all over trying to find the link that would let me just TRY this marvelous thing you're telling me about, but it's all behind sign-ups and accounts and I just want to see the product. Not in a video-- I mean play with it. Can you make a demo session with dummy data for people with play with?
- thingsilearned 7y ago
  Ah yeah, you do have to signup to try it out (though it does start you off with demo data). We wanted to offer a non-login version but it makes things more complicated for a company that obviously needs to prioritize security highly. This video is maybe the closest you can get without trying yourself - https://www.youtube.com/watch?v=YBXMTipHGfQ https://www.youtube.com/watch?v=YBXMTipHGfQ
  - eterm 7y ago
    Is the security story why you prevent signups from gmail.com also?
    - thingsilearned 7y ago
      No, sorry that's a marketing/sales team decision. I'll talk with them about it.
      - aloisdg 7y ago
        Why would they want to ban them?
        nujabe 7y ago
        They want users to signup with their company email. I think a few companies in the enterprise software space do something similar.
- SamuelAdams 7y ago
stephen82 7y ago
For some reason, it reminded me of Kirix Strata [1]...well, it feels like it in a way. https://www.kirix.com/
exabrial 7y ago
If you have a solid data model and team or person that is charge of preventing people from implementing bad ideas, it becomes trivial for people to do data discovery on their own. The problem is many orgs have pushed far to the opposite direction, unstructured, undocumented, unspecified data structure. While this is better than "no data" is puts tremendous burden on the user to interpret it later. As the old rule says: crap in crap out
gklitt 7y ago
An awesome research project in this area: https://people.csail.mit.edu/ebakke/sieuferd/ https://people.csail.mit.edu/ebakke/sieuferd/
pbreit 7y ago
Looks interesting. Alas, no support for VPNing into DB. Like what mode.com is doing with its "bridge": https://mode.com/help/articles/how-mode-connects/#bridge https://mode.com/help/articles/how-mode-connects/#bridge
- thingsilearned 7y ago
  We do have a reverse SSH tunnel connection https://chartio.com/docs/data-sources/tunnel-connection/ https://chartio.com/docs/data-sources/tunnel-connection/
  - pbreit 7y ago
    If I understand, it involves changes to the DB server? That would be a show-stopper for me.
    - MechanicalTwerk 7y ago
      Haven't used Chartio in a while but when I did the reverse ssh tunnel was just a standard reverse ssh that you could deploy on any server in your vpn. It doesn't have to be on the actual db server, it could be on its own server acting as a bridge between Chartio and your db. Also, you could use whatever ssh client you wanted - you didn't have to install anything from Chartio in your vpn.
      - pbreit 7y ago
        This made me think id need to do something on the DB server which I cannot: “On your database server, run the following to install autossh”
krat0sprakhar 7y ago
I have some data in BigQuery (for some personal project) for which I'd love to use Chartio and the visual SQL builder. The sign up page just shows company login - is the product not intended for standalone consumers?
deleted 7y ago
[deleted]
figital 7y ago
I typed in a basic text query to "count reps per account". Then I clicked the visualize button ... no re-visualization. This is an extremely basic feature in every visual query tool the past 20 years (at least the ones worth paying for). You would be limiting yourself to their canned charts and "demo magic".
deleted 7y ago
[deleted]
didip 7y ago
If the alerting story is amazing, then this thing would be very compelling. Nowadays people use an amalgamation of random tools: Grafana, Redash, Superset, Tableau, etc. But they all have rudimentary alerting or none at all. Self hosting is another feature that is important. Enterprise customers simply won't buy if it's SaaS only.
samokhvalov 7y ago
Looks good! Worth noting, the idea of "visual SQL" appeared long, long ago, it was called "Query by Example" https://en.wikipedia.org/wiki/Query_by_Example https://en.wikipedia.org/wiki/Query_by_Example. Congratulation on the launch and thank you for having PostgreSQL support. (Side-question: any reasons why you have mirrored Postgres logo?)
- thingsilearned 7y ago
  Thanks! Hadn't noticed that the icon was mirrored, thanks for pointing it out!
ken 7y ago
As someone who used to work on ETL systems, from a distance, this looks basically like Tableau to me. (My first thought was actually "oh, neat, someone made an open-source Tableau clone!", but it's not open-source, is it?) You should describe how it's different. If I'm going to pay money for a hosted application for serious work, I'm going to tend to prefer an established service that I'm confident will be around for a while. I'm sure you've got something good here but I'm not sure what it is.
- thingsilearned 7y ago
  We do compete very directly with Tableau and are often used along side it as well. In the article I explain a lot of what's different about it from other BI products like Tableau. The main things are - it's way more flexible as a visual language, and through a lot of prototypes and user tests has shown 80% of business users can write significant queries on their first try. As for an established service - Chartio's been around for almost 10 years, we're profitable, and are the main data interface behind some really great companies and brands https://chartio.com/customers/ https://chartio.com/customers/
longtermd 7y ago
Postgres' pgAdmin software is still a huge pain for many operations, especially for creating functions and views. I believe they could learn a lot from this.
- grenoire 7y ago
  To me, pgAdmin seemed like a very light header on the administrative (hah) end of pg. Checking which instances are running, and seeing if the databases are healthy etc.
techolic 7y ago
For some reason I read 'Viral SQL', these days...
ComodoHacker 7y ago
Signup to try — no, thanks.
- boublepop 7y ago
  Seems like a fair trade, you get to try out for free as long as you let them know that you are interesting in trying it out. They are aiming for business users who don’t mind dropping an e-mail for a quick eval.
larodi 7y ago
No visual has beaten SQL for the last 40 years
- bmn__ 7y ago
  It's not a competition where one solution has to beat the other. Querying through a textual language and through a visual interface complement each other. I consider a product to be a success when it combines the best features of both and makes them available to the user so that he may organise his work flow according to the own preferred style.
  - larodi 7y ago
    That is true, you are right. What I'm saying is that eventually things that need be SQL are either SQL... or sometimes R. The language is more expressive and it takes less time to 'select something from something where conditions group by something', rather than drag and drop it. and is the reason why SQL is so fundamental to all areas of information technology and particularly when data is involved. besides, SQL is DATALOG which is PROLOG which is to 'program logic', which gets as close as it can to 'wiring' logic. many attempts to visualize this process has failed so far, because... they are more complex and more specific. SQL takes like 30 chars and dozen keywords to master. any visual paradigm - takes more.
stiray 7y ago
Just a detail. Have you seen Microsoft Access query builder? Please do check it, especially how you create joins. As far as I have seen tools for building queries none came close. Maybe sqlyog. https://www.youtube.com/watch?v=pcjXQqKWWNw https://www.youtube.com/watch?v=pcjXQqKWWNw
mjirv 7y ago
I like the user-research-first approach here a lot. Very different than many other BI tools in a refreshing way. In the past I've hesitated to consider Chartio for my organization because most people don't know SQL, but I'm excited to give it a try now. I also love how everything is a table. Looker does this, and it's great (though Chartio's implementation looks better). Tableau does not, which makes certain things a headache. I do wonder how well this will scale to large datasets and columnar databases, though. Showing a preview of the tables is really cool and useful, but it is not what data warehouses like Redshift are designed for. The preview probably wouldn't load in time for my users to see it unless they just sit there waiting for it.
- thingsilearned 7y ago
  Thanks, the user-research-first part was definitely enlightening and important to do. Our first many iterations were very wrong. As for large scale datasets - we do cache those sample tables and we only grab the first 10 rows. Also, BigQuery actually has a great API for fetching a set of sample data that we utilize heavily. They made it because of exactly the potential concerns you outlined around columnar stores.
  - mjirv 7y ago
    Oh neat, I didn't know that about BigQuery. Thanks for the tip!
BLanen 7y ago
I made this in an internship except that I get the metadata from Entity Framework and those subqueries but those woudn't have been hard to implement.
rco8786 7y ago
Seems reasonably interesting, but I can’t get past the claim that they made 21 figma boards every day, including weekends, for 3 months.
johnthescott 7y ago
"sql easy, data is hard". perfectly written. why even use sql if you are simply looking for patterns in static data? why not use an app dedicated to analytics, of which many exist? sql comes into play when data needs to be shared, and those types of apps need coders who would not struggle with sql. i simply do not buy the argument that sql is difficult to learn for competent coders.

Dave, founder of Chartio here. We're so excited to launch what we call Visual SQL today. It's been a lot of work based on customer feedback and extensive prototyping and user testing. If you'd rather skip the story attached here you can also check out our product walkthrough video or give it a spin yourself here: https://chartio.com/product/visual-sql/ https://chartio.com/product/visual-sql/

russell_h 7y ago
Congratulation on the launch! I sit in a monthly "metrics review" meeting where we mostly speculate about why the dashboards are broken, and what the data team will have to do to fix it. If our TPM was able to self-service this through an intuitive interface it would be a massive productivity win.
- BenS 7y ago
  Same here! Seeing how useful tableau (a far less powerful tool) has been for our company, it's clear that there is a big market for empowering everyone to dig for data-driven insights.
rapind 7y ago
It's not clear to me, but this looks like a SaaS offering and not a client I can run in my own environment? If that's the case, I'm sure there's a market, but definitely not something I could use.
- thingsilearned 7y ago
  Yes, right now it's part of Chartio. I'd love to eventually open it outside of that, but there are no immediate plans.
  - rapind 7y ago
    Good to know. Thanks. Looks pretty sweet.
yani 7y ago
I love the product. I am sure most people don't want to allow direct connections to their databases. I haven't used the software so it might put a lot of load on my database that the server is not able to handle. It will be best if I can import a SQL dump and then use the software on a copy of my data. The alternative right now is to run a separate server just so I can try out the software.

wayneftw 7y ago

All of my other SQL tools are running right on my machine. I hope there will at least a self-host version of this (or something just like it) available. It would be amazing to have something like this embedded in Azure Data Studio.

thingsilearned 7y ago
Ah, unfortunately there isn't. We're 100% cloud hosted - though we call ourselves a Hybrid cloud solution because through our reverse SSH tunnel connections (where your servers SSH into ours) we also work well with on-prem data. If security is your main reason for wanting self-hosting you may be interested to know we're also SOC2, HIPAA, and GDPR compliant.
- ulkesh 7y ago
  That's great you're compliant with those! For HIPAA, do you enter into a BAA with prospective customers who deal in HIPAA data (PHI)? EDIT>> The implication being that you may not be HIPAA compliant if you don't enter into a BAA. IANAL, so I defer to the experts, but I figured I'd ask the original question anyway. If you are above board on all of this, I will definitely take this tool to my boss.
  - ecpreston 7y ago
    yep, Chartio does enter BAAs with customers to deal with HIPAA data! I worked with several healthcare customers and lawyers here to make that happen. Happy to connect and share more.
    - ulkesh 7y ago
      Awesome, thanks for the info! I will gladly forward this up the chain to my boss, the first chance I get. I think this tool would be great for us, just had to be certain about that. I would love a self-hosting option, but I understand that your core business model might not allow for such an option.

navs 7y ago

Awesome app. I'm keen to give this a go. I'm currently running my data visualisations using python and plotly. Flexible but tedious sometimes.

mhink 7y ago

Over the years, we’ve found that even power users who know they’ll eventually go into SQL mode prefer to start in the visual mode, as the grouping, date formatting, and joins are all done automatically. This is huge, right here. I can use and understand SQL well enough, but the syntax annoys me constantly. Avoiding that would be a huge boon.

thingsilearned 7y ago
Yeah - changing date formats, double typing things you're grouping and ordering by, remembering the oddities of each dialect, typing out full join paths - it's a nice experience to have that done automatically. Right now the SQL we write is very proper - with quotes around all of the column names and table names listed before each column name. It's not what a human would write. I'd love to one day make it a little more human so that the switching into SQL mode will feel even better. It'll be a fun project.
- infinite8s 7y ago
  What do you think of Looker's idea of turning SQL into a higher level analytic DSL (which can be written/implemented by data engineers)? This DSL then makes it easy for end users to do a more complicated analytics. One issue I've had with pure SQL for analytic purposes is that a user can't build a library of higher-order functions that can be parameterized (imagine a window function to do a cumulative returns across various time periods, applied to different underlying source columns), so its difficult to build up a common business domain language without taking the typical approach of constructing SQL through glueing string fragments together typical of most programming language SQL libraries.

vvern 7y ago

How does this product relate to https://www.sigmacomputing.com/ https://www.sigmacomputing.com/? It superficially feels very similar.

corentin88 7y ago
How does your hairdresser relate to yet another hairdresser? It’s a business. People have ideas and implement them in different ways. That’s just business.
thingsilearned 7y ago
Sigma and Chartio have similar missions but a very different product approach and functionality. Sigma also tries to bring the influence of spreadsheets into the query building/data exploration phase. We do this though as a very flexible visual language, where you can create a pipeline of actions including merging queries from multiple different datasets and then doing some post query computation.

ulkesh 7y ago

Can we self-host, please? Is there a standalone app? We deal in PHI (HIPAA) and we cannot have a tool like this have any access to our data store unless it's private and secure.

yingw787 7y ago
I think there are health-specific AWS clouds; are they strictly client-side, or is there a managed instance you can deploy from AWS marketplace? https://aws.amazon.com/health/healthcare-compliance/?nc=sn&loc=3&dn=2 https://aws.amazon.com/health/healthcare-compliance/?nc=sn&l...
- ulkesh 7y ago
  We already have a live product using servers from a host that specializes in HIPAA. I'm not asking about how to host HIPAA-related data. I'm asking specifically about being able to use this tool. I'm not going to hand over credentials to our database to an untrusted party (meaning, not our company). So the hope is that we can either self-host this tool, or use it as a desktop application.
klohto 7y ago
They are HIPAA compliant
- thingsilearned 7y ago
  Yes we are! No plans to go on-prem yet but security is obviously a giant focus and necessity https://chartio.com/product/security/ https://chartio.com/product/security/
  - ulkesh 7y ago
    Does that include signing a BAA? (I asked this in another thread on this post, sorry for double-posting, but I'm curious the answer).

duxup 7y ago

That is super cool. But as a sort of tangent / related thing: As a visual learner I don't feel like I usually have problems visualizing SQL (that doesn't mean I wouldn't have use for this product, quite the opposite). On the other hand I personally have been playing with Firebase and OH MAN do I have problems visualizing what is going on with document type databases, I start writing code and think of the DB and my brain starts sputter like there's sand in the gears ;)

thingsilearned 7y ago
To be clear - this isn't about making visualizations from SQL results (though that does happen). This was us making a visual query language. It's a very flexible interface that writes SQL for you - and has been proven to be intuitive enough for 80% of business users, so they can write SQL now too. The interface right now only works for structured data sources and my brain also starts to sputter when I think about making a visual language for anything not structured :).

methyl 7y ago

The pricing is a bit too high – if I'm reading correctly it starts at $200/month. It's hard to justify such a spending for a startup where utility of the tool is unknown and 14 days trial is not enough to evaluate that. It makes sense to charge that much power users, but to get involved I'd love to see a cheaper plan.

thingsilearned 7y ago
I appreciate the feedback. We are discussing a few different pricing models, and if your team size is below the 5 seat minimum you can reach out and we're quite flexible there.
dwild 7y ago
I'm not them, but a price too high can be a way to make sure to get only a few client at first. They clearly like to test stuff, and it's much easier to do it with a few client, than thousands of them. We also often forget the indirect cost of supporting a client, at my current job our software is great for small companies, but we found out that under 50 employees, it was just not worth it to support them. You can also always lower a price, nobody will complains for a lower price ;), but making it higher without justification is much harder. So all that to say, it can makes sense to start with a price way too high,

davidjhall 7y ago

>Please enter your business email address. This form does not accept addresses from yahoo.com. But my business address, right now, is @ yahoo.com Or, I don't want to give you my real business address yet because I don't want spam at that account. Or...

minikomi 7y ago
... you might work at yahoo?
- bmn__ 7y ago
  https://news.ycombinator.com/item?id=18395589 https://news.ycombinator.com/item?id=18395589 Then you would have a mail account @yahoo-inc.com

ta1234567890 7y ago

Looks like a modern version of Microsoft Access :)

phkahler 7y ago
I didn't see the visual joins that you can do in MS Access, though I just looked at the first page. If it's anywhere near that useful this product is way overdue.
- thingsilearned 7y ago
  Yup - if you grab columns from multiple different connected tables (or schemas) the join will happen automatically for you and can be adjusted to different paths. This is actually a really hard problem, especially since 70% of datasources these days don't have foreign key information and we have to detect that automatically. We also have the ability to with our visual language join data from multiple different queries/datasources. You can see how that happens here https://chartio.com/docs/visual-sql/merge-queries/ https://chartio.com/docs/visual-sql/merge-queries/
dgudkov 7y ago
More like Power Query.

xrd 7y ago

Reminds me a lot of DabbleDB, acquired by Twitter long ago.

ptman 7y ago
airtable seems to be dabbledb du jour

oarabbus_ 7y ago

I would honestly much rather have a powerful IDE where I write my own sql and my query output is visualized than any of this UI stuff.

byproxy 7y ago
I think LINQPad does something similar? https://www.linqpad.net/ https://www.linqpad.net/
thingsilearned 7y ago
Yeah we're somewhat targeting a different audience here but I also think this will end up being a much better experience even for the power users than an IDE. I believe most data exploration just shouldn't be SQL/text based. It's just faster to do it visually if there aren't extra steps added and the interface is good/flexible enough. Even as developers we spend most of our time in a GUI environment vs the command line. It's not just that it's easier/better for business users - it's also just better for everyone.
- nojvek 7y ago
  Really like what you’ve done Dave. I’m working on a very similar idea but a different niche (turning databases into a cms + simple charts) at https://boomadmin.com https://boomadmin.com Totally the future is hybrid. Quickly press some buttons to get to a starting SQL and then go complex from there on if needed. My principle is that “simple things should be simple, complex things are possible”

simjue 7y ago

Congratulations, looks like a really nice product! As many people here ask for a self-hosted solution, Apache Drill and SuperSet might be an alternative.

thingsilearned 7y ago
Yes we do require a work email to signup as our target audiences are businesses. We've discussed launching a public version which I'd love to have sometime soon.

trts 7y ago

I've seen a lot of products similar to this, and it looks like one of the nicer ones. But the use case in the video and screenshots make me immediately think it wouldn't be viable in any of the organizations I've written SQL for in the past 10 years. At least 95% of the challenge in querying data doesn't come from the difficulty of writing SQL statements IMO, but from the complexity/brokenness of the data itself. Especially with the move to unstructured data streams, most data warehouses seem to have become pretty bad when it comes to extracting truth out of the information within. Many business users know some SQL but blanch at creating coherent reports from what's available to them. What BI people are paid for seems to be having knowledge about the problems particular to a domain. If your data is relatively clean and follows a good model, this would be a great way to help someone join, group, and aggregate their data without knowing SQL. I think a lot of people would use Tableau for this if it didn't also have a steep learning curve / high expense. However as soon as you get into data where you're writing case statements, coalescing null fields, matching on different data types, decoding, partitioning over streaming data etc., it doesn't help someone without knowledge of the caveats within the data sources themselves. Show me someone who doesn't know SQL using this to produce insight out of compromised data and I will be impressed.

thingsilearned 7y ago
I fully agree with your experience here. We have a super hard mission at Chartio - it's not just about the interface but also how the data is setup. The interface, being as flexible as it is and also enabling full schema (instead of dataset) browsing is a pretty big part though in also allowing a more agile version of data modeling. It had that very much in mind and we've written a book (soon to be published with Wiley) on proper modern data governance techniques. https://dataschool.com/data-governance/ https://dataschool.com/data-governance/ Our next phase is to help people get to that cleaner source of truth much more quickly than traditional dimensional modeling approaches. Tools like Visual SQL and DBT (https://www.getdbt.com https://www.getdbt.com) are really changing the complexities here.
- jeanloolz 7y ago
  I would love to be involved with that book. I'm a data engineer myself and I have built SQLBucket, a python library I have been told is similar to getdbt.com (although I'm not familiar with DBT, the similarities have been mentioned to me on various occasions). Shameless plug: http://github.com/socialpoint-labs/sqlbucket http://github.com/socialpoint-labs/sqlbucket

solarkraft 7y ago

The page stopped my media playback (which happens when another piece of media is played). Why?

thingsilearned 7y ago
No idea. Thanks for reporting. We'll look into it
- shrew 7y ago
  I think it's because your looped demos are autoplaying videos. Modern browsers can control media playback, and presumably these autoplaying videos are hijacking the currently played media for some people? As an aside, I was actually looking for a comment about the opposite issue: I have a browser which blocks autoplaying videos, it took me a while to realise that what I thought were static images, were in fact videos without playback controls (and a little bit longer to work out how to re-enable playback controls!). Either way, once I got them playing, it looked like a slick experience, congrats on the launch!

punnerud 7y ago

Is there a self hosting solution an engineer can test for free? I love Metabase, but this solutions seems worth a try. The comparison with AWS/Google Cloud on being SaaS is not viable because there is almost always (?) an engineer testing out the software at the workplace, that’s why it have to be free to test and OnPrem.

tmpz22 7y ago

It's always a tug of war between "power-user features" and "simplified-interfaces for common users". It seems like Chartio is using visual interfaces to bridge this very gap and it seems like a very polished outcome. I think the elephant in the room though is that this is a technical solution to a non-technical problem. You can't teach all users to be highly proficient in your app through a perfect interface - because the perfect interface does not exist. You teach them through traditional training methods and continuous training investments and infrastructure. Going through their landing pages it seems like Chartio is offering support and training packages, so there's definitely nothing wrong with them supplementing that with better visualization tools. In general however I just wish more companies would invest in the training layer earlier and not try to bail themselves out with the perfect redesign - cough reddit cough.

thingsilearned 7y ago
Totally - it's not a solution to everything. We can though do as much as possible making the UI intuitive, flexible and fast. Those gains are not just for business users but for everyone. Every product needs some education (excel is still of the most popular online courses) and we knew ours would be no exception which is why we've also last year launched DataSchool - our free online community driven courses on data https://dataschool.com https://dataschool.com

mstade 7y ago

Looks nice, but like others here we need something that can be self hosted or even better – run offline. Somewhat related: does anyone know of any good components/libraries that can be embedded in a web page to let users interactively build SQL queries of reasonable complexity? By that I mean something that not only lets you pick columns and filter and maybe group things, but also do joins on other tables as well. Edit: just to clarify, it doesn't have to be be visual only. In fact, something that lets you edit a query interactively either by visual components or actual SQL code is perfect, especially if it's bidirectional so that edits in the code also show in the UI and vice versa.

danpelota 7y ago
I don't know about embedding, but I've been happy with the Metabase UI for building these kinds of queries: https://www.metabase.com/docs/latest/users-guide/custom-questions.html https://www.metabase.com/docs/latest/users-guide/custom-ques...
hermitcrab 7y ago
If you want an offline, no code, visual data transformation tool, please check out our new product: https://www.easydatatransform.com/ https://www.easydatatransform.com/
- mstade 7y ago
  Thank you, I'll take a look! Just want to clarify – it doesn't have to be no code. In fact, a bidirectional workflow where you can edit the query either by writing SQL or clicking buttons is ideal.
texodus 7y ago
https://perspective.finos.org/ https://perspective.finos.org/ supports simple SQL-like operations, group by, filters and sort e.g.
nojvek 7y ago
Thanks for this comment. I am working on something very close to this vision at https://boomadmin.com https://boomadmin.com The idea is to have an open source (but commercial use paid license) that people can self host and totally have it offline or have it hosted on cloud with a single click. I quit my job last month to work on it fulltime to work on an MVP. If you’re interested please add your email for an early beta. I would love your feedback.

graphememes 7y ago

I'm not going to lie, this really hurts my brain to try and wrap around, it's a lot of cognitive overload in one place.

thingsilearned 7y ago
Ha, sorry about that! Are you talking about the interface, the idea, or the content?!
- graphememes 7y ago
  The interface, it's very dense

aerovistae 7y ago

It would reallllly be nice if you set up a web demo. I looked all over trying to find the link that would let me just TRY this marvelous thing you're telling me about, but it's all behind sign-ups and accounts and I just want to see the product. Not in a video-- I mean play with it. Can you make a demo session with dummy data for people with play with?

thingsilearned 7y ago
Ah yeah, you do have to signup to try it out (though it does start you off with demo data). We wanted to offer a non-login version but it makes things more complicated for a company that obviously needs to prioritize security highly. This video is maybe the closest you can get without trying yourself - https://www.youtube.com/watch?v=YBXMTipHGfQ https://www.youtube.com/watch?v=YBXMTipHGfQ
- eterm 7y ago
  Is the security story why you prevent signups from gmail.com also?
  - thingsilearned 7y ago
    No, sorry that's a marketing/sales team decision. I'll talk with them about it.
    - aloisdg 7y ago
      Why would they want to ban them?
      - nujabe 7y ago
        They want users to signup with their company email. I think a few companies in the enterprise software space do something similar.
SamuelAdams 7y ago

stephen82 7y ago

For some reason, it reminded me of Kirix Strata [1]...well, it feels like it in a way. https://www.kirix.com/

exabrial 7y ago

If you have a solid data model and team or person that is charge of preventing people from implementing bad ideas, it becomes trivial for people to do data discovery on their own. The problem is many orgs have pushed far to the opposite direction, unstructured, undocumented, unspecified data structure. While this is better than "no data" is puts tremendous burden on the user to interpret it later. As the old rule says: crap in crap out

gklitt 7y ago

An awesome research project in this area: https://people.csail.mit.edu/ebakke/sieuferd/ https://people.csail.mit.edu/ebakke/sieuferd/

pbreit 7y ago

Looks interesting. Alas, no support for VPNing into DB. Like what mode.com is doing with its "bridge": https://mode.com/help/articles/how-mode-connects/#bridge https://mode.com/help/articles/how-mode-connects/#bridge

thingsilearned 7y ago
We do have a reverse SSH tunnel connection https://chartio.com/docs/data-sources/tunnel-connection/ https://chartio.com/docs/data-sources/tunnel-connection/
- pbreit 7y ago
  If I understand, it involves changes to the DB server? That would be a show-stopper for me.
  - MechanicalTwerk 7y ago
    Haven't used Chartio in a while but when I did the reverse ssh tunnel was just a standard reverse ssh that you could deploy on any server in your vpn. It doesn't have to be on the actual db server, it could be on its own server acting as a bridge between Chartio and your db. Also, you could use whatever ssh client you wanted - you didn't have to install anything from Chartio in your vpn.
    - pbreit 7y ago
      This made me think id need to do something on the DB server which I cannot: “On your database server, run the following to install autossh”

krat0sprakhar 7y ago

I have some data in BigQuery (for some personal project) for which I'd love to use Chartio and the visual SQL builder. The sign up page just shows company login - is the product not intended for standalone consumers?

deleted 7y ago

[deleted]

figital 7y ago

I typed in a basic text query to "count reps per account". Then I clicked the visualize button ... no re-visualization. This is an extremely basic feature in every visual query tool the past 20 years (at least the ones worth paying for). You would be limiting yourself to their canned charts and "demo magic".

deleted 7y ago

[deleted]

didip 7y ago

If the alerting story is amazing, then this thing would be very compelling. Nowadays people use an amalgamation of random tools: Grafana, Redash, Superset, Tableau, etc. But they all have rudimentary alerting or none at all. Self hosting is another feature that is important. Enterprise customers simply won't buy if it's SaaS only.

samokhvalov 7y ago

Looks good! Worth noting, the idea of "visual SQL" appeared long, long ago, it was called "Query by Example" https://en.wikipedia.org/wiki/Query_by_Example https://en.wikipedia.org/wiki/Query_by_Example. Congratulation on the launch and thank you for having PostgreSQL support. (Side-question: any reasons why you have mirrored Postgres logo?)

thingsilearned 7y ago
Thanks! Hadn't noticed that the icon was mirrored, thanks for pointing it out!

ken 7y ago

As someone who used to work on ETL systems, from a distance, this looks basically like Tableau to me. (My first thought was actually "oh, neat, someone made an open-source Tableau clone!", but it's not open-source, is it?) You should describe how it's different. If I'm going to pay money for a hosted application for serious work, I'm going to tend to prefer an established service that I'm confident will be around for a while. I'm sure you've got something good here but I'm not sure what it is.

thingsilearned 7y ago
We do compete very directly with Tableau and are often used along side it as well. In the article I explain a lot of what's different about it from other BI products like Tableau. The main things are - it's way more flexible as a visual language, and through a lot of prototypes and user tests has shown 80% of business users can write significant queries on their first try. As for an established service - Chartio's been around for almost 10 years, we're profitable, and are the main data interface behind some really great companies and brands https://chartio.com/customers/ https://chartio.com/customers/

longtermd 7y ago

Postgres' pgAdmin software is still a huge pain for many operations, especially for creating functions and views. I believe they could learn a lot from this.

grenoire 7y ago
To me, pgAdmin seemed like a very light header on the administrative (hah) end of pg. Checking which instances are running, and seeing if the databases are healthy etc.

techolic 7y ago

For some reason I read 'Viral SQL', these days...

ComodoHacker 7y ago

Signup to try — no, thanks.

boublepop 7y ago
Seems like a fair trade, you get to try out for free as long as you let them know that you are interesting in trying it out. They are aiming for business users who don’t mind dropping an e-mail for a quick eval.

larodi 7y ago

No visual has beaten SQL for the last 40 years

bmn__ 7y ago
It's not a competition where one solution has to beat the other. Querying through a textual language and through a visual interface complement each other. I consider a product to be a success when it combines the best features of both and makes them available to the user so that he may organise his work flow according to the own preferred style.
- larodi 7y ago
  That is true, you are right. What I'm saying is that eventually things that need be SQL are either SQL... or sometimes R. The language is more expressive and it takes less time to 'select something from something where conditions group by something', rather than drag and drop it. and is the reason why SQL is so fundamental to all areas of information technology and particularly when data is involved. besides, SQL is DATALOG which is PROLOG which is to 'program logic', which gets as close as it can to 'wiring' logic. many attempts to visualize this process has failed so far, because... they are more complex and more specific. SQL takes like 30 chars and dozen keywords to master. any visual paradigm - takes more.

stiray 7y ago

Just a detail. Have you seen Microsoft Access query builder? Please do check it, especially how you create joins. As far as I have seen tools for building queries none came close. Maybe sqlyog. https://www.youtube.com/watch?v=pcjXQqKWWNw https://www.youtube.com/watch?v=pcjXQqKWWNw

mjirv 7y ago

I like the user-research-first approach here a lot. Very different than many other BI tools in a refreshing way. In the past I've hesitated to consider Chartio for my organization because most people don't know SQL, but I'm excited to give it a try now. I also love how everything is a table. Looker does this, and it's great (though Chartio's implementation looks better). Tableau does not, which makes certain things a headache. I do wonder how well this will scale to large datasets and columnar databases, though. Showing a preview of the tables is really cool and useful, but it is not what data warehouses like Redshift are designed for. The preview probably wouldn't load in time for my users to see it unless they just sit there waiting for it.

thingsilearned 7y ago
Thanks, the user-research-first part was definitely enlightening and important to do. Our first many iterations were very wrong. As for large scale datasets - we do cache those sample tables and we only grab the first 10 rows. Also, BigQuery actually has a great API for fetching a set of sample data that we utilize heavily. They made it because of exactly the potential concerns you outlined around columnar stores.
- mjirv 7y ago
  Oh neat, I didn't know that about BigQuery. Thanks for the tip!

BLanen 7y ago

I made this in an internship except that I get the metadata from Entity Framework and those subqueries but those woudn't have been hard to implement.

rco8786 7y ago

Seems reasonably interesting, but I can’t get past the claim that they made 21 figma boards every day, including weekends, for 3 months.

johnthescott 7y ago

"sql easy, data is hard". perfectly written. why even use sql if you are simply looking for patterns in static data? why not use an app dedicated to analytics, of which many exist? sql comes into play when data needs to be shared, and those types of apps need coders who would not struggle with sql. i simply do not buy the argument that sql is difficult to learn for competent coders.