10 ms·

Evidence.dev – Business Intelligence as Code

391 points by hughess 3y ago

Ideabile 3y ago
Not sure this is posted by the author. Anyway, I find this tools fitting a very nice niece. I’m currently working in a similar extension of markdown. In case, the author pass by, would love to connect.
- hughess 3y ago
  I'm one of the founders of Evidence - would be happy to chat!
jnsaff2 3y ago
Markdown is becoming a very nice platform. With Livebook, Mermaid, MathJax and all the other great tools.
- benatkin 3y ago
  The page says Markdown, but I see MDX in the video
  - amcaskill 3y ago
    It’s actually MDSVEX (the svelte version of MDX), we are weighing out a move to markdoc.
    - benatkin 3y ago
      How about something like [`input`](magic-link)? I came up with this for https://codeberg.org/macchiato https://codeberg.org/macchiato ( though it's not yet implemented in the new project, just in the predecessor, https://gitlab.com/ResourcesCo/notebook https://gitlab.com/ResourcesCo/notebook ). The backquotes differentiate from non-magic links. (I tried badges, but they aligned weirdly.) You could use [`data.mrr`](https://evidence.dev/md/value https://evidence.dev/md/value) or any other internal DSL you can come up with. Another thing you could do is just decide against MD(SVE)X the format and keep the style and transform inline codeblocks that match. (MRR grew by `<Value data={data.mrr}>` yesterday.) That you said Markdown to me says you aren't on sold on using an incompatible syntax.
    - niteshade 3y ago
      Any chance of making Evidence available as a Remark plugin? Would love to be able to use it as a tool for interactive documentation
ihucos 3y ago
That looks amazing.
llanowarelves 3y ago
Wow. This is at just the right level of abstraction for me.
wfhBrian 3y ago
If you like this, Dataview[1] is great for querying your markdown notes. [1] https://blacksmithgu.github.io/obsidian-dataview/queries/structure/ https://blacksmithgu.github.io/obsidian-dataview/queries/str...
- AB1908 3y ago
  Dataview sorely needs maintainers. The author is working on a successor but I still want to use the original one.
- AB1908 3y ago
  Oh, and while we're on the topic, anyone have something like a media database where they record metadata about what they watch and when? I want to create yearly media dashboards and I've run into a few tricky problems around metadata organization. The querying has been easy for the most part.
jensenbox 3y ago
Had this been Python I would be all over it.
_jplc 3y ago
How different is this from a Jupyter Notebook? https://jupyter.org/ https://jupyter.org/ This seems like a simplified version that could reach it's limit pretty soon.
- breakfastduck 3y ago
  because its just markdown and SQL without all the other overhead.
  - _jplc 3y ago
    I guess that where you see overhead I see options.
    - breakfastduck 3y ago
      Well yeah, sure. That's the difference, it has less options and complexity than a notebook. This is simpler - that's a valid reason for it to exist.
- hughess 3y ago
  It looks a bit like a notebook, but the output of an Evidence project is a static site, so you can build a data product that takes advantage of web features like click-based navigation. Queries and pages are built once at build time, so production projects are very fast. Here's an example project from one of our open source community members: https://mdsinabox.com/ https://mdsinabox.com/
  - _jplc 3y ago
    There's Voilà for Jupyter Notebooks, that does exactly that: https://github.com/voila-dashboards/voila https://github.com/voila-dashboards/voila
    - archiewood 3y ago
      Having just tried out Voila, I do think these products are serving pretty different audiences on the front end. When I open examples in the Voila gallery, it spends 15s or so building the site, which a dev may tolerate, but a business exec has already given up on. One of the benefits of a static site for reporting use cases is that it’s just HTML and JS. It loads virtually instantly as it doesn’t hit a database - all the data is shipped with the site. Open it on your phone when you wake up, in a cab, by the gate at the airport: you’re looking at a mobile optimized site, and 5s later you can finish writing that email with the datapoint you needed.
esafak 3y ago
Can it create interactive documents like distill.pub or observablehq? I want my reports to look beautiful, and these sites are exemplars. I can't stand that jupyter junk. And it would allow you to support a use case beyond BI.
- hughess 3y ago
  We care a ton about making reports look beautiful. At the moment, we have some interactivity through data viz components (tooltips, sorting, etc.), but we have a collection of much more interactive features on our roadmap. Here are a couple examples of production Evidence projects: https://evidence-demo.netlify.app/ https://evidence-demo.netlify.app/ https://mdsinabox.com https://mdsinabox.com
dmattia 3y ago
I use a similar tool at my work (airplane.dev) that lets us make dashboards as code using raw SQL or a language of our choice. I love it, and it gets a ton of use from our engineers and customer support folks. But our CEO, sales, etc. teams that are often consumers of BI tools do not like it, as they do not like SQL, nor putting things in code, nor markdown. So we kept the "traditional" BI tools and ETL shenanigans as well. It's expensive to have both, but I don't think either fully replaces the other, and I don't regret going down the path we have at all. I think most folks are happy.
- hughess 3y ago
  That's really interesting - airplane.dev looks great. We've had similar experiences with BI consumers in the past. In our experience, it's really hard to make self-serve work for business users. Initially, we're starting with ways to make it easy for end users to get information out of Evidence (things like CSV download, smooth copy-paste into docs/email/PPT/etc). Over time we'll be putting more effort into features that will let people explore data from within Evidence (which we call "best of BI" internally). We're thinking very carefully about it so we can hit a few of our key principles: maintainable/version controllable by data team, very fast - no waiting time for users, intuitive for end users - limited if any training required.
  - dmattia 3y ago
    Those sounds like nice goals, I'll be watching :)
  - addisonl 3y ago
    Based on what you see about airplane, is there a good feature that evidence has over it? New to both of these tools and am very interested to hear your thoughts.
    - amcaskill 3y ago
      I haven’t used airplane, but it looks like a more dev oriented retool, which is an awesome idea. We’re building evidence specifically for producing reporting suites, automated analysis and other data products. As a result, we’re keenly focused on data analysts, not developers per se (although lots of engineers have chosen to use evidence as well). We offer features focused on that use case and user group. SQL oriented workflow. A large and growing charting library, templates pages, a static build process so your reports are really fast, docs on using it with dbt, those types of things. We have a lot more “best of BI” features to come. All of that said, I have seen people build really valuable data products with retool, so I am sure you could with airplane as well, but it’s definitely aimed at a much more general developer audience and “internal tool” use case.
edublancas 3y ago
If anyone is looking for something like this in Python/Jupyter, check out JupySQL: https://github.com/ploomber/jupysql https://github.com/ploomber/jupysql
- pea 3y ago
  You might be interested in what we're hacking on at Datapane (I'm one of the founders): https://github.com/datapane/datapane https://github.com/datapane/datapane. You can create standalone HTML data reports from Python/Jupyter in ~3 lines of code: https://docs.datapane.com/reports/overview/ https://docs.datapane.com/reports/overview/
  - amcaskill 3y ago
    Very cool! Just to help folks picture what they can build. Here’s an evidence app: https://evidence-demo.netlify.app/ https://evidence-demo.netlify.app/ Here’s a Datapane app: https://dp-examples-1164c73e-stock-ticker.fly.dev/ https://dp-examples-1164c73e-stock-ticker.fly.dev/
    - pea 3y ago
      Thanks for sharing, that looks great! That Datapane example is more a hello world web app that runs Python code in the backend (so requires a backend server - Fly.io in this example). An example of a standalone report would be something like this, from one of our users: https://cloud.datapane.com/reports/dkjbvwk/literature-in-bloom/ https://cloud.datapane.com/reports/dkjbvwk/literature-in-blo... (code: https://github.com/ryancahildebrandt/hanakotoba https://github.com/ryancahildebrandt/hanakotoba) or https://cloud.datapane.com/reports/aAMaqoA/when-fact-is-false-and-false-is-funny-automated-detection-of-fake-and-satire-news/ https://cloud.datapane.com/reports/aAMaqoA/when-fact-is-fals...
- wodenokoto 3y ago
  I found it unclear from the documentation, but does this support auto-complete? I’d love to move my mssql notebooks out of Azure Data Studio.
VWWHFSfQ 3y ago
The problem with all of these "BI as Code" things is that _nobody_ that you want to use this knows Python or Markdown. If they did then they would just use Python and Markdown like they usually do. This stuff is all pointless. There is no audience for it.
- Entinel 3y ago
  You are wrong. I work in FinTech and use Python and markdown. I am very interested in this. The current visualization tools in the Python space are very hamfisted. Dash and Plotly all hit a wall when trying to do complex visualizations.
  - amcaskill 3y ago
    Sean, my cofounder, and I previously led the data team at a PE fund. This absolutely tracks with our experience.
  - ironchef 3y ago
    I'm curious what you hit in dash and plotly that evidence let you overcome. (FTR, I use all of them, lightdash, streamlit, (as well as drop down to flask) etc.)
    - Entinel 3y ago
      I haven't used Evidence so can't speak to its capabilities. It's been a while since I've used Dash and Streamlit so I can't recall exactly what caused me to throw it aside but I do remember that formatting was hell and to even get close to what I wanted I had to use components from plotly express, dash, streamlit, essentially every Python data viz library. It was just too much of a hassle.
      - ironchef 3y ago
        Yeah. The frameworks solve the 80 percent problem quite well but once you start trying to be as specific as the business often wants, I would find them lacking and end up popping JSON to d3 (or some such)
stanislavb 3y ago
And it's Open Source: https://github.com/evidence-dev/evidence https://github.com/evidence-dev/evidence I'd also highly recommend Blazer https://github.com/ankane/blazer https://github.com/ankane/blazer if you are into the Ruby on Rails world. It's super solid, and it's been an indispensable tool integrated to all my projects.
- archiewood 3y ago
  Only came across blazer yesterday, on an awesome-BI list on Github, but I do really dig it. Love how simple it is, the auto-configuration of charts etc. Even just playing around with the demo there was a lot to like: https://blazer.dokkuapp.com/queries/new https://blazer.dokkuapp.com/queries/new
- Aeolun 3y ago
  > open-source I was completely prepared to find another ‘mildly open source but not really’ license, but it’s actually just MIT.
nwbort 3y ago
A bit like Quarto[1] for SQL? [1] https://quarto.org/ https://quarto.org/
- archiewood 3y ago
  100%
skadamat 3y ago
I built 2 little reports using Evidence 6 months ago in just a few hours with new data I hadn't interacted with! It was super fun to work at this level of abstraction, because I can spend most of my time thinking about the data and how I want to represent it. It's a neat mashup of a lot of declarative-style approaches (SQL, ECharts, ~Markdown) glued together elegantly.
- hughess 3y ago
  Thank you for the kind words! Really glad you enjoyed using it
iFire 3y ago
Does Evidence.dev have a newsletter? Not ready to use evidence, but want to at some point.
- hughess 3y ago
  We have a mailing list you can sign up to at the very bottom of our website. We haven't sent much to date, but we are about to start sending consistent updates, so it's a good time to sign up. There are also a few other great ways to keep up to date (Slack, Twitter, and Github) - links here: https://docs.evidence.dev/community https://docs.evidence.dev/community When you try it out, please reach out - would love to hear any feedback
larve 3y ago
I've been using evidence as part of my pipeline for generating reports at work, and it's wonderful. I use my own tool to generate the SQL itself, because I'm tired of writing SQL by itself. Here's a little overview about how this all works: https://share.descript.com/view/ppBx1MZB174 https://share.descript.com/view/ppBx1MZB174 (and https://share.descript.com/view/pMYlLZO4hOo https://share.descript.com/view/pMYlLZO4hOo for generating the commands generating the SQL using GPT). I've got a pretty nifty little reports app for the business I work going now. Beautiful, fast, does what it says, thanks so much for this.
- hughess 3y ago
  That's fantastic to hear - thank you! Looking forward to checking out your overview
- nonethewiser 3y ago
  Is that a particular templating language? Or is it something you rolled?
  - larve 3y ago
    just go templates
- amcaskill 3y ago
  Would you be open to chatting with us about your project? Watching these videos I feel like we (the evidence team) could learn a lot from what you’ve built. If so, feel free to shoot me an email: adam [at] evidence.dev
- vladsanchez 3y ago
  Fantastic! Thanks for sharing! <3
hughess 3y ago
This is getting quite a bit more attention than we expected. For HN readers who are interested, we are currently building a hosting, deployment and auth service for Evidence projects - Evidence Cloud. Evidence Cloud is currently onboarding customers in private beta - for early access, join the waitlist here: https://du3tapwtcbi.typeform.com/to/kwp7ZD3q https://du3tapwtcbi.typeform.com/to/kwp7ZD3q
jaxn 3y ago
Thank you for building this! I am really excited to try it out on a backlog item this week. It seems like you really nailed a sweet spot. Analyst first reports as a product that can be extended / enhanced by devs, and both can continue to maintain the project once it ships.
- amcaskill 3y ago
  Thank you for writing this! Not sure if this is exactly the case you’re thinking of, but one thing that is really nice about Evidence is that (if you want to) you can include custom svelte components in your projects. A surprising (to me) number of organizations have custom d3 viz, or other components that they are using in one piece of software, but their data teams have no way to leverage that work. In evidence both groups have a completely suitable collaboration environment. A dev can drop off a custom component, the data team can use it, and then tweak and maintain the SQL that powers it.
BiteCode_dev 3y ago
Super interesting. Something like this but with duckdb + python support would be nice. In fact, I would imagine this kind of things to be a good use case for introducing templating in jupyter notebooks.
- hughess 3y ago
  We have support for duckdb (and CSVs and Parquet through duckdb). We don't support python, but some people have also told us they have used evidence as the front-end for a python project - used python to do data transformation and calculations, then dumped the results into a duckdb file in an evidence project and built the visuals and narrative in evidence. "Containerized" approaches with evidence are also quite interesting - lets you combine several tools and use evidence as the last mile. Here's a great example: https://github.com/matsonj/nba-monte-carlo https://github.com/matsonj/nba-monte-carlo
franchb 3y ago
Awesome! It works with Deno! Just clone Evidence template and run deno task build.
Ekanyani 3y ago
Nice
smatija 3y ago
Does it support mssql? If not how hard would it be to add it inhouse? Additionaly I think "Connecting to database" guide would be helpful addition to docs. To me it's a bit counterintuitive that I found first mention of database support in section "CLI reference". Maybe it already exists, I just couldn't find it.
- hughess 3y ago
  It doesn’t support MSSQL yet, but several people have requested it, so it’s definitely something we’ll support. We accept open source contributions for anyone who wants to add a database connector (a few of our connectors are open source contributions) - as long as there is an npm package for connecting to a database, setting up the connector is fairly straightforward. Thanks for the feedback. There’s some info under Data Sources in the Core Concepts section, but maybe we should make the db support more prominent in the docs: https://docs.evidence.dev/core-concepts/data-sources/ https://docs.evidence.dev/core-concepts/data-sources/ The db connections are set up in the Settings page when you spin up an Evidence project, so we’ve also left a lot of the documentation in the product itself.
djoldman 3y ago
off topic but... the favicon is 672kb?! https://evidence.dev/favicon.ico https://evidence.dev/favicon.ico
twobitshifter 3y ago
Why not use quarto to call into a DB? Or why not just add sql support to quarto?
- hughess 3y ago
  Quarto's a great tool, but I think it would be hard to maintain it as a reporting solution for a company. My co-founder Adam and I used to love using RMarkdown in our previous job, but found that as we tried to scale that in our team, it became unmaintainable. Keeping the tool limited to SQL, markdown, and components is a way to help data teams maintain a project across a large organization. We also think that reporting outputs should have a very high bar for visual appearance, and it takes quite a bit of work in Quarto to get charts up to that standard of quality. It sounds like a nice-to-have feature, but in our experience, publication quality data viz is a very important way to build trust with businesspeople who read the reports. We spend a ton of effort thinking through design choices at Evidence and want to give data teams a library of components they can pull off the shelf and write in a simple, declarative syntax.
  - twobitshifter 3y ago
    Interesting. I use Observable with quarto, instead of R or Python, which gives you access to any JavaScript charting library. I don’t know what’s available for R charts, but hopefully you can get some higher quality charts built for Evidence. The good thing about quarto is that it’s also pandoc, so you can get the outputs in all the business formats you need.
potamic 3y ago
The online demo doesn't work because Stackblitz is asking for third-party cookies
- archiewood 3y ago
  It doesn’t work on Safari, and WebKit based browsers (and therefore, all browsers on iOS) Shoukd work on Chrome, Firefox etc on all other platforms
LeonB 3y ago
I’ve made a bunch of things that are somewhat like this (combining markdown with sql and charts) and I find it so fun to use, and so fun to implement. Interactive coding stuff is just - it’s like catnip. So addictive to use.
replwoacause 3y ago
This looks really nice.

391 points by hughess 3y ago

Not sure this is posted by the author. Anyway, I find this tools fitting a very nice niece. I’m currently working in a similar extension of markdown. In case, the author pass by, would love to connect.

hughess 3y ago
I'm one of the founders of Evidence - would be happy to chat!

jnsaff2 3y ago

Markdown is becoming a very nice platform. With Livebook, Mermaid, MathJax and all the other great tools.

benatkin 3y ago
The page says Markdown, but I see MDX in the video
- amcaskill 3y ago
  It’s actually MDSVEX (the svelte version of MDX), we are weighing out a move to markdoc.
  - benatkin 3y ago
    How about something like [`input`](magic-link)? I came up with this for https://codeberg.org/macchiato https://codeberg.org/macchiato ( though it's not yet implemented in the new project, just in the predecessor, https://gitlab.com/ResourcesCo/notebook https://gitlab.com/ResourcesCo/notebook ). The backquotes differentiate from non-magic links. (I tried badges, but they aligned weirdly.) You could use [`data.mrr`](https://evidence.dev/md/value https://evidence.dev/md/value) or any other internal DSL you can come up with. Another thing you could do is just decide against MD(SVE)X the format and keep the style and transform inline codeblocks that match. (MRR grew by `<Value data={data.mrr}>` yesterday.) That you said Markdown to me says you aren't on sold on using an incompatible syntax.
  - niteshade 3y ago
    Any chance of making Evidence available as a Remark plugin? Would love to be able to use it as a tool for interactive documentation

ihucos 3y ago

That looks amazing.

llanowarelves 3y ago

Wow. This is at just the right level of abstraction for me.

wfhBrian 3y ago

If you like this, Dataview[1] is great for querying your markdown notes. [1] https://blacksmithgu.github.io/obsidian-dataview/queries/structure/ https://blacksmithgu.github.io/obsidian-dataview/queries/str...

AB1908 3y ago
Dataview sorely needs maintainers. The author is working on a successor but I still want to use the original one.
AB1908 3y ago
Oh, and while we're on the topic, anyone have something like a media database where they record metadata about what they watch and when? I want to create yearly media dashboards and I've run into a few tricky problems around metadata organization. The querying has been easy for the most part.

jensenbox 3y ago

Had this been Python I would be all over it.

_jplc 3y ago

How different is this from a Jupyter Notebook? https://jupyter.org/ https://jupyter.org/ This seems like a simplified version that could reach it's limit pretty soon.

breakfastduck 3y ago
because its just markdown and SQL without all the other overhead.
- _jplc 3y ago
  I guess that where you see overhead I see options.
  - breakfastduck 3y ago
    Well yeah, sure. That's the difference, it has less options and complexity than a notebook. This is simpler - that's a valid reason for it to exist.
hughess 3y ago
It looks a bit like a notebook, but the output of an Evidence project is a static site, so you can build a data product that takes advantage of web features like click-based navigation. Queries and pages are built once at build time, so production projects are very fast. Here's an example project from one of our open source community members: https://mdsinabox.com/ https://mdsinabox.com/
- _jplc 3y ago
  There's Voilà for Jupyter Notebooks, that does exactly that: https://github.com/voila-dashboards/voila https://github.com/voila-dashboards/voila
  - archiewood 3y ago
    Having just tried out Voila, I do think these products are serving pretty different audiences on the front end. When I open examples in the Voila gallery, it spends 15s or so building the site, which a dev may tolerate, but a business exec has already given up on. One of the benefits of a static site for reporting use cases is that it’s just HTML and JS. It loads virtually instantly as it doesn’t hit a database - all the data is shipped with the site. Open it on your phone when you wake up, in a cab, by the gate at the airport: you’re looking at a mobile optimized site, and 5s later you can finish writing that email with the datapoint you needed.

esafak 3y ago

Can it create interactive documents like distill.pub or observablehq? I want my reports to look beautiful, and these sites are exemplars. I can't stand that jupyter junk. And it would allow you to support a use case beyond BI.

hughess 3y ago
We care a ton about making reports look beautiful. At the moment, we have some interactivity through data viz components (tooltips, sorting, etc.), but we have a collection of much more interactive features on our roadmap. Here are a couple examples of production Evidence projects: https://evidence-demo.netlify.app/ https://evidence-demo.netlify.app/ https://mdsinabox.com https://mdsinabox.com

dmattia 3y ago

I use a similar tool at my work (airplane.dev) that lets us make dashboards as code using raw SQL or a language of our choice. I love it, and it gets a ton of use from our engineers and customer support folks. But our CEO, sales, etc. teams that are often consumers of BI tools do not like it, as they do not like SQL, nor putting things in code, nor markdown. So we kept the "traditional" BI tools and ETL shenanigans as well. It's expensive to have both, but I don't think either fully replaces the other, and I don't regret going down the path we have at all. I think most folks are happy.

hughess 3y ago
That's really interesting - airplane.dev looks great. We've had similar experiences with BI consumers in the past. In our experience, it's really hard to make self-serve work for business users. Initially, we're starting with ways to make it easy for end users to get information out of Evidence (things like CSV download, smooth copy-paste into docs/email/PPT/etc). Over time we'll be putting more effort into features that will let people explore data from within Evidence (which we call "best of BI" internally). We're thinking very carefully about it so we can hit a few of our key principles: maintainable/version controllable by data team, very fast - no waiting time for users, intuitive for end users - limited if any training required.
- dmattia 3y ago
  Those sounds like nice goals, I'll be watching :)
- addisonl 3y ago
  Based on what you see about airplane, is there a good feature that evidence has over it? New to both of these tools and am very interested to hear your thoughts.
  - amcaskill 3y ago
    I haven’t used airplane, but it looks like a more dev oriented retool, which is an awesome idea. We’re building evidence specifically for producing reporting suites, automated analysis and other data products. As a result, we’re keenly focused on data analysts, not developers per se (although lots of engineers have chosen to use evidence as well). We offer features focused on that use case and user group. SQL oriented workflow. A large and growing charting library, templates pages, a static build process so your reports are really fast, docs on using it with dbt, those types of things. We have a lot more “best of BI” features to come. All of that said, I have seen people build really valuable data products with retool, so I am sure you could with airplane as well, but it’s definitely aimed at a much more general developer audience and “internal tool” use case.

edublancas 3y ago

If anyone is looking for something like this in Python/Jupyter, check out JupySQL: https://github.com/ploomber/jupysql https://github.com/ploomber/jupysql

pea 3y ago
You might be interested in what we're hacking on at Datapane (I'm one of the founders): https://github.com/datapane/datapane https://github.com/datapane/datapane. You can create standalone HTML data reports from Python/Jupyter in ~3 lines of code: https://docs.datapane.com/reports/overview/ https://docs.datapane.com/reports/overview/
- amcaskill 3y ago
  Very cool! Just to help folks picture what they can build. Here’s an evidence app: https://evidence-demo.netlify.app/ https://evidence-demo.netlify.app/ Here’s a Datapane app: https://dp-examples-1164c73e-stock-ticker.fly.dev/ https://dp-examples-1164c73e-stock-ticker.fly.dev/
  - pea 3y ago
    Thanks for sharing, that looks great! That Datapane example is more a hello world web app that runs Python code in the backend (so requires a backend server - Fly.io in this example). An example of a standalone report would be something like this, from one of our users: https://cloud.datapane.com/reports/dkjbvwk/literature-in-bloom/ https://cloud.datapane.com/reports/dkjbvwk/literature-in-blo... (code: https://github.com/ryancahildebrandt/hanakotoba https://github.com/ryancahildebrandt/hanakotoba) or https://cloud.datapane.com/reports/aAMaqoA/when-fact-is-false-and-false-is-funny-automated-detection-of-fake-and-satire-news/ https://cloud.datapane.com/reports/aAMaqoA/when-fact-is-fals...
wodenokoto 3y ago
I found it unclear from the documentation, but does this support auto-complete? I’d love to move my mssql notebooks out of Azure Data Studio.

VWWHFSfQ 3y ago

The problem with all of these "BI as Code" things is that _nobody_ that you want to use this knows Python or Markdown. If they did then they would just use Python and Markdown like they usually do. This stuff is all pointless. There is no audience for it.

Entinel 3y ago
You are wrong. I work in FinTech and use Python and markdown. I am very interested in this. The current visualization tools in the Python space are very hamfisted. Dash and Plotly all hit a wall when trying to do complex visualizations.
- amcaskill 3y ago
  Sean, my cofounder, and I previously led the data team at a PE fund. This absolutely tracks with our experience.
- ironchef 3y ago
  I'm curious what you hit in dash and plotly that evidence let you overcome. (FTR, I use all of them, lightdash, streamlit, (as well as drop down to flask) etc.)
  - Entinel 3y ago
    I haven't used Evidence so can't speak to its capabilities. It's been a while since I've used Dash and Streamlit so I can't recall exactly what caused me to throw it aside but I do remember that formatting was hell and to even get close to what I wanted I had to use components from plotly express, dash, streamlit, essentially every Python data viz library. It was just too much of a hassle.
    - ironchef 3y ago
      Yeah. The frameworks solve the 80 percent problem quite well but once you start trying to be as specific as the business often wants, I would find them lacking and end up popping JSON to d3 (or some such)

stanislavb 3y ago

And it's Open Source: https://github.com/evidence-dev/evidence https://github.com/evidence-dev/evidence I'd also highly recommend Blazer https://github.com/ankane/blazer https://github.com/ankane/blazer if you are into the Ruby on Rails world. It's super solid, and it's been an indispensable tool integrated to all my projects.

archiewood 3y ago
Only came across blazer yesterday, on an awesome-BI list on Github, but I do really dig it. Love how simple it is, the auto-configuration of charts etc. Even just playing around with the demo there was a lot to like: https://blazer.dokkuapp.com/queries/new https://blazer.dokkuapp.com/queries/new
Aeolun 3y ago
> open-source I was completely prepared to find another ‘mildly open source but not really’ license, but it’s actually just MIT.

nwbort 3y ago

A bit like Quarto[1] for SQL? [1] https://quarto.org/ https://quarto.org/

archiewood 3y ago
100%

skadamat 3y ago

I built 2 little reports using Evidence 6 months ago in just a few hours with new data I hadn't interacted with! It was super fun to work at this level of abstraction, because I can spend most of my time thinking about the data and how I want to represent it. It's a neat mashup of a lot of declarative-style approaches (SQL, ECharts, ~Markdown) glued together elegantly.

hughess 3y ago
Thank you for the kind words! Really glad you enjoyed using it

iFire 3y ago

Does Evidence.dev have a newsletter? Not ready to use evidence, but want to at some point.

hughess 3y ago
We have a mailing list you can sign up to at the very bottom of our website. We haven't sent much to date, but we are about to start sending consistent updates, so it's a good time to sign up. There are also a few other great ways to keep up to date (Slack, Twitter, and Github) - links here: https://docs.evidence.dev/community https://docs.evidence.dev/community When you try it out, please reach out - would love to hear any feedback

larve 3y ago

I've been using evidence as part of my pipeline for generating reports at work, and it's wonderful. I use my own tool to generate the SQL itself, because I'm tired of writing SQL by itself. Here's a little overview about how this all works: https://share.descript.com/view/ppBx1MZB174 https://share.descript.com/view/ppBx1MZB174 (and https://share.descript.com/view/pMYlLZO4hOo https://share.descript.com/view/pMYlLZO4hOo for generating the commands generating the SQL using GPT). I've got a pretty nifty little reports app for the business I work going now. Beautiful, fast, does what it says, thanks so much for this.

hughess 3y ago
That's fantastic to hear - thank you! Looking forward to checking out your overview
nonethewiser 3y ago
Is that a particular templating language? Or is it something you rolled?
- larve 3y ago
  just go templates
amcaskill 3y ago
Would you be open to chatting with us about your project? Watching these videos I feel like we (the evidence team) could learn a lot from what you’ve built. If so, feel free to shoot me an email: adam [at] evidence.dev
vladsanchez 3y ago
Fantastic! Thanks for sharing! <3

hughess 3y ago

This is getting quite a bit more attention than we expected. For HN readers who are interested, we are currently building a hosting, deployment and auth service for Evidence projects - Evidence Cloud. Evidence Cloud is currently onboarding customers in private beta - for early access, join the waitlist here: https://du3tapwtcbi.typeform.com/to/kwp7ZD3q https://du3tapwtcbi.typeform.com/to/kwp7ZD3q

jaxn 3y ago

Thank you for building this! I am really excited to try it out on a backlog item this week. It seems like you really nailed a sweet spot. Analyst first reports as a product that can be extended / enhanced by devs, and both can continue to maintain the project once it ships.

amcaskill 3y ago
Thank you for writing this! Not sure if this is exactly the case you’re thinking of, but one thing that is really nice about Evidence is that (if you want to) you can include custom svelte components in your projects. A surprising (to me) number of organizations have custom d3 viz, or other components that they are using in one piece of software, but their data teams have no way to leverage that work. In evidence both groups have a completely suitable collaboration environment. A dev can drop off a custom component, the data team can use it, and then tweak and maintain the SQL that powers it.

BiteCode_dev 3y ago

Super interesting. Something like this but with duckdb + python support would be nice. In fact, I would imagine this kind of things to be a good use case for introducing templating in jupyter notebooks.

hughess 3y ago
We have support for duckdb (and CSVs and Parquet through duckdb). We don't support python, but some people have also told us they have used evidence as the front-end for a python project - used python to do data transformation and calculations, then dumped the results into a duckdb file in an evidence project and built the visuals and narrative in evidence. "Containerized" approaches with evidence are also quite interesting - lets you combine several tools and use evidence as the last mile. Here's a great example: https://github.com/matsonj/nba-monte-carlo https://github.com/matsonj/nba-monte-carlo

franchb 3y ago

Awesome! It works with Deno! Just clone Evidence template and run deno task build.

Ekanyani 3y ago

Nice

smatija 3y ago

Does it support mssql? If not how hard would it be to add it inhouse? Additionaly I think "Connecting to database" guide would be helpful addition to docs. To me it's a bit counterintuitive that I found first mention of database support in section "CLI reference". Maybe it already exists, I just couldn't find it.

hughess 3y ago
It doesn’t support MSSQL yet, but several people have requested it, so it’s definitely something we’ll support. We accept open source contributions for anyone who wants to add a database connector (a few of our connectors are open source contributions) - as long as there is an npm package for connecting to a database, setting up the connector is fairly straightforward. Thanks for the feedback. There’s some info under Data Sources in the Core Concepts section, but maybe we should make the db support more prominent in the docs: https://docs.evidence.dev/core-concepts/data-sources/ https://docs.evidence.dev/core-concepts/data-sources/ The db connections are set up in the Settings page when you spin up an Evidence project, so we’ve also left a lot of the documentation in the product itself.

djoldman 3y ago

off topic but... the favicon is 672kb?! https://evidence.dev/favicon.ico https://evidence.dev/favicon.ico

twobitshifter 3y ago

Why not use quarto to call into a DB? Or why not just add sql support to quarto?

hughess 3y ago
Quarto's a great tool, but I think it would be hard to maintain it as a reporting solution for a company. My co-founder Adam and I used to love using RMarkdown in our previous job, but found that as we tried to scale that in our team, it became unmaintainable. Keeping the tool limited to SQL, markdown, and components is a way to help data teams maintain a project across a large organization. We also think that reporting outputs should have a very high bar for visual appearance, and it takes quite a bit of work in Quarto to get charts up to that standard of quality. It sounds like a nice-to-have feature, but in our experience, publication quality data viz is a very important way to build trust with businesspeople who read the reports. We spend a ton of effort thinking through design choices at Evidence and want to give data teams a library of components they can pull off the shelf and write in a simple, declarative syntax.
- twobitshifter 3y ago
  Interesting. I use Observable with quarto, instead of R or Python, which gives you access to any JavaScript charting library. I don’t know what’s available for R charts, but hopefully you can get some higher quality charts built for Evidence. The good thing about quarto is that it’s also pandoc, so you can get the outputs in all the business formats you need.

potamic 3y ago

The online demo doesn't work because Stackblitz is asking for third-party cookies

archiewood 3y ago
It doesn’t work on Safari, and WebKit based browsers (and therefore, all browsers on iOS) Shoukd work on Chrome, Firefox etc on all other platforms

LeonB 3y ago

I’ve made a bunch of things that are somewhat like this (combining markdown with sql and charts) and I find it so fun to use, and so fun to implement. Interactive coding stuff is just - it’s like catnip. So addictive to use.

replwoacause 3y ago

This looks really nice.