5 ms·

Offline-First PWA for Plotly Visualization of CSV via SQLite

246 points by lana-k 5y ago

jimsparkman 5y ago
This looks really great, like a local version of redash but way more in the way of charting options. Such simplicity to boot.
technologia 5y ago
This is great, your demo’s music just made the task seem hilariously easy.
klmadfejno 5y ago
Perhaps a dumb question but why is offline stressed so much here? Nothing in the underlying tech seems to suggest one should expect it to not work offline?
- EvilEy3 5y ago
  Because you don't want your application to become potato on each network hiccup?
- osrec 5y ago
  Browser based apps generally struggle to function well offline (this is also true of certain native apps, but that's a whole other discussion). Using the PWA "toolbox" you can have in-browser apps stored locally via service workers so that they work without needing to fetch anything via the internet. It's rather useful, and can provide a much better user experience, especially when you have a patchy network.
  - tmpz22 5y ago
    What’s an example of a large tech product that functions as a PWA that users really love?
    - osrec 5y ago
      There's Twitter's PWA, which isn't too bad in my opinion. My product has approx 300k users and is a PWA: https://usebx.com https://usebx.com - our users seem to like it :)
      - dkarp 5y ago
        If you don’t mind me asking, how does your product make money?
        osrec 5y ago
        We have a quota on the number of documents you can create, after which you must subscribe. It's a very generous quota, but we still have a good number of people subscribing. Our major income comes from larger corporate contracts with medium to large businesses, that deploy our product on-prem. Often these were people who used our SaaS product, liked it, and asked us to deploy it internally for their business. Much less hassle than running a company that's purely SaaS based, and more stable income!
    - echoradio 5y ago
      I don’t know if meets the condition of “users really love,” but don’t Google Docs have the capability of functioning as an offline PWA?
    - jononor 5y ago
      Dunno about "really love" anymore, but gmail?
- aseipp 5y ago
  You need to use specific browser APIs (Service Workers) in order to support "offline mode" for a webpage, including support when there's no internet at all. This means the page and its content is stored locally for later, separately from normal HTTP caching. Yes, if you loaded the page once into a tab and then never closed it ever again, then it doesn't matter so much and you could use it if your internet shat out, but in practice people close and then re-open things later under varying network conditions.
- lana-k 5y ago
  Other users have already explained the technical aspect of PWA (i.e. you can install it and run offline as a desktop app). I just want to add that by offline-first I also mean confidentiality -- your data doesn't leave your device, whether you're offline or not.
osrec 5y ago
Really like the concept and demo. And it looks well made! Just out of interest, is this something you did as a hobby/for fun, or was there a particular use case you were addressing?
- lana-k 5y ago
  It started as a collaboration with https://pypi.org/project/Procpath/ https://pypi.org/project/Procpath/. For me as a frontend developer it's an interesting challenge to develop, in fact, a desktop app (with background workers running a database and exchanging data with the main thread and so on) in the form of progressive web app (PWA). But in general I want to solve a problem: a lot of people know SQL which is actually powerful to wrangle data to prepare it for visualisation and analysis. It can't be simpler than drag-and-dropping a CSV into a browser, producing a result set and consuming it by the visualisation component (now Plotly, but more to come).
  - osrec 5y ago
    Genuinely impressed! For me, the most impressive part is the configurability of the charts. Did you implement each configuration for each chart type individually, or were you leveraging some library to help populate the chart configuration parameters?
chunkyks 5y ago
I was looking for something like this the other day. I ended up just implementing functions directly in sql like plot_bar(category, value), plot_points(xval, yval), which open a popup with the chart.
kwhitefoot 5y ago
PWA?
- code-is-code 5y ago
  Progressive Web App
- rubiquity 5y ago
  Programmaz Wit Attitudes
rebhan 5y ago
Why Plotly, isnt https://echarts.apache.org/en/index.html https://echarts.apache.org/en/index.html more powerful ?
- lana-k 5y ago
  Plotly has this: https://github.com/plotly/react-chart-editor https://github.com/plotly/react-chart-editor. Is there something similar for echarts? Anyway, thank you for a hint.
  - tyingq 5y ago
    Not exactly what you asked for, but close: https://echarts.apache.org/v4/en/theme-builder.html https://echarts.apache.org/v4/en/theme-builder.html
- akdor1154 5y ago
  I can't tell, their lets-reimplement-scrolling-in-js gallery page is breaking in my Firefox.
  - deleted 5y ago
    [deleted]
hmsimha 5y ago
This seems really neat. Curious if you saw this magic SQLite -> Static file tool shared here a couple of weeks ago: https://news.ycombinator.com/item?id=27016630 https://news.ycombinator.com/item?id=27016630 IIUC It would enable this tool to work with a dataset in sqlite that could be recompiled as static files periodically, and the relevant queries could be made on the front-end without having to load the whole data-set or select a CSV file from the filesystem. Would also lead to much better performance if you want users to be able to host generated charts on a static file server, presumably the output could be committed to git and then synced to a github pages or something. edit: It appears the author of that tool is using Plotly as well in the demo website. An integration of these would be incredible.
krick 5y ago
This is absolutely fantastic. I didn't expect much, but this is really, really awesome. Thank you. P.S. Obvious feature seems to be missing: it would be great if one could name columns manually (instead of col1, col2, etc) when importing a csv w/o header.
iddan 5y ago
Beautiful and useful!
bndw 5y ago
Thank you!!! I use plotly to generate one-off graphs regularly and love the idea of offline-first + SQL.

246 points by lana-k 5y ago

This looks really great, like a local version of redash but way more in the way of charting options. Such simplicity to boot.

technologia 5y ago

This is great, your demo’s music just made the task seem hilariously easy.

klmadfejno 5y ago

Perhaps a dumb question but why is offline stressed so much here? Nothing in the underlying tech seems to suggest one should expect it to not work offline?

EvilEy3 5y ago
Because you don't want your application to become potato on each network hiccup?
osrec 5y ago
Browser based apps generally struggle to function well offline (this is also true of certain native apps, but that's a whole other discussion). Using the PWA "toolbox" you can have in-browser apps stored locally via service workers so that they work without needing to fetch anything via the internet. It's rather useful, and can provide a much better user experience, especially when you have a patchy network.
- tmpz22 5y ago
  What’s an example of a large tech product that functions as a PWA that users really love?
  - osrec 5y ago
    There's Twitter's PWA, which isn't too bad in my opinion. My product has approx 300k users and is a PWA: https://usebx.com https://usebx.com - our users seem to like it :)
    - dkarp 5y ago
      If you don’t mind me asking, how does your product make money?
      - osrec 5y ago
        We have a quota on the number of documents you can create, after which you must subscribe. It's a very generous quota, but we still have a good number of people subscribing. Our major income comes from larger corporate contracts with medium to large businesses, that deploy our product on-prem. Often these were people who used our SaaS product, liked it, and asked us to deploy it internally for their business. Much less hassle than running a company that's purely SaaS based, and more stable income!
  - echoradio 5y ago
    I don’t know if meets the condition of “users really love,” but don’t Google Docs have the capability of functioning as an offline PWA?
  - jononor 5y ago
    Dunno about "really love" anymore, but gmail?
aseipp 5y ago
You need to use specific browser APIs (Service Workers) in order to support "offline mode" for a webpage, including support when there's no internet at all. This means the page and its content is stored locally for later, separately from normal HTTP caching. Yes, if you loaded the page once into a tab and then never closed it ever again, then it doesn't matter so much and you could use it if your internet shat out, but in practice people close and then re-open things later under varying network conditions.
lana-k 5y ago
Other users have already explained the technical aspect of PWA (i.e. you can install it and run offline as a desktop app). I just want to add that by offline-first I also mean confidentiality -- your data doesn't leave your device, whether you're offline or not.

osrec 5y ago

Really like the concept and demo. And it looks well made! Just out of interest, is this something you did as a hobby/for fun, or was there a particular use case you were addressing?

lana-k 5y ago
It started as a collaboration with https://pypi.org/project/Procpath/ https://pypi.org/project/Procpath/. For me as a frontend developer it's an interesting challenge to develop, in fact, a desktop app (with background workers running a database and exchanging data with the main thread and so on) in the form of progressive web app (PWA). But in general I want to solve a problem: a lot of people know SQL which is actually powerful to wrangle data to prepare it for visualisation and analysis. It can't be simpler than drag-and-dropping a CSV into a browser, producing a result set and consuming it by the visualisation component (now Plotly, but more to come).
- osrec 5y ago
  Genuinely impressed! For me, the most impressive part is the configurability of the charts. Did you implement each configuration for each chart type individually, or were you leveraging some library to help populate the chart configuration parameters?

chunkyks 5y ago

I was looking for something like this the other day. I ended up just implementing functions directly in sql like plot_bar(category, value), plot_points(xval, yval), which open a popup with the chart.

kwhitefoot 5y ago

PWA?

code-is-code 5y ago
Progressive Web App
rubiquity 5y ago
Programmaz Wit Attitudes

rebhan 5y ago

Why Plotly, isnt https://echarts.apache.org/en/index.html https://echarts.apache.org/en/index.html more powerful ?

lana-k 5y ago
Plotly has this: https://github.com/plotly/react-chart-editor https://github.com/plotly/react-chart-editor. Is there something similar for echarts? Anyway, thank you for a hint.
- tyingq 5y ago
  Not exactly what you asked for, but close: https://echarts.apache.org/v4/en/theme-builder.html https://echarts.apache.org/v4/en/theme-builder.html
akdor1154 5y ago
I can't tell, their lets-reimplement-scrolling-in-js gallery page is breaking in my Firefox.
- deleted 5y ago
  [deleted]

hmsimha 5y ago

This seems really neat. Curious if you saw this magic SQLite -> Static file tool shared here a couple of weeks ago: https://news.ycombinator.com/item?id=27016630 https://news.ycombinator.com/item?id=27016630 IIUC It would enable this tool to work with a dataset in sqlite that could be recompiled as static files periodically, and the relevant queries could be made on the front-end without having to load the whole data-set or select a CSV file from the filesystem. Would also lead to much better performance if you want users to be able to host generated charts on a static file server, presumably the output could be committed to git and then synced to a github pages or something. edit: It appears the author of that tool is using Plotly as well in the demo website. An integration of these would be incredible.

krick 5y ago

This is absolutely fantastic. I didn't expect much, but this is really, really awesome. Thank you. P.S. Obvious feature seems to be missing: it would be great if one could name columns manually (instead of col1, col2, etc) when importing a csv w/o header.

iddan 5y ago

Beautiful and useful!

bndw 5y ago

Thank you!!! I use plotly to generate one-off graphs regularly and love the idea of offline-first + SQL.