4 ms·

I never tire of seeing Datasette on HN. It saved me last week when business folks sent me four huge excel sheets of data that I'd need to answer questions abou

I never tire of seeing Datasette on HN. It saved me last week when business folks sent me four huge excel sheets of data that I'd need to answer questions about during a meeting. I exported to CSV, then into SQLite, and immediately had a shareable web page I could query and filter in realtime. Vastly more useful to me (as an engineer) than Excel, and having the full power of SQL is a delight. Wish list: I wish it were a single executable I could carry around, like fossil, jq, rclone, rg/ag, sqlite3, gitea, micro, etc.

simonw 6y ago
So great to hear it being used like that! I'd really love to have a standalone executable version of Datasette too. I've been meaning to spend some time on that - bundling Python applications is tricky but it should be possible using BeeWare or PyInstaller. The best way to install it at the moment is "brew install datasette" - but that only works if you're on a Mac. I really need to get on top of the Windows installation process too.
- mastazi 6y ago
  You could implement "scoop install datasette" or "choco install datasette". Way easier than creating an installer, for example this is how you create a Scoop manifest: https://github.com/lukesampson/scoop/wiki/Creating-an-app-manifest https://github.com/lukesampson/scoop/wiki/Creating-an-app-ma...
  - simonw 6y ago
    That's really useful, thanks.
  - ErikBjare 6y ago
    It's a common misconception that PyInstaller is something that creates an installer. Scoop and Chocolatey are nice, but you'll still need something like PyInstaller to bundle your Python app into something that can be installed by scoop or choco. (source: building ActivityWatch [1] which is bundled by PyInstaller and available on Chocolatey) [1]: https://activitywatch.net https://activitywatch.net
- tecleandor 6y ago
  Instead of PyInstaller, I had success using Nuitka. Doesn't work with every Python package, but I think it's easier to use.
hobs 6y ago
Worth mentioning that https://github.com/dfinke/ImportExcel https://github.com/dfinke/ImportExcel does excel -> powershell -> excel in a really nice package for that kind of filtering/etc work.
nojito 6y ago
>Vastly more useful to me (as an engineer) than Excel, Open Excel --> Click Data --> Get Data --> From File (Csv) -> If it's clean already just do load to (add to data model) --> alt+n+v to insert pivot and impress your business folks.
- rpdillon 6y ago
  Thanks for the tip! The main issue in this particular case was that I never work with Excel, so I was shipped Excel files via Google Sheets (since I have no Excel license on my Mac). I was mostly trying to get into some environment that I was comfortable with so I could work confidently while others were waiting. Datasette was the best tool I could think of that would allow me full SQL access while also presenting a web page others could grok easily. The terminal is home for me, but I think it's off-putting to some, and Datasette bridges that nicely.
  - leonim 6y ago
    If the terminal is your home, I would suggest looking at something already mentioned by someone else VisiData (http://visidata.org http://visidata.org), it is great for exploring all sorts of data files like CSV, Excel, Sqlite3, etc in the terminal. I find the Datasette author's related tools sqlite-utils (https://github.com/simonw/sqlite-utils https://github.com/simonw/sqlite-utils) and the Dogsheep tools (https://github.com/dogsheep https://github.com/dogsheep) and VisiData are nicely complementary. I prefer the interface of VisiData, but I don't usually need to share links.
    - rpdillon 6y ago
      This is fantastic! Thanks so much for the pointer...the ability to traverse tons of data formats quickly to "get a feel" for them (much like simonw's use case for some of the sqlite-utils) is exactly the tooling I need in cases like this, and the TUI interface is an awesome bonus. Will definitely be playing around with this over the next few days...thanks!
tga 6y ago
I’ve done pretty much the same thing many years ago by connecting directly to the Excel sheet from Access, and then writing queries and reports on top of it. The advantage was that there was no import step, so edits could be sent back right away. If I remember correctly, Excel sheets can be directly used as ODBC data sources. This means that they can be accessed directly by database apps.

simonw 6y ago

So great to hear it being used like that! I'd really love to have a standalone executable version of Datasette too. I've been meaning to spend some time on that - bundling Python applications is tricky but it should be possible using BeeWare or PyInstaller. The best way to install it at the moment is "brew install datasette" - but that only works if you're on a Mac. I really need to get on top of the Windows installation process too.

mastazi 6y ago
You could implement "scoop install datasette" or "choco install datasette". Way easier than creating an installer, for example this is how you create a Scoop manifest: https://github.com/lukesampson/scoop/wiki/Creating-an-app-manifest https://github.com/lukesampson/scoop/wiki/Creating-an-app-ma...
- simonw 6y ago
  That's really useful, thanks.
- ErikBjare 6y ago
  It's a common misconception that PyInstaller is something that creates an installer. Scoop and Chocolatey are nice, but you'll still need something like PyInstaller to bundle your Python app into something that can be installed by scoop or choco. (source: building ActivityWatch [1] which is bundled by PyInstaller and available on Chocolatey) [1]: https://activitywatch.net https://activitywatch.net
tecleandor 6y ago
Instead of PyInstaller, I had success using Nuitka. Doesn't work with every Python package, but I think it's easier to use.

hobs 6y ago

Worth mentioning that https://github.com/dfinke/ImportExcel https://github.com/dfinke/ImportExcel does excel -> powershell -> excel in a really nice package for that kind of filtering/etc work.

nojito 6y ago

>Vastly more useful to me (as an engineer) than Excel, Open Excel --> Click Data --> Get Data --> From File (Csv) -> If it's clean already just do load to (add to data model) --> alt+n+v to insert pivot and impress your business folks.

rpdillon 6y ago
Thanks for the tip! The main issue in this particular case was that I never work with Excel, so I was shipped Excel files via Google Sheets (since I have no Excel license on my Mac). I was mostly trying to get into some environment that I was comfortable with so I could work confidently while others were waiting. Datasette was the best tool I could think of that would allow me full SQL access while also presenting a web page others could grok easily. The terminal is home for me, but I think it's off-putting to some, and Datasette bridges that nicely.
- leonim 6y ago
  If the terminal is your home, I would suggest looking at something already mentioned by someone else VisiData (http://visidata.org http://visidata.org), it is great for exploring all sorts of data files like CSV, Excel, Sqlite3, etc in the terminal. I find the Datasette author's related tools sqlite-utils (https://github.com/simonw/sqlite-utils https://github.com/simonw/sqlite-utils) and the Dogsheep tools (https://github.com/dogsheep https://github.com/dogsheep) and VisiData are nicely complementary. I prefer the interface of VisiData, but I don't usually need to share links.
  - rpdillon 6y ago
    This is fantastic! Thanks so much for the pointer...the ability to traverse tons of data formats quickly to "get a feel" for them (much like simonw's use case for some of the sqlite-utils) is exactly the tooling I need in cases like this, and the TUI interface is an awesome bonus. Will definitely be playing around with this over the next few days...thanks!

tga 6y ago

I’ve done pretty much the same thing many years ago by connecting directly to the Excel sheet from Access, and then writing queries and reports on top of it. The advantage was that there was no import step, so edits could be sent back right away. If I remember correctly, Excel sheets can be directly used as ODBC data sources. This means that they can be accessed directly by database apps.