4 ms·

Interesting project. I would've liked to have heard a brief comparison with Excel - what makes Datasette something that one should reach for over that? Is it

Interesting project. I would've liked to have heard a brief comparison with Excel - what makes Datasette something that one should reach for over that? Is it just the ability to use SQL in-place?

steve_adams_86 4y ago
Some things which stand out to me at a glance: - it’s open source - it allows publishing to the web - it provides handy web-based tooling, like turning your data into a JSON api - it provides a user-friendly UI but also lets you dig deeper with plain old SQL (pro to some, con to others) - apparently has an extensible core with a rich plugin ecosystem which allows it to meet a wide range of needs I’m sure there’s more, but these are all really interesting to me.
- Lio 4y ago
  Just taking the first point, being open source means that I don't need a licence to use Datasette and that I can run it on all three OSs I have to work with (Linux, macOS and Windows).
pletnes 4y ago
It’s got plugins for some stuff Excel don’t. I’ve used one for map/gis data to look at lat/long data on a map. Never seen that in Excel. The dataset was too big for Excel, too.
simonw 4y ago
When I've talked to startups in the past I've always warned them to be careful about accidentally building a product which competes with a potential customer's existing good Excel spreadsheet... so it's ironic that I've ended up myself focusing on something that falls into that bucket! Right now I'd say the big things are: 1. Size of data. Datasette is a great tool for when your data is too big for Excel but not so big that you need a data warehouse - think 100MB to 10GB range 2. Interoperability and automation. Building tools in other languages that interoperate with an Excel spreadsheet - a web crawler that updates it constantly over time for example - is possible but in my experience pretty hard. Writing to a SQLite file is usually much easier in comparison 3. Publishing data. This is a core concept in Datasette: the ability to publish your data online so other people can explore and interact with it. 4. Plugins. Writing plugins for Datasette is pretty easy, and there are over 100 now: https://datasette.io/plugins https://datasette.io/plugins - so if it doesn't do something you need, you can add that feature independently of the core project 5. SQLite generally. Some projects are a better fit for a database than a spreadsheet, and SQLite is a /really/ good (often underestimated) database.
- darkteflon 4y ago
  Hi Simon, thanks so much for taking the time. Been following your excellent work with Django for many years. We took a look at DS several months ago (I believe it popped up here on HN) but I confess that until I read the linked article I hadn’t fully understood that it was a “hammer”, and a good one at that. At that time we set it aside in favour of AirTable which for our purposes - to relate this to one of your other comments here - is sort of Access with a web interface plus authentication and (frankly not very granular) authorisation. The main reason was because it seemed to us that Datasette didn’t have first class support for auth - that it was sort of positioned as a community or internal tool, rather than something we could build out a client interface on. I just had a look at the plugins and noted that there is something for auth, although it too looks rather to be for internal tooling. Is that a fair characterisation, or do we have it wrong? Because we love everything else about DS - open source, built on SQL and SQLite, etc. - and would vastly prefer to use it over AirTable.
  - simonw 4y ago
    Airtable is a fantastic piece of software. I'd love Datasette to work as an alternative for it some day but there is a whole lot of stuff to build to get there. The most relevant plugin for what you're describing at the moment is probably the Auth0 one: https://datasette.io/plugins/datasette-auth0 https://datasette.io/plugins/datasette-auth0 Quite a lot of assembly is required at the moment if you want to do interesting custom things with authentication and permissions - the documentation here is the best starting point: https://docs.datasette.io/en/stable/authentication.html https://docs.datasette.io/en/stable/authentication.html
    - darkteflon 4y ago
      Fantastic - thanks Simon, much appreciated. Yes we generally like the AirTable UX, too, and it’s a good fit for client-facing stuff in addition to internal tooling. It is slow, though, and there are no native desktop or mobile clients. Mainly, though: building a business on it comes with all the usual pitfalls of VC-backed SaaS. It’s very helpful to hear your thoughts on the project’s trajectory. We’ll take another look at DS and the auth0 module, thanks. It looks very promising and we don’t mind a bit of assembly.
morgango 4y ago
I think the best comparison would be with Access, seems to be the all-in-one data tool that tried to be.
- simonw 4y ago
  Yeah, I'm constantly surprised at Microsoft's lack of investment in Access. The world clearly needs a really great, easy to use personal database. Datasette isn't there yet but it's a useful ambition for the future.

Interesting project. I would've liked to have heard a brief comparison with Excel - what makes Datasette something that one should reach for over that? Is it just the ability to use SQL in-place?

steve_adams_86 4y ago

Some things which stand out to me at a glance: - it’s open source - it allows publishing to the web - it provides handy web-based tooling, like turning your data into a JSON api - it provides a user-friendly UI but also lets you dig deeper with plain old SQL (pro to some, con to others) - apparently has an extensible core with a rich plugin ecosystem which allows it to meet a wide range of needs I’m sure there’s more, but these are all really interesting to me.

Lio 4y ago
Just taking the first point, being open source means that I don't need a licence to use Datasette and that I can run it on all three OSs I have to work with (Linux, macOS and Windows).

pletnes 4y ago

It’s got plugins for some stuff Excel don’t. I’ve used one for map/gis data to look at lat/long data on a map. Never seen that in Excel. The dataset was too big for Excel, too.

simonw 4y ago

When I've talked to startups in the past I've always warned them to be careful about accidentally building a product which competes with a potential customer's existing good Excel spreadsheet... so it's ironic that I've ended up myself focusing on something that falls into that bucket! Right now I'd say the big things are: 1. Size of data. Datasette is a great tool for when your data is too big for Excel but not so big that you need a data warehouse - think 100MB to 10GB range 2. Interoperability and automation. Building tools in other languages that interoperate with an Excel spreadsheet - a web crawler that updates it constantly over time for example - is possible but in my experience pretty hard. Writing to a SQLite file is usually much easier in comparison 3. Publishing data. This is a core concept in Datasette: the ability to publish your data online so other people can explore and interact with it. 4. Plugins. Writing plugins for Datasette is pretty easy, and there are over 100 now: https://datasette.io/plugins https://datasette.io/plugins - so if it doesn't do something you need, you can add that feature independently of the core project 5. SQLite generally. Some projects are a better fit for a database than a spreadsheet, and SQLite is a /really/ good (often underestimated) database.

darkteflon 4y ago
Hi Simon, thanks so much for taking the time. Been following your excellent work with Django for many years. We took a look at DS several months ago (I believe it popped up here on HN) but I confess that until I read the linked article I hadn’t fully understood that it was a “hammer”, and a good one at that. At that time we set it aside in favour of AirTable which for our purposes - to relate this to one of your other comments here - is sort of Access with a web interface plus authentication and (frankly not very granular) authorisation. The main reason was because it seemed to us that Datasette didn’t have first class support for auth - that it was sort of positioned as a community or internal tool, rather than something we could build out a client interface on. I just had a look at the plugins and noted that there is something for auth, although it too looks rather to be for internal tooling. Is that a fair characterisation, or do we have it wrong? Because we love everything else about DS - open source, built on SQL and SQLite, etc. - and would vastly prefer to use it over AirTable.
- simonw 4y ago
  Airtable is a fantastic piece of software. I'd love Datasette to work as an alternative for it some day but there is a whole lot of stuff to build to get there. The most relevant plugin for what you're describing at the moment is probably the Auth0 one: https://datasette.io/plugins/datasette-auth0 https://datasette.io/plugins/datasette-auth0 Quite a lot of assembly is required at the moment if you want to do interesting custom things with authentication and permissions - the documentation here is the best starting point: https://docs.datasette.io/en/stable/authentication.html https://docs.datasette.io/en/stable/authentication.html
  - darkteflon 4y ago
    Fantastic - thanks Simon, much appreciated. Yes we generally like the AirTable UX, too, and it’s a good fit for client-facing stuff in addition to internal tooling. It is slow, though, and there are no native desktop or mobile clients. Mainly, though: building a business on it comes with all the usual pitfalls of VC-backed SaaS. It’s very helpful to hear your thoughts on the project’s trajectory. We’ll take another look at DS and the auth0 module, thanks. It looks very promising and we don’t mind a bit of assembly.

morgango 4y ago

I think the best comparison would be with Access, seems to be the all-in-one data tool that tried to be.

simonw 4y ago
Yeah, I'm constantly surprised at Microsoft's lack of investment in Access. The world clearly needs a really great, easy to use personal database. Datasette isn't there yet but it's a useful ambition for the future.