4 ms·

Was hoping to see Web SQL make its way back into Firefox. Yes, it's been deprecated for years, but that hasn't stopped the browser makers with majority of mark

Was hoping to see Web SQL make its way back into Firefox. Yes, it's been deprecated for years, but that hasn't stopped the browser makers with majority of market share (Google > Chrome, Apple > Safari) from continuing to ship it along with the strictly worse alternative (IndexedDb). Now that pretty much every language compiles to Javascript it's a shame to lose SQL functionality on the client. Shared model, shared validation, shared SQL layer, etc. -- it could all be a unified whole, but no, now one has to wait for sqlite-lite to be built on top of IndexedDb; that or throw everything out and go NoSQL on client and server.

neil_macintyre 9y ago
Is IndexDB slow at querying a database or is it just a pain to implement?
- virtualwhys 9y ago
  Totally different API, Sqlite is SQL based while IndexedDb is a key/value store with indexing and transaction capabilities (essentially the bare minimum on which a relational database could be built, the irony). If your model is highly relational then it doesn't matter that querying for one entity is fast when what you really need is to, for example, efficiently join together multiple entities, grouping on N columns to produce some aggregate result. This is completely straightforward in SQL, but a convoluted inefficient, error prone mess in the alternative.
  - littlestymaar 9y ago
    > IndexedDb is a key/value store with indexing and transaction capabilities (essentially the bare minimum on which a relational database could be built, the irony). It's not ironic at all, it's just a good application of the extensible web manifesto : the browser support minimum necessary building blocs and then libraires can enhance the developer experience. In that case, you have this https://google.github.io/lovefield/ https://google.github.io/lovefield/
fiatjaf 9y ago
Why would you want SQL on the browser? You cannot sync it to any SQL backend. Are you shipping pure client-side apps with complex data structures?
- virtualwhys 9y ago
  > Why would you want SQL on the browser? To perform efficient storage and querying of client-side data. I'm having a hard time understanding why one wouldn't want SQL in the browser given the alternatives. > You cannot sync it to any SQL backend. Sure you can, most client-side frameworks these days (React, Angular, ...) keep track of application state. On state change push to client and/or server database via REST API depending on network connectivity.
jgj 9y ago
> but no, now one has to wait for sqlite-lite to be built on top of IndexedDb not exactly sqlite-lite, but it's sqlish and it's build on top of IDB: https://github.com/google/lovefield https://github.com/google/lovefield
- virtualwhys 9y ago
  Thanks for the link, I've been hunting around everywhere looking for a cross platform solution. In a sibling comment I mentioned the fact that most languages compile to Javascript, the primary benefit being able to share code across client and server. So, if you've got a query DSL on the server that hooks into an SQL database you can use the same code on the client, hooking into an SQL database on the frontend. All of the IndexedDb wrappers I've seen do essentially their own thing; in Lovefield's case, "The difference is that Lovefield does not accept raw SQL statements; instead, it has a builder-pattern API to build queries", which means it's effectively a client-side only solution if pure SQL is the output target of your query DSL.
CorpusCalcium 9y ago
What practical benefit would that really have over just using your choice of SQL engine with wasm, though?
- virtualwhys 9y ago
  I don't know, is my choice of SQL engine (sqlite) via WASM a realistic solution? How does one compile sqlite to WASM, and how is this made available to the end user? sql.js[1] is a pure JS implementation of sqlite and it weighs a ton (2.6MB) compared to browser supplied native sqlite. Ideally the user doesn't have to pay the price of downloading a large binary (on top of already large SPA). [1] https://github.com/kripken/sql.js/ https://github.com/kripken/sql.js/
  - CorpusCalcium 9y ago
    Sure, but that's the JS version. wasm is meant to produce smaller, less resource-intensive versions (and there is a wasm version of sqlite I believe [1]). I'd much rather see CDN and caching tech improve along with other low-level tools such as wasm and CSS Houdini to let us build whatever we'd like. I can see that being better for devs than waiting for vendors to standardize a watered-down spec for each feature we might want like SQL (and there's a lot of them). [1] https://github.com/mandel59/sqlite-wasm https://github.com/mandel59/sqlite-wasm
    - virtualwhys 9y ago
      Agreed, browser vendors may not always have the best interests of developers in mind. If WASM approach can deliver a relatively small binary with better-than-js performance, then that may be a viable alternative, thanks.
pcwalton 9y ago
Web SQL was never in Firefox. The problem with Web SQL was that the spec was effectively just "whatever SQLite does". That isn't good enough for the Web. Imagine if the GIF specification had been "whatever GIFLIB does". Libraries like Lovefield can be built way faster than specifications can be written.
- virtualwhys 9y ago
  > Libraries like Lovefield can be built way faster than specifications can be written. That may be true, but when said library uses a builder pattern rather than sql statements you can throw out all of your server-side query DSL code and waste time reimplementing everything on the client. Since SQL has been around for several decades it's not surprising that mature server-side SQL libraries are available in abundance. And these days every language under the sun compiles to javascript -- wouldn't it be great to share the same query DSL code on client and server? The answer is, emphatically, yes! Current state of affairs makes this virtually impossible to do in a widely supported manner.
nickik 9y ago
This would be way better then either: https://github.com/mozilla/mentat https://github.com/mozilla/mentat
- virtualwhys 9y ago
  Sure, same with Haskell's Esqueleto, Scala's Slick and Quill, F#'s LINQ, etc., these would all be wonderful to run in the browser, but that's not the reality, nothing is ready yet. Mentat, for example, is built on sqlite, and in order to use it you have to run an Electron app. If there was a native version of sqlite (or other sqlite-like database) then there would be no such restriction. It seems the best bet all of the above projects have for a friction-free implementation is WASM. When that's widely supported then one can ship whatever one wants (e.g. WASM sqlite) with their client-side application. Then, finally, it will be possible to use the same query DSL on client and server.

neil_macintyre 9y ago

Is IndexDB slow at querying a database or is it just a pain to implement?

virtualwhys 9y ago
Totally different API, Sqlite is SQL based while IndexedDb is a key/value store with indexing and transaction capabilities (essentially the bare minimum on which a relational database could be built, the irony). If your model is highly relational then it doesn't matter that querying for one entity is fast when what you really need is to, for example, efficiently join together multiple entities, grouping on N columns to produce some aggregate result. This is completely straightforward in SQL, but a convoluted inefficient, error prone mess in the alternative.
- littlestymaar 9y ago
  > IndexedDb is a key/value store with indexing and transaction capabilities (essentially the bare minimum on which a relational database could be built, the irony). It's not ironic at all, it's just a good application of the extensible web manifesto : the browser support minimum necessary building blocs and then libraires can enhance the developer experience. In that case, you have this https://google.github.io/lovefield/ https://google.github.io/lovefield/

fiatjaf 9y ago

Why would you want SQL on the browser? You cannot sync it to any SQL backend. Are you shipping pure client-side apps with complex data structures?

virtualwhys 9y ago
> Why would you want SQL on the browser? To perform efficient storage and querying of client-side data. I'm having a hard time understanding why one wouldn't want SQL in the browser given the alternatives. > You cannot sync it to any SQL backend. Sure you can, most client-side frameworks these days (React, Angular, ...) keep track of application state. On state change push to client and/or server database via REST API depending on network connectivity.

jgj 9y ago

> but no, now one has to wait for sqlite-lite to be built on top of IndexedDb not exactly sqlite-lite, but it's sqlish and it's build on top of IDB: https://github.com/google/lovefield https://github.com/google/lovefield

virtualwhys 9y ago
Thanks for the link, I've been hunting around everywhere looking for a cross platform solution. In a sibling comment I mentioned the fact that most languages compile to Javascript, the primary benefit being able to share code across client and server. So, if you've got a query DSL on the server that hooks into an SQL database you can use the same code on the client, hooking into an SQL database on the frontend. All of the IndexedDb wrappers I've seen do essentially their own thing; in Lovefield's case, "The difference is that Lovefield does not accept raw SQL statements; instead, it has a builder-pattern API to build queries", which means it's effectively a client-side only solution if pure SQL is the output target of your query DSL.

CorpusCalcium 9y ago

What practical benefit would that really have over just using your choice of SQL engine with wasm, though?

virtualwhys 9y ago
I don't know, is my choice of SQL engine (sqlite) via WASM a realistic solution? How does one compile sqlite to WASM, and how is this made available to the end user? sql.js[1] is a pure JS implementation of sqlite and it weighs a ton (2.6MB) compared to browser supplied native sqlite. Ideally the user doesn't have to pay the price of downloading a large binary (on top of already large SPA). [1] https://github.com/kripken/sql.js/ https://github.com/kripken/sql.js/
- CorpusCalcium 9y ago
  Sure, but that's the JS version. wasm is meant to produce smaller, less resource-intensive versions (and there is a wasm version of sqlite I believe [1]). I'd much rather see CDN and caching tech improve along with other low-level tools such as wasm and CSS Houdini to let us build whatever we'd like. I can see that being better for devs than waiting for vendors to standardize a watered-down spec for each feature we might want like SQL (and there's a lot of them). [1] https://github.com/mandel59/sqlite-wasm https://github.com/mandel59/sqlite-wasm
  - virtualwhys 9y ago
    Agreed, browser vendors may not always have the best interests of developers in mind. If WASM approach can deliver a relatively small binary with better-than-js performance, then that may be a viable alternative, thanks.

pcwalton 9y ago

Web SQL was never in Firefox. The problem with Web SQL was that the spec was effectively just "whatever SQLite does". That isn't good enough for the Web. Imagine if the GIF specification had been "whatever GIFLIB does". Libraries like Lovefield can be built way faster than specifications can be written.

virtualwhys 9y ago
> Libraries like Lovefield can be built way faster than specifications can be written. That may be true, but when said library uses a builder pattern rather than sql statements you can throw out all of your server-side query DSL code and waste time reimplementing everything on the client. Since SQL has been around for several decades it's not surprising that mature server-side SQL libraries are available in abundance. And these days every language under the sun compiles to javascript -- wouldn't it be great to share the same query DSL code on client and server? The answer is, emphatically, yes! Current state of affairs makes this virtually impossible to do in a widely supported manner.

nickik 9y ago

This would be way better then either: https://github.com/mozilla/mentat https://github.com/mozilla/mentat

virtualwhys 9y ago
Sure, same with Haskell's Esqueleto, Scala's Slick and Quill, F#'s LINQ, etc., these would all be wonderful to run in the browser, but that's not the reality, nothing is ready yet. Mentat, for example, is built on sqlite, and in order to use it you have to run an Electron app. If there was a native version of sqlite (or other sqlite-like database) then there would be no such restriction. It seems the best bet all of the above projects have for a friction-free implementation is WASM. When that's widely supported then one can ship whatever one wants (e.g. WASM sqlite) with their client-side application. Then, finally, it will be possible to use the same query DSL on client and server.