5 ms·

The Google Chrome team is working with the SQLite team to standardise a WASM based SQLite in the browser to replace WebSQL. https://developer.chrome.com/blog/d

The Google Chrome team is working with the SQLite team to standardise a WASM based SQLite in the browser to replace WebSQL. https://developer.chrome.com/blog/deprecating-web-sql/ https://developer.chrome.com/blog/deprecating-web-sql/ https://twitter.com/chromiumdev/status/1565105522092695553 https://twitter.com/chromiumdev/status/1565105522092695553 Which would be pretty cool. Projects like SQL.js and absurd-sql are awesome but naturally a bit rough around the edges and not highly maintained.

modeless 4y ago
This is good but it can never be as good as Web SQL could have been, because it can't be a truly shared library. Like all WASM modules, It has to be downloaded and JIT compiled separately for every site that uses it. Not for technical reasons, but privacy reasons, so it's basically unfixable: https://developer.chrome.com/en/blog/http-cache-partitioning/ https://developer.chrome.com/en/blog/http-cache-partitioning... Maybe if half the sites on the web start using it, browsers can finally be convinced that it would be OK to add SQLite to the base platform.
- thruflo 4y ago
  Interesting — so basically an overhead until it’s fully native? I was reading the Chrome initiative as a pathway to native. Would that mean it can’t be WASM?
  - modeless 4y ago
    That tweet is not implying that it will ever be native. Quite the opposite; Web SQL is native today and they are going to remove it since it was rejected by other browsers.
- samwillis 4y ago
  I disagree, this is much better than WebSQL. WebSQL would have been tied to one specific version of SQLite, and developers would have to work to the lowest supported version. There would have been no extension mechanism. WASM SQLite is the correct solution. It’s extendable by the developer using it, they can uses SQLite extension modules, and build their own. But almost more so it proves the idea of a WASM db engine backed by a low level block FS api. We will see other db engines uses this architecture. DuckDB have already done it. I’m sure MongoDBs Realm and CouchBase Mobile will do the same soon too. We are in for an exciting time in the next few years.
  - modeless 4y ago
    > WebSQL would have been tied to one specific version of SQLite, and developers would have to work to the lowest supported version. There would have been no extension mechanism. Why not? Web SQL would not be different from WebGL/GLSL or even JavaScript itself in that respect. Developers use feature detection and and work to the lowest supported version. APIs and languages can evolve, but in (mostly) backwards compatible ways. Extension and versioning mechanisms can be made. That's how the web works and Web SQL could have worked that way too. More broadly, you could apply arguments like this against everything in the entire web platform. Maybe everything should be a WASM module that developers could choose themselves! Image loading, video decoding, font rendering, DOM, JS engine, why not? It's actually a beautiful vision and I'd be all for it if not for cache partitioning. Every site would have to re-download and re-JIT an entire browser engine before it could do anything. The base platform needs to include a diverse set of commonly used features so that apps don't have to download the world, and on a list of ubiquitous libraries SQLite is right up there with other libraries backing the web platform, like zlib.
    - samwillis 4y ago
      WebGL/GLSL is a low level API the equivalent of the OPFS api. Standardising WebSQL would be like browsers standardising on Three.JS rather than WebGL/GLSL. WebGL/GLSL give the developer a low level api to the graphics hardware. Video decoding apis talk to the hardware video decoding hardware. OPFS gives developers a low level API to the persistent file system / HDD / SSD.
      - modeless 4y ago
        The web platform has tons of very high level stuff in it, much higher level than SQLite, and more is being added. Even specifically on the topic of 3D, they're trying to add a <model> tag and standard 3D model file format to HTML right now. It doesn't make sense to reject Web SQL, which is much more foundational, on the grounds of being too high level. Not now, but even less so back when the decision was made in 2010, when the web itself was all higher level and the lower level APIs you mentioned didn't even exist.
- yarg 4y ago
  Dependency resolution for the web would fix this. It can be done, and needs to be - and more securely than the half-assed efforts of the likes of NPM and Maven. All dependencies signed - let's encrypt has made this a viable option. I don't see how there's anything approximating an unresolvable privacy concern here.
  - coder543 4y ago
    Signing dependencies is related to integrity, which is orthogonal to privacy. Sharing a cache between websites has proven to be a privacy issue. Read more here: https://developer.chrome.com/en/blog/http-cache-partitioning/ https://developer.chrome.com/en/blog/http-cache-partitioning... or here: https://www.peakhour.io/blog/cache-partitioning-firefox-chrome/ https://www.peakhour.io/blog/cache-partitioning-firefox-chro...
    - yarg 4y ago
      As per your second link: > This means if you visited a website and it loaded the resource: > https://www.somesite.com/foo.js https://www.somesite.com/foo.js > and you then visited a second website, and it also included the same resource, then the resource would be loaded from the shared cache rather than being downloaded from the internet a second time. Cookies set by these resources would also be shared. The privacy problem is not a result of the shared dependency, it's a result of the shared cookies. Yes, if you share the execution space between multiple programs running the same lib, there's a privacy concern. No shit - don't fucking do that.
      - coder543 4y ago
        > The privacy problem is not a result of the shared dependency, it's a result of the shared cookies. No, it's not the result of the shared cookies. You just ignored all of the timing attacks and fingerprinting which a shared cache allows, as those articles discuss. The cookie thing is honestly completely irrelevant to the topic of privacy, if you understand how the shared cache used to work. If you loaded the same library from separate CDNs on different websites, the shared cache didn't come into play at all. The library was loaded twice anyways. There was no chance for cookies from different CDNs to accidentally cross the streams. Browsers weren't attempting to heuristically determine if you were trying to load the same asset from different hosts. The shared cache only came into play if you loaded the same asset from the same third-party CDN on multiple websites. The host serving an asset is the one responsible for setting the cookies, and sharing the cookies is helpful in that case, since the same CDN is the host serving the asset to the browser for both websites. These aren't cookies controlled by separate websites, they're cookies supplied by the CDN, and they should be the same for both requests anyways, outside of maybe some remote possibility of a theoretical attack involving a malicious CDN intentionally setting some weird request-dependent cookies, but I can't see how that would even do anything harmful anyways. So, the cookies get shared because the browser is serving a cached response for the same asset from the same host to both sites, which makes sense. So, cookies aren't the problem here, and they're not the reason the shared cache was partitioned. If you want browsers to undo that in any form, you would have to solve the actual privacy problems here.
- Beltalowda 4y ago
  > It has to be downloaded and JIT compiled separately for every site that uses it I don't really see the problem with that. Looking at the sql.js demo[1] the WASM binary is 610K (305K compressed/transferred), and it seems to run pretty fast even on my slow laptop. > Maybe if half the sites on the web start using it Most websites have no reason to use it; simple key/value localStorage is enough for many sites or apps that need some sort of storage. It's kind of a niche thing. Many regular desktop applications have no need SQLite, either.
  - wruza 4y ago
    It’s an ubiquitous need for any app that has a dynamic collection view. You can’t take all sites, see that 99% of them are static documents developed as degenerate apps and then conclude that “most apps” have no reason to use it. It’s like saying that most cats are on jpegs, so cat food is kind of a niche thing for a cat owner.
    - Beltalowda 4y ago
      > “most apps” Please do not put words in quotes as if I said them when, in fact, I did not. Thank you.
      - wruza 4y ago
        What would that trivial correction change, in your opinion?
  - ilyt 4y ago
    > I don't really see the problem with that. Looking at the sql.js demo[1] the WASM binary is 610K (305K compressed/transferred), and it seems to run pretty fast even on my slow laptop. ...and that is small by your standards ?
    - Beltalowda 4y ago
      It's not small; it's also not huge. For the type of website that has use for this it's not really all that much.
samwillis 4y ago
They aren’t working to standardise SQLite, the are working to standardise a low level block based file system api that SQLite and other database engine can use. Thats much more exciting than “standardising SQLite” for the web. The Origin Privet File System api is going to provide the opportunity for any db engine to be used in offline first PWAs. There is no “one size fits all” database engine, that was proved by both WebSQL and IndexedDB. The OPFS in combination with WASM is the correct solution to in browser DBs.
- justin66 4y ago
  > There is no “one size fits all” database engine, that was proved by both WebSQL and IndexedDB. Thanks to a few very small people, WebSQL was deprecated before it had a chance to prove anything.
  - chrismorgan 4y ago
    WebSQL was killed off because the direction it was heading was certain to cause major compatibility problems down the road, and no one was willing to do the work that would be required to avert that (and no one was sold that it would be worth it even at that).
  - kevingadd 4y ago
    Most likely prove its utility as an attack surface and source of compatibility issues
  - nine_k 4y ago
    They had a,point. Making something a standard when only one implementation of it exists is not prudent. There was (and still is) no independent implementation of SQLite that was battle-tested even a bit.
- ilyt 4y ago
  > There is no “one size fits all” database engine, that was proved by both WebSQL and IndexedDB. It was not proven at all. WebSQL was deprecated based on "we don't want to standarize on single project". IndexedDB happened because they wanted to standarize on single API (lmao), but it was just too inept. There is no one size fits all but WebSQL fit A LOT of use cases. Low level storage that works for DBs is interesting idea but that also means you need to ship additional megabytes of code with every app.

modeless 4y ago

This is good but it can never be as good as Web SQL could have been, because it can't be a truly shared library. Like all WASM modules, It has to be downloaded and JIT compiled separately for every site that uses it. Not for technical reasons, but privacy reasons, so it's basically unfixable: https://developer.chrome.com/en/blog/http-cache-partitioning/ https://developer.chrome.com/en/blog/http-cache-partitioning... Maybe if half the sites on the web start using it, browsers can finally be convinced that it would be OK to add SQLite to the base platform.

thruflo 4y ago
Interesting — so basically an overhead until it’s fully native? I was reading the Chrome initiative as a pathway to native. Would that mean it can’t be WASM?
- modeless 4y ago
  That tweet is not implying that it will ever be native. Quite the opposite; Web SQL is native today and they are going to remove it since it was rejected by other browsers.
samwillis 4y ago
I disagree, this is much better than WebSQL. WebSQL would have been tied to one specific version of SQLite, and developers would have to work to the lowest supported version. There would have been no extension mechanism. WASM SQLite is the correct solution. It’s extendable by the developer using it, they can uses SQLite extension modules, and build their own. But almost more so it proves the idea of a WASM db engine backed by a low level block FS api. We will see other db engines uses this architecture. DuckDB have already done it. I’m sure MongoDBs Realm and CouchBase Mobile will do the same soon too. We are in for an exciting time in the next few years.
- modeless 4y ago
  > WebSQL would have been tied to one specific version of SQLite, and developers would have to work to the lowest supported version. There would have been no extension mechanism. Why not? Web SQL would not be different from WebGL/GLSL or even JavaScript itself in that respect. Developers use feature detection and and work to the lowest supported version. APIs and languages can evolve, but in (mostly) backwards compatible ways. Extension and versioning mechanisms can be made. That's how the web works and Web SQL could have worked that way too. More broadly, you could apply arguments like this against everything in the entire web platform. Maybe everything should be a WASM module that developers could choose themselves! Image loading, video decoding, font rendering, DOM, JS engine, why not? It's actually a beautiful vision and I'd be all for it if not for cache partitioning. Every site would have to re-download and re-JIT an entire browser engine before it could do anything. The base platform needs to include a diverse set of commonly used features so that apps don't have to download the world, and on a list of ubiquitous libraries SQLite is right up there with other libraries backing the web platform, like zlib.
  - samwillis 4y ago
    WebGL/GLSL is a low level API the equivalent of the OPFS api. Standardising WebSQL would be like browsers standardising on Three.JS rather than WebGL/GLSL. WebGL/GLSL give the developer a low level api to the graphics hardware. Video decoding apis talk to the hardware video decoding hardware. OPFS gives developers a low level API to the persistent file system / HDD / SSD.
    - modeless 4y ago
      The web platform has tons of very high level stuff in it, much higher level than SQLite, and more is being added. Even specifically on the topic of 3D, they're trying to add a <model> tag and standard 3D model file format to HTML right now. It doesn't make sense to reject Web SQL, which is much more foundational, on the grounds of being too high level. Not now, but even less so back when the decision was made in 2010, when the web itself was all higher level and the lower level APIs you mentioned didn't even exist.
yarg 4y ago
Dependency resolution for the web would fix this. It can be done, and needs to be - and more securely than the half-assed efforts of the likes of NPM and Maven. All dependencies signed - let's encrypt has made this a viable option. I don't see how there's anything approximating an unresolvable privacy concern here.
- coder543 4y ago
  Signing dependencies is related to integrity, which is orthogonal to privacy. Sharing a cache between websites has proven to be a privacy issue. Read more here: https://developer.chrome.com/en/blog/http-cache-partitioning/ https://developer.chrome.com/en/blog/http-cache-partitioning... or here: https://www.peakhour.io/blog/cache-partitioning-firefox-chrome/ https://www.peakhour.io/blog/cache-partitioning-firefox-chro...
  - yarg 4y ago
    As per your second link: > This means if you visited a website and it loaded the resource: > https://www.somesite.com/foo.js https://www.somesite.com/foo.js > and you then visited a second website, and it also included the same resource, then the resource would be loaded from the shared cache rather than being downloaded from the internet a second time. Cookies set by these resources would also be shared. The privacy problem is not a result of the shared dependency, it's a result of the shared cookies. Yes, if you share the execution space between multiple programs running the same lib, there's a privacy concern. No shit - don't fucking do that.
    - coder543 4y ago
      > The privacy problem is not a result of the shared dependency, it's a result of the shared cookies. No, it's not the result of the shared cookies. You just ignored all of the timing attacks and fingerprinting which a shared cache allows, as those articles discuss. The cookie thing is honestly completely irrelevant to the topic of privacy, if you understand how the shared cache used to work. If you loaded the same library from separate CDNs on different websites, the shared cache didn't come into play at all. The library was loaded twice anyways. There was no chance for cookies from different CDNs to accidentally cross the streams. Browsers weren't attempting to heuristically determine if you were trying to load the same asset from different hosts. The shared cache only came into play if you loaded the same asset from the same third-party CDN on multiple websites. The host serving an asset is the one responsible for setting the cookies, and sharing the cookies is helpful in that case, since the same CDN is the host serving the asset to the browser for both websites. These aren't cookies controlled by separate websites, they're cookies supplied by the CDN, and they should be the same for both requests anyways, outside of maybe some remote possibility of a theoretical attack involving a malicious CDN intentionally setting some weird request-dependent cookies, but I can't see how that would even do anything harmful anyways. So, the cookies get shared because the browser is serving a cached response for the same asset from the same host to both sites, which makes sense. So, cookies aren't the problem here, and they're not the reason the shared cache was partitioned. If you want browsers to undo that in any form, you would have to solve the actual privacy problems here.
Beltalowda 4y ago
> It has to be downloaded and JIT compiled separately for every site that uses it I don't really see the problem with that. Looking at the sql.js demo[1] the WASM binary is 610K (305K compressed/transferred), and it seems to run pretty fast even on my slow laptop. > Maybe if half the sites on the web start using it Most websites have no reason to use it; simple key/value localStorage is enough for many sites or apps that need some sort of storage. It's kind of a niche thing. Many regular desktop applications have no need SQLite, either.
- wruza 4y ago
  It’s an ubiquitous need for any app that has a dynamic collection view. You can’t take all sites, see that 99% of them are static documents developed as degenerate apps and then conclude that “most apps” have no reason to use it. It’s like saying that most cats are on jpegs, so cat food is kind of a niche thing for a cat owner.
  - Beltalowda 4y ago
    > “most apps” Please do not put words in quotes as if I said them when, in fact, I did not. Thank you.
    - wruza 4y ago
      What would that trivial correction change, in your opinion?
- ilyt 4y ago
  > I don't really see the problem with that. Looking at the sql.js demo[1] the WASM binary is 610K (305K compressed/transferred), and it seems to run pretty fast even on my slow laptop. ...and that is small by your standards ?
  - Beltalowda 4y ago
    It's not small; it's also not huge. For the type of website that has use for this it's not really all that much.

samwillis 4y ago

They aren’t working to standardise SQLite, the are working to standardise a low level block based file system api that SQLite and other database engine can use. Thats much more exciting than “standardising SQLite” for the web. The Origin Privet File System api is going to provide the opportunity for any db engine to be used in offline first PWAs. There is no “one size fits all” database engine, that was proved by both WebSQL and IndexedDB. The OPFS in combination with WASM is the correct solution to in browser DBs.

justin66 4y ago
> There is no “one size fits all” database engine, that was proved by both WebSQL and IndexedDB. Thanks to a few very small people, WebSQL was deprecated before it had a chance to prove anything.
- chrismorgan 4y ago
  WebSQL was killed off because the direction it was heading was certain to cause major compatibility problems down the road, and no one was willing to do the work that would be required to avert that (and no one was sold that it would be worth it even at that).
- kevingadd 4y ago
  Most likely prove its utility as an attack surface and source of compatibility issues
- nine_k 4y ago
  They had a,point. Making something a standard when only one implementation of it exists is not prudent. There was (and still is) no independent implementation of SQLite that was battle-tested even a bit.
ilyt 4y ago
> There is no “one size fits all” database engine, that was proved by both WebSQL and IndexedDB. It was not proven at all. WebSQL was deprecated based on "we don't want to standarize on single project". IndexedDB happened because they wanted to standarize on single API (lmao), but it was just too inept. There is no one size fits all but WebSQL fit A LOT of use cases. Low level storage that works for DBs is interesting idea but that also means you need to ship additional megabytes of code with every app.