10 ms·

This is funny and sad to me. We had SQLite in the browser[0]. I only did a little bit of work with it but it seemed actually pretty nice. It was torpedoed beca

This is funny and sad to me. We had SQLite in the browser[0]. I only did a little bit of work with it but it seemed actually pretty nice. It was torpedoed because it was SQL-based (and not trendy "key value" and "web scale"). There was the whole excuse that the specification was "whatever SQLite does" and, therefore, not suitable for being a standard. There would be worse things than SQLite upon which to base a standard, all things considered. I still believe it was torpedoed because of lack of trendiness and "not invented here". [0] https://www.w3.org/TR/webdatabase/ https://www.w3.org/TR/webdatabase/

jlongster 5y ago
Yeah, it adds to the absurdity of all of this. Although I do empathize with browers vendors. I worked at Mozilla at the time and was aware that this is a lot of things to think about when integrating something onto the web. I get why it happened, but practically speaking maybe it should have won. It's not like Chrome seems to care much about cross-browser standards these days. I'm hopeful for a storage layer like this though: https://web.dev/storage-foundation/ https://web.dev/storage-foundation/ It might actually be a better outcome if we get a storage layer with close to native performance, and then you can compile and db/lib/etc and it gets to use it.
- 7952 5y ago
  I think offset based file access could be really powerful just based on what people are achieving in the browser with things like Flat buffers, proto buffers and even http range queries.
daleharvey 5y ago
The excuse was that a standard needs to have multiple implementations otherwise we are standardising implementation details and bugs. Hindsight shows that was entirely correct, as SQLite bugs were then found that could be exploited directly via WebSQL, Firefox of course was not vunerable. (https://hub.packtpub.com/an-sqlite-magellan-rce-vulnerability-exposes-billions-of-apps-including-all-chromium-based-browsers/ https://hub.packtpub.com/an-sqlite-magellan-rce-vulnerabilit...) As a sidenote, I worked a lot with the WebSQL API and it was not a very good API in the slightest, immaturity may excuse some of its flaws, and it isnt like Safari did a much better job with IndexedDB, its just a buggy browser and thats where WebSQL was used most, but a large part of the problem is that it was bolting an API that assumed a single threaded client when that is not the reality with web pages where multiple tabs exist
- hitekker 5y ago
  I view the "standards" argument as a red herring for building a NoSQL db in the browser. Which, to this day, is slow, buggy and requires third party libraries to be usable [1] For those who are able to stomach an uncomfortable political history instead of an easy, technical answer, you can take a look at [2]. It's interesting that 7 years later, many the folks who pushed hard to get rid of SQL in favor of NoSQL seem to no longer occupy positions of prominence in the industry. [1] https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API#see_also https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_A... [2] https://nolanlawson.com/2014/04/26/web-sql-database-in-memoriam/ https://nolanlawson.com/2014/04/26/web-sql-database-in-memor...
  - daleharvey 5y ago
    I am familiar with Nolans article, I created PouchDB (the project he is discussing), you seem to have misred the post as it discusses the technical nuance and tradeoffs involved in the decision at many points entirely agreeing with the position against WebSQL. While Nolan came to the a different conclusion than I did (a point he made in the post) he laid out challenges very well and made it very clear there was no obvious technical answer. Regardless of how you view it, the benefit of hindsight shows the exact thing that people warned would happen did in fact happen (a widespread venerability in SQLite exposed across various browsers). Its also a fairly strange point to be personally insulting people involved in the process whose careers are doing perfectly well.
    - hitekker 5y ago
      Edit: Nolan replied before and corrected me. I've removed my misinterpretation and kept my main point below. Back in the day, there were people who strongly suggested MongoDB and IndexedDB were the future, and that PostgreSQL, MySQL, SQLite were trash. I've noticed the folks who rode that hype-train moved into other kinds of occupations that aren't exactly engineering-focused anymore.
      - nolanl 5y ago
devwastaken 5y ago
No, SQLite was not safe to be used with arbitrary queries. There were multiple memory vulnerabilities that allowed escape of the browser sandbox. https://www.sqlite.org/cves.html https://www.sqlite.org/cves.html
- infogulch 5y ago
  This article is about how to defend against potential websql vulns. https://www.sqlite.org/security.html https://www.sqlite.org/security.html
  - nightpool 5y ago
    This article only contains mostly information on adding more limits to avoid DoS—nothing there would harden WebSQL against SQLite vulnerabilities except for SQLITE_DBCONFIG_DEFENSIVE, which was only added in response to the zero-day bugs that were found in Chrome after implementing SQLite.
- SilverRed 5y ago
  Would this not be solvable by just making it all arbitary WASM code? As long as the simple WASM sandbox is secure, it doesn't matter how buggy your internal SQLite implementation is.
kall 5y ago
The sad irony is that if HN is anything to go by, SQLite is super trendy now. I feel like "just sqlite" was really a very practical idea and now, because it has already failed once, we can‘t really try it again, or can we? Having the spec basically be "bundle one particular piece of software into the browser" might not be in the spirit of web standards, but on the other hand SQLite is so widely bundled into everything, does it really matter?
- EMM_386 5y ago
  > The sad irony is that if HN is anything to go by, SQLite is super trendy now. If you look at the recent StackOverflow survey, the majority of the developers only have around 5 years of professional experience in the industry. SQL has been around since the 1970s and is still around in force for reason. There's a good chance a lot of developers, especially the enormous number on the front end, do not have experience with SQL or are just getting into back-end work where they are exposed to it. There have been a lot of SQLite articles recently, as well as node.js libraries to query SQL. I have a hunch this may be why.
  - kall 5y ago
    Yeah I think so too. It‘s a cliche by now to say these things go in cycles but they do. If we like this part of the cycle, let‘s ride it and let SQL get hyped, even though its been around forever and never went away.
    - 411111111111111 5y ago
      What gives you the confidence that it's going on circles? I cannot see our industry repeating the mantra that NoSQL solves all our persistence problems ever again. There are use cases for them, but not everything is a good fit and I'm pretty sure we found that out as an industry. I'm sure there are always going to be individuals that will claim that they're better at everything, but that doesn't really mean anything. There are people that believe in a flat earth as well after all
      - 5y ago
TehShrike 5y ago
I used it back in the day and was quite sad when it disappeared. Nolan Lawson does a good job telling the sordid tale at https://nolanlawson.com/2014/04/26/web-sql-database-in-memoriam/ https://nolanlawson.com/2014/04/26/web-sql-database-in-memor...
- mickael-kerjean 5y ago
  same here. Firefox pulling out from websql is the reason I stopped from using it and recommending it. If the standard wasn't good enough, it should have been improved and refined, not thrown out.
SahAssar 5y ago
The justification was more that the standardization process requires at least 2 independent implementations, and nobody was working on rewriting SQLite from scratch. I would have loved WebSQL, but it is reasonable to require multiple implementations for full standardization.
- coliveira 5y ago
  I don't buy this reasoning because they could have standardized only the interface, letting it open for developers to choose SQLite or some other future implementation. There is no need to standardize SQLite behavior, including bugs.
  - SahAssar 5y ago
    That's not how the process works though. You could argue that they should change the process, but the reasoning behind it seems solid to me: > Implementation experience is required to show that a specification is sufficiently clear, complete, and relevant to market needs, to ensure that independent interoperable implementations of each feature of the specification will be realized. From https://www.w3.org/2015/Process-20150901/#implementation-experience https://www.w3.org/2015/Process-20150901/#implementation-exp...
  - aseipp 5y ago
    What actually happens in that scenario is that one group now writes an adapter on top of SQLite to make it "standardized" to whatever interface you design, increasing the complexity and scope for bugs. To be clear: you're literally reimplementing something that SQLite already does, except probably much worse, in the name of the "standard." Then that adapter is carbon copied by everyone, because again, nobody is going to reimplement an 80kLOC SQL database as well as whatever 10k LOC parser/lexer/typechecking adapter someone wrote, if they can avoid it. Then everyone just uses that forever, and you're back to square 1, using one implementation everywhere, which is the exact situation standards are supposed to avoid anyway. The working group was correct to reject a "compromise" like that because that's never how it works out in practice; it's a submarine suggestion from the start. And a big part of this is all because, as evidenced by numerous responses in this thread, modern computer programmers seem to value their own immediate satisfaction and time over literally every other potential concern, no matter how significant.
kwhitefoot 5y ago
> not trendy "key value" You can put anything in an SQLite record so it can certainly be used as a key-value table. Where at least the values can be arbitrary binary blobs.
zzzeek 5y ago
first thought, was there a "standard" for key/value stores? mongodb the reference implementation? overall, if you looked at HN like five years ago, every DB headline was key/value, mongodb, maybe some cassandra / couchdb, links to the "web scale" cartoon. these days, it's SQL SQL SQL, with a heavy dose of SQLite and PostgreSQL. SQL survived the key/value fad despite the nebulousness of a workable "standard" (yes there's a SQL standard but no vendor DB implements all of it or doesn't add many many features, syntaxes, and behaviors on top of it). In particular SQLite recently seems to look to Postgresql for guidance on new syntaxes such as how it implemented upsert, it's RETURNING syntax is explicitly from PostgreSQL, and it interestingly uses the same "VACUUM" term for db utility cleanup.
- apavlo 5y ago
  > SQL survived the key/value fad SQL has survived every fad since the 1970s: Stonebraker "What Goes Around Comes Around" https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/pape...
- jbergens 5y ago
  I think sql is a bad fit for front end code in js. We may have ended up with another fight over ORMs. The Vietnam war of computer science as I think it was called.
  - SilverRed 5y ago
    How else would you suggest full applications run in the browser. Desktop apps make use of SQLite all the time even when they are mostly server backed. An example is that Telegram maintains a local db which stores the most recently accessed content so you can scroll through past messages and media without having to request them every time you view them.
    - lmm 5y ago
      Desktop apps use SQLite because it's the embedded structured datastore that already exists, not because they want to use SQL. If it was easy to embed, IDK, some kind of file-backed redis, I'm pretty sure a lot of apps would use that instead.
pornel 5y ago
I think the current state is fine. You ship your WASM-blob of SQLite, which has the exact bug-compatible version of SQLite that you've tested your app against. The browsers are not burdened with maintaining a huge API surface that can "break the web". Otherwise you'd have to deal with different versions of SQLite in different browsers, most likely outdated, with many options turned off. SQLite is full of quirks and gotchas, so it's safest to ship your version in your app.
- deleted 5y ago
  [deleted]
- deleted 5y ago
  [deleted]
matharmin 5y ago
SQLite as a library is amazing, SQLite as a spec for a standard web API is not. The issue is not just that there wasn't a competing implementation at the time, it is that you could not feasibly create a competing implementation. The set of features supported by SQLite is _massive_. And even you stick with SQLite, which version? And which extensions? Which features do you remove because they're not safe or not applicable for web? Alternatively you could start from the ground up and create a new database spec that is similar to SQLite, but more limited in scope. But then it will not be directly compatible with SQLite, and need a translation layer. The browser would likely have to re-implement large parts of SQLite for this to work. And you'd miss out on so much of the functionality that makes SQLite attractive to use. Arguably that would still have been better than IndexedDB. All that IndexedDB has going for it, is that it is simple to implement. But even then Safari manages to get the implementation horribly broken so often. What we need instead is to have a low-level storage api that can be used to implement other databases as libraries, including SQLite. The Storage Foundation API, mentioned in the post, might just give us that.
- treis 5y ago
  >SQLite as a library is amazing, SQLite as a spec for a standard web API is not. >The issue is not just that there wasn't a competing implementation at the time, it is that you could not feasibly create a competing implementation. The set of features supported by SQLite is _massive_. And even you stick with SQLite, which version? And which extensions? Which features do you remove because they're not safe or not applicable for web? This seems like a case of perfect being the enemy of good.
jmull 5y ago
Building off what others have pointed out about sqlite not being a good choice for a browser standard, I just want to note that the approach laid out in the article is exactly the right approach to integrate sqlite into a web app: a web app links a version from a well-maintained “distro”. The issues inherent in trying to use an implementation as a specification and standard go away, while a “distro” maintained by experts will lower the bar to entry. I think devs should be happy, not sad. It looks like we’re finally getting it right, and will have a feasible way to add the incredible sqlite to the set tools we have available to make web apps. Hopefully absurd-sql keeps going and browsers adopt a good storage standard.
gunapologist99 5y ago
Another, probably not insurmountable, issue with SQLite in the browser (without having a formal specification that could produce cross-language alternatives): Using SQLite results in a new, built-in reliance on a C library. Even though most languages can use C library bindings, it does present some issues in some build scenarios, such as static builds, and, given the enormous scope of SQLite, would prevent anyone from ever achieving a 100% rust browser.
- Cyberdog 5y ago
  I find this an odd comment. Is a 100% Rust browser something anyone is clamoring for? At any rate, if that really is your goal for some reason, rendering engine is going to be your first problem anyway. Then a JS runtime.
  - lmm 5y ago
    > I find this an odd comment. Is a 100% Rust browser something anyone is clamoring for? A 100% memory-safe browser is definitely something I'm clamouring for. > At any rate, if that really is your goal for some reason, rendering engine is going to be your first problem anyway. Then a JS runtime. Mozilla is working on the rendering engine. Memory-safe JS runtimes already exist (e.g. Nashorn).
gunapologist99 5y ago
> It was torpedoed because it was SQL-based (and not trendy "key value" and "web scale"). This is almost certainly not even close to correct. There are substantial reasons why it wouldn't be a good idea, but this might be the biggest one: it's very hard to adequately sandbox an external C library. (... and, also, Apple probably would prefer that the web didn't exist at all, but that's a different pandora's box...)
- EvanAnderson 5y ago
  My "web scale" bit is snark. The "because it was SQL based" comes from the feeling I got at the time that Mozilla absolutely wouldn't accept anything was SQL-based. I strongly believe anti-SQL attitudes at Mozilla, and little else, are what killed WebSQL. This is also how I justify my opinion that the "no independent implementations" was more of an afterthought excuse, and less of a primary motivation on Mozilla's part. (I'm a little pressed on time here, but I could probably find more examples if I had more time to search... Sorry!) Quoting Jonas Sicking from a W3C IRC log[0]: > we've talked to a lot of developers ... > the feedback we got is that we really don't want SQL Quoting Maciej Stachowiak from the "public-webapps@w3.org" list[1]: > Hixie has said before he's willing to fully spec the SQL dialect used by Web Database. But since Mozilla categorically refuses to implement the spec (apparently regardless of whether the SQL dialect is specified), ... > At the face-to-face, Mozilla representatives said that most if not all of the developers they spoke to said they wanted "anything but SQL" in a storage solution. The comments relating to JOIN in the comments on Mozilla's blog post comparing IndexedDB and WebSQL[2] betray the anti-SQL (and, arguably, anti-relational database) stance at Mozilla. Mozilla's people didn't like SQL, so any excuse for dismissing WebSQL (or even a simplified SQL dialect that didn't have the "quirks" of SQLite) was a foregone conclusion. And now here we are shipping >1MB transpiled WASM payloads around when we could have agreed on a query language feature set implemented in native code using a consistent back-end standard storage format. [0] https://www.w3.org/2009/11/02-webapps-irc https://www.w3.org/2009/11/02-webapps-irc [1] https://lists.w3.org/Archives/Public/public-webapps/2009OctDec/0529.html https://lists.w3.org/Archives/Public/public-webapps/2009OctD... [2] https://hacks.mozilla.org/2010/06/comparing-indexeddb-and-webdatabase/ https://hacks.mozilla.org/2010/06/comparing-indexeddb-and-we...
qwerty456127 5y ago
> It was torpedoed because it was SQL-based (and not trendy "key value" and "web scale"). Whenever I need simple (but indexed) key-value (unless that's a hi-load server-side) I always just use SQLite anyway. I really don't understand why do we need any data storage other than SQLite (and HDF5 perhaps) to exist on the client side.
bayesian_horse 5y ago
I think security had more to do with it...
niutech 5y ago
You can still use WebSQL in Chrome & Safari and use a polyfill backed by IndexedDB for Firefox using YDN-DB: https://yathit.github.io/ydn-db/doc/setup/polyfill.html https://yathit.github.io/ydn-db/doc/setup/polyfill.html

jlongster 5y ago

Yeah, it adds to the absurdity of all of this. Although I do empathize with browers vendors. I worked at Mozilla at the time and was aware that this is a lot of things to think about when integrating something onto the web. I get why it happened, but practically speaking maybe it should have won. It's not like Chrome seems to care much about cross-browser standards these days. I'm hopeful for a storage layer like this though: https://web.dev/storage-foundation/ https://web.dev/storage-foundation/ It might actually be a better outcome if we get a storage layer with close to native performance, and then you can compile and db/lib/etc and it gets to use it.

7952 5y ago
I think offset based file access could be really powerful just based on what people are achieving in the browser with things like Flat buffers, proto buffers and even http range queries.

daleharvey 5y ago

The excuse was that a standard needs to have multiple implementations otherwise we are standardising implementation details and bugs. Hindsight shows that was entirely correct, as SQLite bugs were then found that could be exploited directly via WebSQL, Firefox of course was not vunerable. (https://hub.packtpub.com/an-sqlite-magellan-rce-vulnerability-exposes-billions-of-apps-including-all-chromium-based-browsers/ https://hub.packtpub.com/an-sqlite-magellan-rce-vulnerabilit...) As a sidenote, I worked a lot with the WebSQL API and it was not a very good API in the slightest, immaturity may excuse some of its flaws, and it isnt like Safari did a much better job with IndexedDB, its just a buggy browser and thats where WebSQL was used most, but a large part of the problem is that it was bolting an API that assumed a single threaded client when that is not the reality with web pages where multiple tabs exist

hitekker 5y ago
I view the "standards" argument as a red herring for building a NoSQL db in the browser. Which, to this day, is slow, buggy and requires third party libraries to be usable [1] For those who are able to stomach an uncomfortable political history instead of an easy, technical answer, you can take a look at [2]. It's interesting that 7 years later, many the folks who pushed hard to get rid of SQL in favor of NoSQL seem to no longer occupy positions of prominence in the industry. [1] https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API#see_also https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_A... [2] https://nolanlawson.com/2014/04/26/web-sql-database-in-memoriam/ https://nolanlawson.com/2014/04/26/web-sql-database-in-memor...
- daleharvey 5y ago
  I am familiar with Nolans article, I created PouchDB (the project he is discussing), you seem to have misred the post as it discusses the technical nuance and tradeoffs involved in the decision at many points entirely agreeing with the position against WebSQL. While Nolan came to the a different conclusion than I did (a point he made in the post) he laid out challenges very well and made it very clear there was no obvious technical answer. Regardless of how you view it, the benefit of hindsight shows the exact thing that people warned would happen did in fact happen (a widespread venerability in SQLite exposed across various browsers). Its also a fairly strange point to be personally insulting people involved in the process whose careers are doing perfectly well.
  - hitekker 5y ago
    Edit: Nolan replied before and corrected me. I've removed my misinterpretation and kept my main point below. Back in the day, there were people who strongly suggested MongoDB and IndexedDB were the future, and that PostgreSQL, MySQL, SQLite were trash. I've noticed the folks who rode that hype-train moved into other kinds of occupations that aren't exactly engineering-focused anymore.
    - nolanl 5y ago

devwastaken 5y ago

No, SQLite was not safe to be used with arbitrary queries. There were multiple memory vulnerabilities that allowed escape of the browser sandbox. https://www.sqlite.org/cves.html https://www.sqlite.org/cves.html

infogulch 5y ago
This article is about how to defend against potential websql vulns. https://www.sqlite.org/security.html https://www.sqlite.org/security.html
- nightpool 5y ago
  This article only contains mostly information on adding more limits to avoid DoS—nothing there would harden WebSQL against SQLite vulnerabilities except for SQLITE_DBCONFIG_DEFENSIVE, which was only added in response to the zero-day bugs that were found in Chrome after implementing SQLite.
SilverRed 5y ago
Would this not be solvable by just making it all arbitary WASM code? As long as the simple WASM sandbox is secure, it doesn't matter how buggy your internal SQLite implementation is.

kall 5y ago

The sad irony is that if HN is anything to go by, SQLite is super trendy now. I feel like "just sqlite" was really a very practical idea and now, because it has already failed once, we can‘t really try it again, or can we? Having the spec basically be "bundle one particular piece of software into the browser" might not be in the spirit of web standards, but on the other hand SQLite is so widely bundled into everything, does it really matter?

EMM_386 5y ago
> The sad irony is that if HN is anything to go by, SQLite is super trendy now. If you look at the recent StackOverflow survey, the majority of the developers only have around 5 years of professional experience in the industry. SQL has been around since the 1970s and is still around in force for reason. There's a good chance a lot of developers, especially the enormous number on the front end, do not have experience with SQL or are just getting into back-end work where they are exposed to it. There have been a lot of SQLite articles recently, as well as node.js libraries to query SQL. I have a hunch this may be why.
- kall 5y ago
  Yeah I think so too. It‘s a cliche by now to say these things go in cycles but they do. If we like this part of the cycle, let‘s ride it and let SQL get hyped, even though its been around forever and never went away.
  - 411111111111111 5y ago
    What gives you the confidence that it's going on circles? I cannot see our industry repeating the mantra that NoSQL solves all our persistence problems ever again. There are use cases for them, but not everything is a good fit and I'm pretty sure we found that out as an industry. I'm sure there are always going to be individuals that will claim that they're better at everything, but that doesn't really mean anything. There are people that believe in a flat earth as well after all
    - 5y ago

TehShrike 5y ago

I used it back in the day and was quite sad when it disappeared. Nolan Lawson does a good job telling the sordid tale at https://nolanlawson.com/2014/04/26/web-sql-database-in-memoriam/ https://nolanlawson.com/2014/04/26/web-sql-database-in-memor...

mickael-kerjean 5y ago
same here. Firefox pulling out from websql is the reason I stopped from using it and recommending it. If the standard wasn't good enough, it should have been improved and refined, not thrown out.

SahAssar 5y ago

The justification was more that the standardization process requires at least 2 independent implementations, and nobody was working on rewriting SQLite from scratch. I would have loved WebSQL, but it is reasonable to require multiple implementations for full standardization.

coliveira 5y ago
I don't buy this reasoning because they could have standardized only the interface, letting it open for developers to choose SQLite or some other future implementation. There is no need to standardize SQLite behavior, including bugs.
- SahAssar 5y ago
  That's not how the process works though. You could argue that they should change the process, but the reasoning behind it seems solid to me: > Implementation experience is required to show that a specification is sufficiently clear, complete, and relevant to market needs, to ensure that independent interoperable implementations of each feature of the specification will be realized. From https://www.w3.org/2015/Process-20150901/#implementation-experience https://www.w3.org/2015/Process-20150901/#implementation-exp...
- aseipp 5y ago
  What actually happens in that scenario is that one group now writes an adapter on top of SQLite to make it "standardized" to whatever interface you design, increasing the complexity and scope for bugs. To be clear: you're literally reimplementing something that SQLite already does, except probably much worse, in the name of the "standard." Then that adapter is carbon copied by everyone, because again, nobody is going to reimplement an 80kLOC SQL database as well as whatever 10k LOC parser/lexer/typechecking adapter someone wrote, if they can avoid it. Then everyone just uses that forever, and you're back to square 1, using one implementation everywhere, which is the exact situation standards are supposed to avoid anyway. The working group was correct to reject a "compromise" like that because that's never how it works out in practice; it's a submarine suggestion from the start. And a big part of this is all because, as evidenced by numerous responses in this thread, modern computer programmers seem to value their own immediate satisfaction and time over literally every other potential concern, no matter how significant.

kwhitefoot 5y ago

> not trendy "key value" You can put anything in an SQLite record so it can certainly be used as a key-value table. Where at least the values can be arbitrary binary blobs.

zzzeek 5y ago

first thought, was there a "standard" for key/value stores? mongodb the reference implementation? overall, if you looked at HN like five years ago, every DB headline was key/value, mongodb, maybe some cassandra / couchdb, links to the "web scale" cartoon. these days, it's SQL SQL SQL, with a heavy dose of SQLite and PostgreSQL. SQL survived the key/value fad despite the nebulousness of a workable "standard" (yes there's a SQL standard but no vendor DB implements all of it or doesn't add many many features, syntaxes, and behaviors on top of it). In particular SQLite recently seems to look to Postgresql for guidance on new syntaxes such as how it implemented upsert, it's RETURNING syntax is explicitly from PostgreSQL, and it interestingly uses the same "VACUUM" term for db utility cleanup.

apavlo 5y ago
> SQL survived the key/value fad SQL has survived every fad since the 1970s: Stonebraker "What Goes Around Comes Around" https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/pape...
jbergens 5y ago
I think sql is a bad fit for front end code in js. We may have ended up with another fight over ORMs. The Vietnam war of computer science as I think it was called.
- SilverRed 5y ago
  How else would you suggest full applications run in the browser. Desktop apps make use of SQLite all the time even when they are mostly server backed. An example is that Telegram maintains a local db which stores the most recently accessed content so you can scroll through past messages and media without having to request them every time you view them.
  - lmm 5y ago
    Desktop apps use SQLite because it's the embedded structured datastore that already exists, not because they want to use SQL. If it was easy to embed, IDK, some kind of file-backed redis, I'm pretty sure a lot of apps would use that instead.

pornel 5y ago

I think the current state is fine. You ship your WASM-blob of SQLite, which has the exact bug-compatible version of SQLite that you've tested your app against. The browsers are not burdened with maintaining a huge API surface that can "break the web". Otherwise you'd have to deal with different versions of SQLite in different browsers, most likely outdated, with many options turned off. SQLite is full of quirks and gotchas, so it's safest to ship your version in your app.

deleted 5y ago
[deleted]
deleted 5y ago
[deleted]

matharmin 5y ago

SQLite as a library is amazing, SQLite as a spec for a standard web API is not. The issue is not just that there wasn't a competing implementation at the time, it is that you could not feasibly create a competing implementation. The set of features supported by SQLite is _massive_. And even you stick with SQLite, which version? And which extensions? Which features do you remove because they're not safe or not applicable for web? Alternatively you could start from the ground up and create a new database spec that is similar to SQLite, but more limited in scope. But then it will not be directly compatible with SQLite, and need a translation layer. The browser would likely have to re-implement large parts of SQLite for this to work. And you'd miss out on so much of the functionality that makes SQLite attractive to use. Arguably that would still have been better than IndexedDB. All that IndexedDB has going for it, is that it is simple to implement. But even then Safari manages to get the implementation horribly broken so often. What we need instead is to have a low-level storage api that can be used to implement other databases as libraries, including SQLite. The Storage Foundation API, mentioned in the post, might just give us that.

treis 5y ago
>SQLite as a library is amazing, SQLite as a spec for a standard web API is not. >The issue is not just that there wasn't a competing implementation at the time, it is that you could not feasibly create a competing implementation. The set of features supported by SQLite is _massive_. And even you stick with SQLite, which version? And which extensions? Which features do you remove because they're not safe or not applicable for web? This seems like a case of perfect being the enemy of good.

jmull 5y ago

Building off what others have pointed out about sqlite not being a good choice for a browser standard, I just want to note that the approach laid out in the article is exactly the right approach to integrate sqlite into a web app: a web app links a version from a well-maintained “distro”. The issues inherent in trying to use an implementation as a specification and standard go away, while a “distro” maintained by experts will lower the bar to entry. I think devs should be happy, not sad. It looks like we’re finally getting it right, and will have a feasible way to add the incredible sqlite to the set tools we have available to make web apps. Hopefully absurd-sql keeps going and browsers adopt a good storage standard.

gunapologist99 5y ago

Another, probably not insurmountable, issue with SQLite in the browser (without having a formal specification that could produce cross-language alternatives): Using SQLite results in a new, built-in reliance on a C library. Even though most languages can use C library bindings, it does present some issues in some build scenarios, such as static builds, and, given the enormous scope of SQLite, would prevent anyone from ever achieving a 100% rust browser.

Cyberdog 5y ago
I find this an odd comment. Is a 100% Rust browser something anyone is clamoring for? At any rate, if that really is your goal for some reason, rendering engine is going to be your first problem anyway. Then a JS runtime.
- lmm 5y ago
  > I find this an odd comment. Is a 100% Rust browser something anyone is clamoring for? A 100% memory-safe browser is definitely something I'm clamouring for. > At any rate, if that really is your goal for some reason, rendering engine is going to be your first problem anyway. Then a JS runtime. Mozilla is working on the rendering engine. Memory-safe JS runtimes already exist (e.g. Nashorn).

gunapologist99 5y ago

> It was torpedoed because it was SQL-based (and not trendy "key value" and "web scale"). This is almost certainly not even close to correct. There are substantial reasons why it wouldn't be a good idea, but this might be the biggest one: it's very hard to adequately sandbox an external C library. (... and, also, Apple probably would prefer that the web didn't exist at all, but that's a different pandora's box...)

EvanAnderson 5y ago
My "web scale" bit is snark. The "because it was SQL based" comes from the feeling I got at the time that Mozilla absolutely wouldn't accept anything was SQL-based. I strongly believe anti-SQL attitudes at Mozilla, and little else, are what killed WebSQL. This is also how I justify my opinion that the "no independent implementations" was more of an afterthought excuse, and less of a primary motivation on Mozilla's part. (I'm a little pressed on time here, but I could probably find more examples if I had more time to search... Sorry!) Quoting Jonas Sicking from a W3C IRC log[0]: > we've talked to a lot of developers ... > the feedback we got is that we really don't want SQL Quoting Maciej Stachowiak from the "public-webapps@w3.org" list[1]: > Hixie has said before he's willing to fully spec the SQL dialect used by Web Database. But since Mozilla categorically refuses to implement the spec (apparently regardless of whether the SQL dialect is specified), ... > At the face-to-face, Mozilla representatives said that most if not all of the developers they spoke to said they wanted "anything but SQL" in a storage solution. The comments relating to JOIN in the comments on Mozilla's blog post comparing IndexedDB and WebSQL[2] betray the anti-SQL (and, arguably, anti-relational database) stance at Mozilla. Mozilla's people didn't like SQL, so any excuse for dismissing WebSQL (or even a simplified SQL dialect that didn't have the "quirks" of SQLite) was a foregone conclusion. And now here we are shipping >1MB transpiled WASM payloads around when we could have agreed on a query language feature set implemented in native code using a consistent back-end standard storage format. [0] https://www.w3.org/2009/11/02-webapps-irc https://www.w3.org/2009/11/02-webapps-irc [1] https://lists.w3.org/Archives/Public/public-webapps/2009OctDec/0529.html https://lists.w3.org/Archives/Public/public-webapps/2009OctD... [2] https://hacks.mozilla.org/2010/06/comparing-indexeddb-and-webdatabase/ https://hacks.mozilla.org/2010/06/comparing-indexeddb-and-we...

qwerty456127 5y ago

> It was torpedoed because it was SQL-based (and not trendy "key value" and "web scale"). Whenever I need simple (but indexed) key-value (unless that's a hi-load server-side) I always just use SQLite anyway. I really don't understand why do we need any data storage other than SQLite (and HDF5 perhaps) to exist on the client side.

bayesian_horse 5y ago

I think security had more to do with it...

niutech 5y ago

You can still use WebSQL in Chrome & Safari and use a polyfill backed by IndexedDB for Firefox using YDN-DB: https://yathit.github.io/ydn-db/doc/setup/polyfill.html https://yathit.github.io/ydn-db/doc/setup/polyfill.html