6 ms·

How would you fix WebSQL? Writing a from-scratch implementation of SQLite seems like the only option, and I don't think Mozilla had the resources for that at th

How would you fix WebSQL? Writing a from-scratch implementation of SQLite seems like the only option, and I don't think Mozilla had the resources for that at the time. I agree the situation sucks, but I'm not sure how to fix it even today.

kylec 11y ago
One thing I never understood is why there needed to be a second implementation of WebSQL. SQLite is public domain, any browser vendor can choose to embed it in what they ship.
- brighteyes 11y ago
  It's a standards process requirement. A standard must have a well-written spec, well-written enough that multiple implementations are possible. Showing multiple implementations that are compatible is enough to prove that. When there is just a single codebase, even if it is open source, it isn't a standard. Now, not everything needs to be a standard, but on the web standardization is a big part of why it is so successful. One example of why a single codebase is bad: Imagine that SQLite version X is the implementation of WebSQL, which all browsers must use. SQLite development is of course continuing, fixing security issues and making it faster, etc. Will browsers just keep using SQLite past version X? What if the SQLite community takes SQLite in a direction that isn't optimal for web browsers? Perhaps web browsers will fork it. What if browsers end up not agreeing on the fork, and some use the original project, and others use the fork - or there are multiple forks? All of that makes it very risky that compatibility issues will appear between browsers. Having a properly specced standard, with multiple compatible implementations, is so far the best way that we have found that can prevent such things.
- BrendanEich 11y ago
  More succinctly: SQLite version whatever is specified only by its C source code, and C is woefully underspecified. Lots of Undefind Behavior on top of the memory-unsafe codebase itself. This is not a specification from which two independently implemented interoperable implementations could be written. Ergo, not a standard.
Lazare 11y ago
Like I said, I'm not sure this was a winnable fight. And yes, writing a good DB implementation from scratch is really, really hard. But... First, keep in mind that lot of this decision was political; if Mozilla had lobbied for it a lot of fudging could have been done. Instead Mozilla made sure that a strict reading of the rules was used by which, yes, WebSQL clearly failed. And no, Mozilla probably didn't have the resources to implement SQLite from scratch, but: 1. They could have worked with others. Apple, quite clearly, was much more invested in WebSQL than IndexedDB. Could they have contributed to a join effort? We'll never know. 2. In particular, the rule about needing two seperate implementations is good, but it could have been fudged here; a bigger problem was the lack of a spec beyond "whatever Sqlite does". Did Mozilla have the resources to clearly specify how a reference implementation should work, even if the only actual implementation remained Sqlite? Maybe. Would that have been enough? Arguably. 3. In the event, IndexedDB was not a real replacement for WebSQL; at most it was an important tool that could be used to build one, some day; a process that is still ongoing (eg, with PouchDB) and has no real end in site. Even if we lacked the resources to really bring WebSQL up to par and re-implement Sqlite from scratch, we apparently lack the resources to bring IndexedDB up to par too. If we're doomed to spend years struggling with broken implementations and poor browser support either way, what did we gain with IndexedDB? Like I said, I don't know that there was a good solution to this. But I feel like Mozilla helped ensure we got a bad one.
- brighteyes 11y ago
  I guess I just disagree that it would have been good for Mozilla to fudge the standardization rules. The rule of having multiple implementations is an important one. And it might have been possible - surely Google and Microsoft, who supported IndexedDB, had the resource to write an alternative implementation. I don't know the history enough to know why they didn't.
  - Lazare 11y ago
    I think I'm more cynical about the process than you are. From my point of view, the process has been intensely political from day one, with rules honoured mainly in the breach. The best retelling of events, I think, is this one: http://nolanlawson.com/2014/04/26/web-sql-database-in-memoriam/ http://nolanlawson.com/2014/04/26/web-sql-database-in-memori... Has some very good details. And it's certainly funny to see what was said during the discussion with how things turned out in practice. Edit: I'd sum it up as "Google and Apple argue for a solution that will work on mobile; Mozilla and Oracle don't care about mobile, are weirdly obsessed with the idea that developers hate SQL, and use political manoeuvring to win the fight". (Oracle's role in this seems especially odd, but perhaps they had some longer term strategy in mind.) In any case, in retrospect ignoring mobile was a bad idea, and ditching SQL seems to have had no real benefit.

kylec 11y ago

One thing I never understood is why there needed to be a second implementation of WebSQL. SQLite is public domain, any browser vendor can choose to embed it in what they ship.

brighteyes 11y ago
It's a standards process requirement. A standard must have a well-written spec, well-written enough that multiple implementations are possible. Showing multiple implementations that are compatible is enough to prove that. When there is just a single codebase, even if it is open source, it isn't a standard. Now, not everything needs to be a standard, but on the web standardization is a big part of why it is so successful. One example of why a single codebase is bad: Imagine that SQLite version X is the implementation of WebSQL, which all browsers must use. SQLite development is of course continuing, fixing security issues and making it faster, etc. Will browsers just keep using SQLite past version X? What if the SQLite community takes SQLite in a direction that isn't optimal for web browsers? Perhaps web browsers will fork it. What if browsers end up not agreeing on the fork, and some use the original project, and others use the fork - or there are multiple forks? All of that makes it very risky that compatibility issues will appear between browsers. Having a properly specced standard, with multiple compatible implementations, is so far the best way that we have found that can prevent such things.
BrendanEich 11y ago
More succinctly: SQLite version whatever is specified only by its C source code, and C is woefully underspecified. Lots of Undefind Behavior on top of the memory-unsafe codebase itself. This is not a specification from which two independently implemented interoperable implementations could be written. Ergo, not a standard.

Lazare 11y ago

Like I said, I'm not sure this was a winnable fight. And yes, writing a good DB implementation from scratch is really, really hard. But... First, keep in mind that lot of this decision was political; if Mozilla had lobbied for it a lot of fudging could have been done. Instead Mozilla made sure that a strict reading of the rules was used by which, yes, WebSQL clearly failed. And no, Mozilla probably didn't have the resources to implement SQLite from scratch, but: 1. They could have worked with others. Apple, quite clearly, was much more invested in WebSQL than IndexedDB. Could they have contributed to a join effort? We'll never know. 2. In particular, the rule about needing two seperate implementations is good, but it could have been fudged here; a bigger problem was the lack of a spec beyond "whatever Sqlite does". Did Mozilla have the resources to clearly specify how a reference implementation should work, even if the only actual implementation remained Sqlite? Maybe. Would that have been enough? Arguably. 3. In the event, IndexedDB was not a real replacement for WebSQL; at most it was an important tool that could be used to build one, some day; a process that is still ongoing (eg, with PouchDB) and has no real end in site. Even if we lacked the resources to really bring WebSQL up to par and re-implement Sqlite from scratch, we apparently lack the resources to bring IndexedDB up to par too. If we're doomed to spend years struggling with broken implementations and poor browser support either way, what did we gain with IndexedDB? Like I said, I don't know that there was a good solution to this. But I feel like Mozilla helped ensure we got a bad one.

brighteyes 11y ago
I guess I just disagree that it would have been good for Mozilla to fudge the standardization rules. The rule of having multiple implementations is an important one. And it might have been possible - surely Google and Microsoft, who supported IndexedDB, had the resource to write an alternative implementation. I don't know the history enough to know why they didn't.
- Lazare 11y ago
  I think I'm more cynical about the process than you are. From my point of view, the process has been intensely political from day one, with rules honoured mainly in the breach. The best retelling of events, I think, is this one: http://nolanlawson.com/2014/04/26/web-sql-database-in-memoriam/ http://nolanlawson.com/2014/04/26/web-sql-database-in-memori... Has some very good details. And it's certainly funny to see what was said during the discussion with how things turned out in practice. Edit: I'd sum it up as "Google and Apple argue for a solution that will work on mobile; Mozilla and Oracle don't care about mobile, are weirdly obsessed with the idea that developers hate SQL, and use political manoeuvring to win the fight". (Oracle's role in this seems especially odd, but perhaps they had some longer term strategy in mind.) In any case, in retrospect ignoring mobile was a bad idea, and ditching SQL seems to have had no real benefit.