3 ms·

Why run a database in WASM when IndexedDB exists? Browsers already have a database built in, I don't see the need to download another one.

isodev 2y ago
It feels very impractical indeed. Also the size of the binary to load the compiled wasm. All this would be much better done on the server and if really needed, users may be given a way to download results (ideally with their own preferred tool for fetching files)
- jampekka 2y ago
  A server sounds quite inpractical if you could otherwise serve the application statically. Or offline. Also having user data on server causes problems with privacy etc.
  - isodev 2y ago
    Well the data has to come from somewhere right? If the goal is to facilitate client-side (bring your own data) scenarios, I'd make a proper native desktop app and take full advantage of the system. A hybrid something running in the browser feels like a compromise between both solutions.
    - jampekka 2y ago
      Why would you make a "proper" native desktop app (or more specifically apps for every platform you want to support) if you can do it with a PWA (which you can do for vast majority of apps).
      - isodev 2y ago
        Because PWAs are limited in more ways than practical to list here, just to name a few - the browser they run in, restricted by the availability and quality of internet connection, they're not sustainable as build tools don't even bother with backwards compatibility given the pace of evolution, practically no user control over a pwa "app" running in the browser. Remember, PWAs exist as a work-around for gatekeepy OS vendors making it hard to create cross-platform apps. PWAs don't resolve anything - PWAs move the problem to the browser space, where (today at least) we only have closed, proprietary, very-much revenue-driven browser implementations. The related web standards have also largely been influenced by FAANGS as the likes of Google wanting to turn "the web as their webstore".
        jampekka 2y ago
        Please do some listing. Most of the time people worrying about PWA limitations are decade or so out of date of what the contemporary web apis are capable of. For example, PWAs are trivial to cache with service workers (a lot easier than app install for both the developer and the user), and after the first load they work totally offline. You have almost total control of the PWA. Just launch the browser DevTools and you can even edit the code on the fly. With e.g. iOS apps you have no control or even knowledge of what the apps are doing, unless you manage decrypt them, manage to decompile or inspect the binary and/or root your own device through exploits. All major browser engines are open source and the APIs are open standards (even Google deprecated their proprietary APIs). PWAs are way less tied to likes of Google than native apps. I think you should update your information about browser APIs. You'll be pleasantly surprised.
jampekka 2y ago
Because of the IndexedDB API?
- tehbeard 2y ago
  That and Apple breaking it in new and "exciting" ways...
paulgb 2y ago
IndexedDB is a key/value store; DuckDB is a relational analytic database. You would use IndexedDB for something like application data, and DuckDB for (e.g.) a data-heavy dashboard or ad-hoc query tool.
- niutech 2y ago
  You can leverage SQL on top of IndexedDB by using AlaSQL (https://github.com/AlaSQL/alasql https://github.com/AlaSQL/alasql) or absurd-sql (https://github.com/jlongster/absurd-sql https://github.com/jlongster/absurd-sql).
- quotemstr 2y ago
  So what? We should be structuring a lot of databases as layers on top of transactional key value stores. We should be able to swap out the key value store without changing the higher level query engine.
  - jampekka 2y ago
    We maybe[1] should, but we are not. AFAIK there's no relational query engine for IndexedDB. Also, there are a lot more people who can get things done with SQL than using indexed KV-stores. KV stores tending to have horrible APIs doesn't help the situation. [1] E.g. SQLite started with a gdbm backend, but later rolled their own because of limitations caused by it.
    - niutech 2y ago
      There is SQL on top of IndexedDB: absurd-sql or AlaSQL.
  - paulgb 2y ago
    When you build that, I will upvote the HN post to that too ;)
    - quotemstr 2y ago
      Don't tempt me. :-) This is what SQLite 4 was supposed to have been, BTW.
pjc50 2y ago
IndexedDB should have been SQLite, but for various squishy reasons it isn't, it's something less good.
Retr0id 2y ago
Per [0], surprisingly, you can beat IndexedDB performance by layering another DB engine on top. Discussed on HN: https://news.ycombinator.com/item?id=28156831 https://news.ycombinator.com/item?id=28156831 [0] https://jlongster.com/future-sql-web https://jlongster.com/future-sql-web
jt2190 2y ago
Maybe you’re thinking of WebSQL? https://caniuse.com/sql-storage https://caniuse.com/sql-storage
robmccoll 2y ago
If you use https://dexie.org/ https://dexie.org/ on top of IndexDB, you can turn it into a fairly useful document store with secondary indices, an improved query building interface, and reactive queries. It's doing exactly what you would expect on top of IndexDB if you did it yourself, but saves a ton of boilerplate. They have a sync service to go with it, but in my usage I managed that myself.
lmeyerov 2y ago
Columnar analytics are slow in non-columnar engines

isodev 2y ago

It feels very impractical indeed. Also the size of the binary to load the compiled wasm. All this would be much better done on the server and if really needed, users may be given a way to download results (ideally with their own preferred tool for fetching files)

jampekka 2y ago
A server sounds quite inpractical if you could otherwise serve the application statically. Or offline. Also having user data on server causes problems with privacy etc.
- isodev 2y ago
  Well the data has to come from somewhere right? If the goal is to facilitate client-side (bring your own data) scenarios, I'd make a proper native desktop app and take full advantage of the system. A hybrid something running in the browser feels like a compromise between both solutions.
  - jampekka 2y ago
    Why would you make a "proper" native desktop app (or more specifically apps for every platform you want to support) if you can do it with a PWA (which you can do for vast majority of apps).
    - isodev 2y ago
      Because PWAs are limited in more ways than practical to list here, just to name a few - the browser they run in, restricted by the availability and quality of internet connection, they're not sustainable as build tools don't even bother with backwards compatibility given the pace of evolution, practically no user control over a pwa "app" running in the browser. Remember, PWAs exist as a work-around for gatekeepy OS vendors making it hard to create cross-platform apps. PWAs don't resolve anything - PWAs move the problem to the browser space, where (today at least) we only have closed, proprietary, very-much revenue-driven browser implementations. The related web standards have also largely been influenced by FAANGS as the likes of Google wanting to turn "the web as their webstore".
      - jampekka 2y ago
        Please do some listing. Most of the time people worrying about PWA limitations are decade or so out of date of what the contemporary web apis are capable of. For example, PWAs are trivial to cache with service workers (a lot easier than app install for both the developer and the user), and after the first load they work totally offline. You have almost total control of the PWA. Just launch the browser DevTools and you can even edit the code on the fly. With e.g. iOS apps you have no control or even knowledge of what the apps are doing, unless you manage decrypt them, manage to decompile or inspect the binary and/or root your own device through exploits. All major browser engines are open source and the APIs are open standards (even Google deprecated their proprietary APIs). PWAs are way less tied to likes of Google than native apps. I think you should update your information about browser APIs. You'll be pleasantly surprised.

jampekka 2y ago

Because of the IndexedDB API?

tehbeard 2y ago
That and Apple breaking it in new and "exciting" ways...

paulgb 2y ago

IndexedDB is a key/value store; DuckDB is a relational analytic database. You would use IndexedDB for something like application data, and DuckDB for (e.g.) a data-heavy dashboard or ad-hoc query tool.

niutech 2y ago
You can leverage SQL on top of IndexedDB by using AlaSQL (https://github.com/AlaSQL/alasql https://github.com/AlaSQL/alasql) or absurd-sql (https://github.com/jlongster/absurd-sql https://github.com/jlongster/absurd-sql).
quotemstr 2y ago
So what? We should be structuring a lot of databases as layers on top of transactional key value stores. We should be able to swap out the key value store without changing the higher level query engine.
- jampekka 2y ago
  We maybe[1] should, but we are not. AFAIK there's no relational query engine for IndexedDB. Also, there are a lot more people who can get things done with SQL than using indexed KV-stores. KV stores tending to have horrible APIs doesn't help the situation. [1] E.g. SQLite started with a gdbm backend, but later rolled their own because of limitations caused by it.
  - niutech 2y ago
    There is SQL on top of IndexedDB: absurd-sql or AlaSQL.
- paulgb 2y ago
  When you build that, I will upvote the HN post to that too ;)
  - quotemstr 2y ago
    Don't tempt me. :-) This is what SQLite 4 was supposed to have been, BTW.

pjc50 2y ago

IndexedDB should have been SQLite, but for various squishy reasons it isn't, it's something less good.

Retr0id 2y ago

Per [0], surprisingly, you can beat IndexedDB performance by layering another DB engine on top. Discussed on HN: https://news.ycombinator.com/item?id=28156831 https://news.ycombinator.com/item?id=28156831 [0] https://jlongster.com/future-sql-web https://jlongster.com/future-sql-web

jt2190 2y ago

Maybe you’re thinking of WebSQL? https://caniuse.com/sql-storage https://caniuse.com/sql-storage

robmccoll 2y ago

If you use https://dexie.org/ https://dexie.org/ on top of IndexDB, you can turn it into a fairly useful document store with secondary indices, an improved query building interface, and reactive queries. It's doing exactly what you would expect on top of IndexDB if you did it yourself, but saves a ton of boilerplate. They have a sync service to go with it, but in my usage I managed that myself.

lmeyerov 2y ago

Columnar analytics are slow in non-columnar engines