4 ms·

I don't understand what the big deal is with grouping a bunch of writes into a transaction? Isn't that how you use a DB?

deleted 5y ago
[deleted]
sieabah 5y ago
That's the first thing I noticed when reading, it's complaining that if you don't batch your writes the performance suffers substantially. It warrants an audible "duh". More work is always going to run slower.
setr 5y ago
I don’t think you should be expecting a local in-memory db to require batching for reasonable performance; the main reason it’s a problem for normal DB’s is the network — you’re constantly waiting for the roundtrip. Which you obviously still have for an in-memory DB, but you should expect it to require a lot more picked low-hanging fruit before that becomes your bottleneck.
- couchand 5y ago
  Sibling already pointed it out, but it's worth clarifying: IndexedDB is most definitely NOT and in-memory database.
- sieabahlpark 5y ago
  IndexedDB isn't an in memory DB?
- vlovich123 5y ago
  Each transaction typically has a non trivial amount of overhead as committing it requires you to verify that there wasn’t a conflict. Why would you expect that you wouldn’t need to do this just because it’s an in-memory database? You might still be modifying it concurrently through multiple tabs open to the same site modifying the same DB or because you are interleaving “concurrent” actions for some reason within a page. Has nothing to do with I/O to my knowledge. Try this experiment out with any transactional in-memory tables on any database and they should all show some amount of degradation (although maybe not necessarily quite as severe as IndexDB - 10k inserted rows/s seems below what SQLite should be putting out).
- stingraycharles 5y ago
  I am fairly certain that the reason you want to batch writes in a normal database is not because of RTT latency, but rather transaction overhead; e.g. indexes have to be updated only once, rather than thousands of times. Disks have to be flushed only once, etc. I am fairly certain the issues are similar for IndexedDB.
hutzlibu 5y ago
Hm, but what if I have lots of transactions, going to lots of different object stores? Because every object store requires its own transaction. (grouping there does not seem to help) The solution would probably be, to not have lots of object stores, but this would have made my db setup much more complicated. But FileSystem API seems to finally getting broad support.
- tehbeard 5y ago
  You can "open" multiple stores in a transaction so there's no need to have dozens for a dozen stores. There's some onus on you to work out the batching as IndexedDB is quite low level for what we usually get in the browser. > But FileSystem API seems to finally getting broad support. I'm not seeing much beyond Chrome & it's associates for it? That said that could just be because "filesystem" as a word is mentioned in a dozen different web APIs from allowing file input reading locally, to the deprecated API chrome had for local file access, to the new thing that came out of "project Fugu" for PWAs to have native file access and I'm missing which one is which in terms of support & docs.
  - hutzlibu 5y ago
    Thanks, I will look into that. (Is this a recent addition to the API, or did I just missed that option from the beginning? My db code is pretty much unchanged since a few years) And yes, file system api is confusing, I was also unsure for a moment to have looked at the wrong API, but I was refering to there, the local sandbox filesystem: https://developer.mozilla.org/en-US/docs/Web/API/FileSystem https://developer.mozilla.org/en-US/docs/Web/API/FileSystem Which says it mostly works now, but I never worked with it, because of chrome only, despite it seemed so much better suited for my use case than indexedDB.

deleted 5y ago

[deleted]

sieabah 5y ago

That's the first thing I noticed when reading, it's complaining that if you don't batch your writes the performance suffers substantially. It warrants an audible "duh". More work is always going to run slower.

setr 5y ago

I don’t think you should be expecting a local in-memory db to require batching for reasonable performance; the main reason it’s a problem for normal DB’s is the network — you’re constantly waiting for the roundtrip. Which you obviously still have for an in-memory DB, but you should expect it to require a lot more picked low-hanging fruit before that becomes your bottleneck.

couchand 5y ago
Sibling already pointed it out, but it's worth clarifying: IndexedDB is most definitely NOT and in-memory database.
sieabahlpark 5y ago
IndexedDB isn't an in memory DB?
vlovich123 5y ago
Each transaction typically has a non trivial amount of overhead as committing it requires you to verify that there wasn’t a conflict. Why would you expect that you wouldn’t need to do this just because it’s an in-memory database? You might still be modifying it concurrently through multiple tabs open to the same site modifying the same DB or because you are interleaving “concurrent” actions for some reason within a page. Has nothing to do with I/O to my knowledge. Try this experiment out with any transactional in-memory tables on any database and they should all show some amount of degradation (although maybe not necessarily quite as severe as IndexDB - 10k inserted rows/s seems below what SQLite should be putting out).
stingraycharles 5y ago
I am fairly certain that the reason you want to batch writes in a normal database is not because of RTT latency, but rather transaction overhead; e.g. indexes have to be updated only once, rather than thousands of times. Disks have to be flushed only once, etc. I am fairly certain the issues are similar for IndexedDB.

hutzlibu 5y ago

Hm, but what if I have lots of transactions, going to lots of different object stores? Because every object store requires its own transaction. (grouping there does not seem to help) The solution would probably be, to not have lots of object stores, but this would have made my db setup much more complicated. But FileSystem API seems to finally getting broad support.

tehbeard 5y ago
You can "open" multiple stores in a transaction so there's no need to have dozens for a dozen stores. There's some onus on you to work out the batching as IndexedDB is quite low level for what we usually get in the browser. > But FileSystem API seems to finally getting broad support. I'm not seeing much beyond Chrome & it's associates for it? That said that could just be because "filesystem" as a word is mentioned in a dozen different web APIs from allowing file input reading locally, to the deprecated API chrome had for local file access, to the new thing that came out of "project Fugu" for PWAs to have native file access and I'm missing which one is which in terms of support & docs.
- hutzlibu 5y ago
  Thanks, I will look into that. (Is this a recent addition to the API, or did I just missed that option from the beginning? My db code is pretty much unchanged since a few years) And yes, file system api is confusing, I was also unsure for a moment to have looked at the wrong API, but I was refering to there, the local sandbox filesystem: https://developer.mozilla.org/en-US/docs/Web/API/FileSystem https://developer.mozilla.org/en-US/docs/Web/API/FileSystem Which says it mostly works now, but I never worked with it, because of chrome only, despite it seemed so much better suited for my use case than indexedDB.