3 ms·
Large databases are thousands of rows? My small database already has a few tens of thousands. Who considers a few thousand rows large?
by Aeolun 5y ago
Large databases are thousands of rows? My small database already has a few tens of thousands.
Who considers a few thousand rows large?
- hanche 5y agoOne person’s small is another person’s large. It all comes down to your typical use cases; no surprise there. I have lots of tiny sqlite databases myself, so for me, a few tens of thousands rows is large. But at least I am well aware that multi-gigabyte sqlite databases can and do exist.
- mathaou 5y agoOptimizations like that will be part of the next alpha release after the one I'm working on.
- Aeolun 5y agoI just honestly cannot imagine how it could start to be slow at a few thousand rows. Calling it an optimization feels wrong somehow. Glad you hear you are planning to look at it though :)
- laumars 5y agoLarger querying larger databases should be a problem for the RDBMS not for the UI wrapper. So I’d wager he is processing every record on the db irrespective of whether it’s visible on screen or not. It would explain the TUI panic posted elsewhere too.
- mathaou 5y agoI have no control over which portions of the screen get updated when without updating the library I'm using.
- laumars 5y agoOne way to optimise this is to pull back the entire SQL results in an [][]interface{} (row, column) but then only pass a sub-slice to your TUI (depending on how many rows are visible). I assume your term library doesn’t create the actual table? So you’d know the term height and you’re then plotting the cells? If that’s not the case then I don’t think you’ll get much further with this project without switching to another term library (or rolling your own — which is actually easier than it sounds).
- mathaou 5y agoLol I just did this. Now way faster and can handle arbitrary amounts of data (there is some flickering on scroll for large data sets I'm trying to sort out).
- zerr 5y agoFor SQLite seems reasonable? I remember it advertised as ini file replacement as well :)
- laumars 5y agoSqlite easily manages millions of rows. I’ve used it in the past as part of a lazier implementation of an ELK-like stack and it worked fine.