4 ms·
For my performance book, I looked at some sample code for converting public transport data in CSV format to an embedded SQLite DB for use on mobile. A little bi
by mpweiher 8y ago
For my performance book, I looked at some sample code for converting public transport data in CSV format to an embedded SQLite DB for use on mobile. A little bit of data optimization took the time from 22 minutes to under a second, or ~1000x, for well over 100MB of Source data.
The target data went fro almost 200MB of SQLite to 7MB of binary that could just be mapped into memory. Oh, and lookup on the device also became 1000x faster.
There is a LOT of that sort of stuff out there, our “standard” approaches are often highly inappropriate for a wide variety of problems.
- tzahola 8y agoHa! I did the same for my BSc thesis: On-device route planning from GTFS data for Budapest’s public transport network iOS used to kill processes above 50 megs of RAM usage. Good times!
- mpweiher 8y agoCool! Link?
- tzahola 8y agoUnfortunately it's no longer in the AppStore (ubiquitous cellular data made it obsolete) and the thesis itself is written in Hungarian :D
- mpweiher 8y ago:-( Not so sure it's really obsolete, because connections are often spotty and local data can provide some initial information while waiting for the servers. Github?
- tzahola 8y agoIt wasn't put on GitHub back then, and I don't want to publish it now. But I've sent it to you in email - check it out if you're interested.
- mcguire 8y agoNormal developer behavior has gone from "optimize everything for machine usage" (cpu time, memory, etc.) to "optimize everything for developer convenience". The former is frequently inappropriate, but the latter is, as well. (And some would say that it then went to "optimize everything for resume keywords," which is almost always inappropriate, but I don't want to be too cynical.)
- mpweiher 8y agoOh, it was also less code.