3 ms·
I wonder if this could be improved by just using the standard C library strftime(3) instead of going through sqlite?
by 0x0 13y ago
I wonder if this could be improved by just using the standard C library strftime(3) instead of going through sqlite?
- mjn 13y agoIf date formatting is a bottleneck for me (it is surprisingly often, because it's very slow in some languages) I typically just run it through the command-line program 'convdate' [1] from crush-tools, which is more or less just a wrapper around strptime+strftime. [1] https://code.google.com/p/crush-tools/wiki/ConvdateUserDocs https://code.google.com/p/crush-tools/wiki/ConvdateUserDocs
- justincormack 13y agoIf it is a bottleneck shelling out is not a great solution...
- pestaa 13y agoShelling out for each piece of data is indeed not great. Shelling out for batch-processing loads of data is on the other hand great.
- mjn 13y agoYes, and that's the workload convdate is intended for: it batch-converts an entire column of a tab-delimited file. The larger crush-tools suite is intended for unix-style batch processing of tabular data, but fills in some functionality that the classic set of POSIX tools (cut, sort, paste, join, etc.) didn't cover.
- deleted 13y ago[deleted]
- jurre 13y agoI was wondering the same thing since that's also what apple recommends in situations like these. This is what I got on the same hardware: strptime_l took 58.803 seconds NSDateFormatter took 107.570 seconds sqlite3 took 7.022 seconds And with MishraAnurag's suggestion of using timegm instead of mktime: strptime_l took 21.656 seconds NSDateFormatter took 108.163 seconds sqlite3 took 7.096 seconds
- coldcode 13y agoWhy not see what sqlite is doing and do something in C yourself that solves the actual problem. It's not surprising that a general purpose Obj-C (or any language) class isn't terribly fast at one specific thing.
- jurre 13y agoYeah that would probably be the way to go ultimately if you're doing a lot of date parsing, I agree!
- MishraAnurag 13y agoAre you using mktime to get the unix timestamp? That might be the slower part as opposed to strptime.
- jurre 13y agoI am, source is here: https://gist.github.com/jurre/6475263 https://gist.github.com/jurre/6475263 My c is quite poor so if you have any suggestions on how to improve I'd love to hear them!
- MishraAnurag 13y agoI'd suggest using timegm instead of mktime, or set the TZ environment variable to UTC to ensure all implementations return an identical date. I ran the same tests and found that the strptime was quite fast, but gmtime was taking most of the time. To speed that up, you could borrow SQLite's implementation. Checkout the computeJD function from SQLite's date.c - http://www.sqlite.org/src/doc/trunk/src/date.c http://www.sqlite.org/src/doc/trunk/src/date.c
- jurre 13y agotimegm actually already makes a huge difference, thanks! Might be useful to make a small fast date parsing library based on the sqlite source code.