4 ms·
I know that writing to log files is hard drive intensive. I also know that the preferred method is to write to a database No. Databases are far more disk inte
by cperciva 17y ago
I know that writing to log files is hard drive intensive. I also know that the preferred method is to write to a database
No. Databases are far more disk intensive than logging; they use vastly higher disk bandwidth, and depending on the database, often use lots of disk seeks too.
- vicaya 17y agoIt's generally true for traditional DBs, a few new generation DBs have options to buffer writes in memory and achieve durability via in memory replication in other nodes and only writes to disk sequentially in large chunks, which is better than simple logging as you'd have to sync to avoid data loss upon hardware failure.
- geuis 17y agoDo you have any recommendations for software I should use for a db? I like the idea of a db because doing queries on it will be easier.
- cperciva 17y agoIf you're going to consider a database which buffers writes in memory, you should also consider logging which buffers writes in memory.
- vicaya 17y agoMaybe my comment was not clear enough: buffering writes in memory on a single node alone has no durability to speak of. You need either a DFS or distributed DB that does replication of the buffers on different nodes to provide some reasonable durability.
- geuis 17y agoOh man, I had no idea.
- scott_s 17y agoThink about it this way: will the database write more or less information than writing the raw data to the disk? Will using the database require more or less instructions than writing the raw data to the disk? I know it's said all the time, but it's said for a reason: do the simple thing first. If after measurement it turns out to be slow, optimize.
- scumola 17y agoYea, log to a file and turn on write-caching on your HD controller.