3 ms·
I would use the term 'unrealistic' vs. unscientific. Points not discussed: Filesystems are not equal, thus portability may be a concern. For example, filesystem
by contingencies 5y ago
I would use the term 'unrealistic' vs. unscientific. Points not discussed: Filesystems are not equal, thus portability may be a concern. For example, filesystems often have path or filename length limits, number of files or directory entries per directory limits, and cannot always store all characters as part of path or file names. Losses may not be evident when they occur or exhibit well-defined behavior, particularly if arbitrary filesystem-related tooling is in use. Filesystems do not permit complex indexing structures, rather typically use only an hierarchical indexing paradigm plus OS-level caching. Efficiency in real-world use is dependent on the approximate ratio of disparate reads, writes, and read-and-write access paths you will actually use (typically not just a contrived direct FS-hierarchical single-record write-or-retrieval) also noting the OS cache settings and backend block storage specified, and whether there is any chance of contention from other processes. Finally, persistence expectations are critical. If you want all writes to last you are in a very different boat to if you just need raw performance. In the latter case, never touching the disk during normal operations and only periodically flushing memory state to disk is a clear win. With SQLite3 you can do this easily via https://sqlite.org/inmemorydb.html https://sqlite.org/inmemorydb.html or slightly less efficiently at the OS level by providing a RAM disk for it to work on. OP could try this and add to the benchmarks. Finally, filesystems do not permit the additional features of SQLite such as triggers and well defined error and write persist behavior, which are well advised to prevent entire categories of issues on datastores anticipating a degree of longevity and criticality. Oh, and SQLite3/FS performance via a bunch of go libraries isn't necessarily the same as raw.