3 ms·
If a large graph is needed to be read multiple times, sure memory bandwidth will result in the most performance possible under the context of this workload like
by worthless443 4y ago
If a large graph is needed to be read multiple times, sure memory bandwidth will result in the most performance possible under the context of this workload like interacting with PageRank (and going further with optimization techniques on memory allocation and management, will boost the performance even further).
So to my understanding (and a novice one at that), the graph should be stored on disk first, upon initializing the objects will have to be an one-time copy to volatile memory but I question, memory regions are more likely to yield faults and get corrupt and thus graph stored in-memory is also completely flushed? (unless the results are being saved to disk in-between specific intervals of time?) Does that make any sense?
- mbuda 4y agoI'm not sure I understand the part about corruption. How would data in memory become corrupted? How Memgraph currently works, it stores data in memory, and async starts writing data to disk in small data chunks called deltas, later these chunks are deleted and replaced with the whole graph snapshot (there is also a sync option, but that's slower in terms of committing a transaction, letting the user know data is written, e.g., RocksDB works similarly). All disk-related stuff is purely for durability (recovery after the Memgraph process restarts and all interactions with the disk are made automatically in the background during standard system runtime and startup time).
- worthless443 4y ago> it stores data in memory, and async starts writing data to disk in small data chunks called deltas, later these chunks are deleted and replaced with the whole graph snapshot Thanks, that fairly answers my question of recoverability of in-memory graphs.
- mbuda 4y agoPerfect!