5 ms·
rsync, rclone really depends on what formats the data is in, how it will be used, how long it needs to last, how long the code that reads and writes the data w
by sitkack 6y ago
rsync, rclone
really depends on what formats the data is in, how it will be used, how long it needs to last, how long the code that reads and writes the data will last, how much out of band information vs inband information will be sent along with the data.
You talked about movement, movement is the last of your worries, formats and provenance should rank higher on your list of concerns.
My gotos are sqlite, xml and avro. Try to keep things under 2GB so you don't run into lame data transfer issues.