3 ms·

Thank you! I tried hard to keep the code very simple sacrificing some 20%-scenario requirements (such as automatic conflict resolution). I am using this to syn

Thank you! I tried hard to keep the code very simple sacrificing some 20%-scenario requirements (such as automatic conflict resolution). I am using this to synchronize my Markdown based notes files across machines. There are under a hundred files now so I haven't hit any issues in that department yet. However, there is a gradual memory leak which persists even when I kill and restart my process. I observed this only on a KVM based Linux guest - not sure if this is because of inotify based listening but I'm going to have to dig deeper to find out.

jamiesonbecker 11y ago
Nice clean code. A couple of places to start looking are probably in keeprunning(). Any file handle issues on repeated runs? The threads in lines 245-247 feel like they could deadlock or have a race, esp if changes happen nearly simultaneously on both ends, or if a change on one end is received and sync'ed to the other. I'm not sure how git handles multiple writes? Almost wonder if forking might cure the memory leak too or at least make it easier to see (see my other comment of some shell code).
- prashanthellina 11y ago
  The memory "leak" persists across restarts of my process. That is what beats me. If memory were leaking while the process was alive and was reclaimed by the OS upon killing the process, then I could debug the process itself. I read that running multiple git commands at the same time on the same repo should be fine (I can't remember where I read this). It is probably better to synchronize regardless!
jamiesonbecker 11y ago
Forking might be easier to debug.. but I think I might have missed some of the functionality in your script.. something like (totally untested): (while true do inotifywait \ --exclude '.*\.sw.' \ --recursive \ --event close_write \ --event delete \ --event moved_to \ --event moved_from \ dir1/ git push sleep 0.1 done ) 2>&1 >/var/log/push_log.log & (while true do cat << 'EOF' | ssh remotebox inotifywait \ --exclude '.*\.sw.' \ --recursive \ --event close_write \ --event delete \ --event moved_to \ --event moved_from \ dir1/ git push #?? sleep 0.1 EOF # (second EOF cannot be indented) done ) 2>&1 >/var/log/pull_log.log & echo "Kicked off pull and push processes." wait
- prashanthellina 11y ago
  That is Gold! Implementing the functionality in shell itself was what I considered first however I needed two things 1. Make it work on OSX 2. Use a lock file to prevent multiple instances from running when I put this in crontab I found watchman[0]. Got to see if I can use that in place of "inotifywait". [0]: https://facebook.github.io/watchman/docs/install.html https://facebook.github.io/watchman/docs/install.html
  - jamiesonbecker 11y ago
    Yeah that should work!

jamiesonbecker 11y ago

Nice clean code. A couple of places to start looking are probably in keeprunning(). Any file handle issues on repeated runs? The threads in lines 245-247 feel like they could deadlock or have a race, esp if changes happen nearly simultaneously on both ends, or if a change on one end is received and sync'ed to the other. I'm not sure how git handles multiple writes? Almost wonder if forking might cure the memory leak too or at least make it easier to see (see my other comment of some shell code).

prashanthellina 11y ago
The memory "leak" persists across restarts of my process. That is what beats me. If memory were leaking while the process was alive and was reclaimed by the OS upon killing the process, then I could debug the process itself. I read that running multiple git commands at the same time on the same repo should be fine (I can't remember where I read this). It is probably better to synchronize regardless!

jamiesonbecker 11y ago

Forking might be easier to debug.. but I think I might have missed some of the functionality in your script.. something like (totally untested): (while true do inotifywait \ --exclude '.*\.sw.' \ --recursive \ --event close_write \ --event delete \ --event moved_to \ --event moved_from \ dir1/ git push sleep 0.1 done ) 2>&1 >/var/log/push_log.log & (while true do cat << 'EOF' | ssh remotebox inotifywait \ --exclude '.*\.sw.' \ --recursive \ --event close_write \ --event delete \ --event moved_to \ --event moved_from \ dir1/ git push #?? sleep 0.1 EOF # (second EOF cannot be indented) done ) 2>&1 >/var/log/pull_log.log & echo "Kicked off pull and push processes." wait

prashanthellina 11y ago
That is Gold! Implementing the functionality in shell itself was what I considered first however I needed two things 1. Make it work on OSX 2. Use a lock file to prevent multiple instances from running when I put this in crontab I found watchman[0]. Got to see if I can use that in place of "inotifywait". [0]: https://facebook.github.io/watchman/docs/install.html https://facebook.github.io/watchman/docs/install.html
- jamiesonbecker 11y ago
  Yeah that should work!