4 ms·

_Why_ is it faster? (particularly on Linux, I see you're using syscalls on mac that mac's cp doesn't, since apple doesn't really maintain their unix userspace)

• Svetlitski 5y ago
  The primary reason is that it uses all of your computer's available cores to walk through directories and copy files, effectively speeding things up by a factor directly proportional to how many cores you have. On macOS, this is augmented by regular files being copied using the OS's fclonefileat and fcopyfile mechanisms under the hood, which allows even very large files to be copied near-instantly (when compared to copying them block-by-block as classic cp does).
  • CalChris 5y ago
    Is it available on homebrew?
    • Svetlitski 5y ago
      No, it is not. I would've submitted a Homebrew formula for it, but that would go directly against Homebrew's guidelines, which frown upon authors submitting formulas for their own tools [1]. [1] https://docs.brew.sh/Acceptable-Formulae#niche-or-self-submitted-stuff https://docs.brew.sh/Acceptable-Formulae#niche-or-self-submi...
  • fulafel 5y ago
    This works out assuming that file copying is CPU bound. Traditionally this hasn't been the case but maybe things have changed with widespread fast SSDs and usage of disk crypto. Traditionally the reason to use threads in small-requests i/o is to get multiple outstanding requests in flight in the hardware io stack which some hardware can use to get you more IOPS throughput. (They might end up hitting different disks in RAID, or taking advantage SSD internal parallelism, or whatever tricks spinning disks use to get more throughput with TCQ) (Re your MacOS remark, Linux has support for copy-on-write file copies as well, see reflink and FICLONE / copy_file_range)
    • yokaze 5y ago
      > Traditionally the reason to use threads in small-requests i/o is to get multiple outstanding requests in flight in the hardware io stack which some hardware can use to get you more IOPS throughput I would assume that is the case here too. The threads are likely hiding the latency of the IOPS. I can't see any CPU load statistics, so it is hard to say, if it is indeed CPU bound or not. A quick run with time shows user+sys being only slightly higher than real, so with 6 threads for my 6 cpus, it uses a single cpu. (and practically no time in user-space)
      • fulafel 5y ago
        Ok, so the core count is not the right level of parallelism (except by accident).
  • mnw21cam 5y ago
    Does this software gain any speed by sacrificing any safety? As in, does it switch off fsync in any way? Are the files guaranteed durable by the time the software terminates?
  • jjav 5y ago
    > The primary reason is that it uses all of your computer's available cores to walk through directories and copy files, effectively speeding things up by a factor directly proportional to how many cores you have. CPU cores can't read/write files, those are IO operations by the storage device. The CPU could have 64 cores but on a system with a single hard disk that can do just one concurrent operation.
• Siira 5y ago
  Does GNU cp (installed from brew) use those syscalls?