Also, while you're copying you can inverse grep filter to avoid duplicates. And since you're copying, you could maybe ensure sorting, but I don't think doing t

Also, while you're copying you can inverse grep filter to avoid duplicates. And since you're copying, you could maybe ensure sorting, but I don't think doing that with "bash" (plus off the shelf utils) makes a lot of sense. That's what DJBs CDB (cdbget, cdbmake, etc) is for: https://cr.yp.to/cdb.html https://cr.yp.to/cdb.html