3 ms·
Show HN: DeepSearch – a high-performance SMB directory scanner in Rust
- dohuyhoangvn93 10mo agoHi HN, I'm the author of DeepSearch. I built this while dealing with large SMB shares over WiFi, where traditional directory scanning tools were either too slow or caused unnecessary network load. The main idea is a layered scanning strategy: - Minimize SMB round-trips - Use Rayon for controlled parallelism - Avoid spawning excessive IO-bound tasks - Keep memory usage predictable This is not meant to be a fancy UI tool, but a fast and deterministic building block that can be embedded into other systems. The project is open-source (Apache 2.0). I'm especially interested in feedback on: - The concurrency model - SMB traversal strategy - Any edge cases you've seen in similar systems Thanks for taking a look.
- holg 10mo agoHi thx for sharing, in general about concurrency Rayon excels on CPU-heavy tasks, but this is more IO for traversal avoid any extra round-trips, simple the metadata already given edge-cases would be infinite loops on symlinks, the Windows 255 char limits and on WIFI packet loss due to too many reqs
- dohuyhoangvn93 10mo agoThank you for your feedback. In the initial CLI test, I tried using Tokio and found that performance didn't improve and the earliest result return speed was quite poor. I skipped the root cause analysis and kept the original rayon design. I will try redoing the tests and taking more detailed measurements for evaluation. Thank you again.