3 ms·

>When you're using the USB interface, there is some chip in between the actual SSD and your computer. USB doesn't directly handle reading and writing to SATA or

>When you're using the USB interface, there is some chip in between the actual SSD and your computer. USB doesn't directly handle reading and writing to SATA or NVMe drives, so software on your computer encodes it to go out USB, and then some chip takes that USB signaling and converts it to read/write to the SSD, and vice-versa. If the controller on the USB interface is poor quality, it may reduce the performance of the overall set up. I've had external regular SATA/USB enclosures have varying performance despite being the same underlying drives inside. Yes, I understand this. What I am thinking of is the overhead of reading and constructing each block from flash. Is this really limited by the USB controller capability(I guess theoretically yes since its in the chain?) or is it just limited by the Flash controller and typical USB controllers. I guess we can test a theoretical bottleneck by running an IOPS test and comparing it to official Benchmark numbers of that Optaine model. That way we can eliminate the USB controller if the numbers match up. This is the structure from what I understand. PC Host Controller<--->USB Controller(in the enclosure)<--->Flash Controller(On the SSD)<--->Flash chip(On the SSD). Is this wrong? So here with Optane we have a massively high throughput of small files but a lower throughput of large files. Is this scenario something that a USB controller would noticeably slow down? I have never tested this before. A regular SSD has lower throughput of small files in exchange for high throughput of large files(slower throughput but higher bulk transfer) which is noticeable if the USB controller was not up to the task (See most USB Drives which have this + flash controller on the same board) Basically the whole reason this idea came up in my head is that I notice that despite using high bandwidth drives to install clean Windows and Linux copies, I notice certain bottlenecks in the install process that I suspect are a result of many small files being copied/unpacked(either copying package files in Linux or unpacking the WIM in Windows). So that led to me wanting to benchmark a High IOPS drive and comparing it to low IOPS to see once and for all if the install process is choking on the drive vs something else (ie. the CPU capability for decompressing). Its just a weirdo side project that I wanted to try. Eventually I even wanted to somehow try a ramdisk type scenario although I don't think that sort of hardware exists. This is also motivated by the fact that I don't have a way to intercept the file reading process on the drive as they are being read to see if the drive really is the source of any bottlenecks or if its the installer code.

the8472 6y ago
> A regular SSD has lower throughput of small files in exchange for high throughput of large files(slower throughput but higher bulk transfer) which is noticeable if the USB controller was not up to the task (See most USB Drives which have this + flash controller on the same board) To a large extent that is a software problem. The installers/package managers are neither pipelining nor parallelizing their operations properly. If you're dealing with many small files then on the reader side you want to issue a window of readaheads for a certain size across multiple files rather than doing one file at a time and on the writer side you need to batch your fsyncs. > So that led to me wanting to benchmark a High IOPS drive and comparing it to low IOPS to see once and for all if the install process is choking on the drive vs something else (ie. the CPU capability for decompressing). Its just a weirdo side project that I wanted to try. There are PCIe slot to m.2 adapter cards, those are more efficient and cheaper than external adapters.

the8472 6y ago

> A regular SSD has lower throughput of small files in exchange for high throughput of large files(slower throughput but higher bulk transfer) which is noticeable if the USB controller was not up to the task (See most USB Drives which have this + flash controller on the same board) To a large extent that is a software problem. The installers/package managers are neither pipelining nor parallelizing their operations properly. If you're dealing with many small files then on the reader side you want to issue a window of readaheads for a certain size across multiple files rather than doing one file at a time and on the writer side you need to batch your fsyncs. > So that led to me wanting to benchmark a High IOPS drive and comparing it to low IOPS to see once and for all if the install process is choking on the drive vs something else (ie. the CPU capability for decompressing). Its just a weirdo side project that I wanted to try. There are PCIe slot to m.2 adapter cards, those are more efficient and cheaper than external adapters.