jared_hulbert

5 ms·

1.
by jared_hulbert 1y ago
Just ran a version with 6 prefetching threads. I get 5.81GB/s. Same as the io_uring with 2 drives, but still a lot slower than the in memory solution.
2.
by jared_hulbert 1y ago
Someone else suggested this, results are even worse by 2.5s.
3.
by jared_hulbert 1y ago
When I run the 50GB in-mem setup I still have 40GB+ of free memory, I drop the page cache before I run "sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'" there wouldn't really be anything to evi
4.
by jared_hulbert 1y ago
The io_uring solution avoids this whole effort of mapping. It doesn't have to map the already-in-RAM pages at all. It reuses a small set of buffers. So there is a lot of random cache-miss prone work that mmap() has to do that the io_
5.
by jared_hulbert 1y ago
I get that. But I do actually show a scenario where accessing data from memory using a very standard mechanism IS slower than a newer but equally standard way of accessing data from an NVMe drive. "Accessing memory is slower in some c
6.
by jared_hulbert 1y ago
Adding MAP_HUGE_1GB and not MAP_HUGETLB does compile and run for me. Not convinced that its' actually doing anything. Performance is the same.
7.
by jared_hulbert 1y ago
Not by a ton but if you add up the DDR5 channel bandwidth and the PCIe lanes most systems the PCIe bandwidth is higher. Yes. HBM and L3 cache will be higher than the PCIe.
8.
by jared_hulbert 1y ago
Hmm. tmpfs was slower. hugetlbfs wasn't working for me.
9.
by jared_hulbert 1y ago
spdk and I go way back. I'm confident it'd be about the same, possibly ~200-300MB/s more, I was pretty close to the rated throughput of the drives. Io_uring has really closed the gap that used to exist between the in kernel
10.
by jared_hulbert 1y ago
YES! gcc and clang don't like to optimize this. But they do if you hardcode the size_bytes to an aligned value. It kind of makes sense, what if a user passes size_bytes as 3? With enough effort the compilers could handle this, but i
11.
by jared_hulbert 1y ago
I worked on SSDs for years. Too many people are suffering from insufficiently solid values of "disk" IMHO.
12.
by jared_hulbert 1y ago
int fd = open(filename, O_RDONLY); void* buffer = mmap(NULL, size_bytes, PROT_READ, (MAP_HUGETLB | MAP_HUGE_1GB), fd, 0); This doesn't work with a file on my ext4 volume. What am I missing?
13.
by jared_hulbert 1y ago
The original blog post title is intentionally clickbaity. You know, to bait people into clicking. Also I do want to challenge people to really think here. Seeing if the cached file data can be accessed quickly is the point of the experime
14.
by jared_hulbert 1y ago
https://www.intel.com/content/www/us/en/io/data-direct-i-o-t... AMD has something similar. The PCIe bus and memory bus both originate from the processor or IO die of the "CPU" when you use
15.
by jared_hulbert 1y ago
I tried with the log scale before. They failed to express the exponential hockey stick growth unless you really spend the time with the charts and know what log scale is. I'll work on incorporating log scale due to popular demand. Th
16.
by jared_hulbert 1y ago
Would this actually create huge page page cache entries?
17.
by jared_hulbert 1y ago
Lol. Thanks.
18.
by jared_hulbert 1y ago
Oh man... I'd have look into that. Off the top of my head I don't know how you'd make that happen. Way back when I'd have said no. Now with all the folio updates to the Linux kernel memory handling I'm not sure.
19.
by jared_hulbert 1y ago
Cool. Original author here. AMA.
20.
by jared_hulbert 1y ago
In this context CXL is kinda storage and kinda higher latency RAM. It's latency is worse than standard DDR5 DIMMs but the bandwidth is on the same trajectory as storage/networking. I'm inclined to think of it like storage in
21.
by jared_hulbert 1y ago
We are seeing more SRAM in the form of CPU caches, this is one of the things that is actually scaling still. Are you asking why not use SRAM in something like a DIMM? You could do this. Here's why I wouldn't advocate for this. A
22.
by jared_hulbert 1y ago
Original author AMA
23.
by jared_hulbert 1y ago
https://www.bitflux.ai/blog/memory-is-slow-part2/
24.
by jared_hulbert 1y ago
I found https://aokumo.ai/ Who else?
25.
Show HN: Agents for DevOps? Demo(bitflux.ai)
2 points by jared_hulbert 1y ago|1 comments
26.
by jared_hulbert 3y ago
memverge.com does some cool work around making that happen.
27.
by jared_hulbert 3y ago
Silly hack of a product. x2 PCIe for Optane, 2x for NAND. Meant compromised bandwidth for all! The fact that it actually had value in some cases highlights how cool it could have been if it was a proper hybrid design.