8 ms·
DRAM has a design flaw from 1966. I bypassed it [video]
Related: Tailslayer: Library for reducing tail latency in RAM reads - https://news.ycombinator.com/item?id=47680023 https://news.ycombinator.com/item?id=47680023 - April 2026 (23 comments)
- dragonsenseiguy 7mo ago[flagged]
- mzajc 7mo agoPreviously: https://news.ycombinator.com/item?id=47680023 https://news.ycombinator.com/item?id=47680023
- dang 7mo agoThanks! We'll put that in the toptext too. (It didn't get much frontpage time, so we won't treat the current post as a dupe)
- kreelman 7mo agoThis is very much worth watching. It is a tour de force. Laurie does an amazing job of reimagining Google's strange job optimisation technique (for jobs running on hard disk storage) that uses 2 CPUs to do the same job. The technique simply takes the result of the machine that finishes it first, discarding the slower job's results... It seems expensive in resources, but it works and allows high priority tasks to run optimally. Laurie re-imagines this process but for RAM!! In doing this she needs to deal with Cores, RAM channels and other relatively undocumented CPU memory management features. She was even able to work out various undocumented CPU/RAM settings by using her tool to find where timing differences exposed various CPU settings. She's turned "Tailslayer" into a lib now, available on Github, https://github.com/LaurieWired/tailslayer https://github.com/LaurieWired/tailslayer You can see her having so much fun, doing cool victory dances as she works out ways of getting around each of the issues that she finds. The experimentation, explanation and graphing of results is fantastic. Amazing stuff. Perhaps someone will use this somewhere? As mentioned in the YT comments, the work done here is probably a Master's degrees worth of work, experimentation and documentation. Go Laurie!
- ufocia 7mo agoI like the video, but this is hardly groundbreaking. You send out two or more messengers hoping at least one of them will get there on time.
- npunt 7mo agoand dropbox was just rsync
- UltraSane 7mo agoThe clever part is figuring out what RAM is controlled by which controllers.
- saidnooneever 7mo agoeveryone says this but no one says why it was clever. i find her videos have cool results but i cant have patience for them usually because its recycled old stuff (can be cool but its not ground breaking). there is a ton of info you can pull from: smbios, acpi, msrs, cpuid etc. etc. about cpu/ram topology and connecticity, latencies etc etc. isnt the info on what controllers/ram relationships exists somewhere in there provided by firmware or platform? i can hardly imagine it is not just plainly in there with the plethtora info in there... theres srat/slit/hmat etc. in acpi, then theres MSRs with info (amd expose more than intel ofc, as always) and then there is registers on memory controller itself as well as socket to socket interconnects from upi links.. its just a lot of reading and finding bits here n there. LLms are actually really good at pulling all sorts of stuff from various 6-10k page documents if u are too lazy to dig yourself -_-
- UltraSane 7mo agoThe exact mapping between RAM addresses and memory controllers is intentionally abstracted by the memory subsystem with many abstraction layers between you and the physical RAM locations. Because documentation is sometimes incomplete or proprietary, security researchers often have to write software that probes memory and times the access speeds to reverse-engineer the exact interleaving functions of a specific CPU. in the video she says that ARM CPUs have the least data about this and she had to rely entirely on statistical methods.
- rationalist 7mo ago[flagged]
- foltik 7mo agoLove the format, and super cool to see a benchmark that so clearly shows DRAM refresh stalls, especially avoiding them via reverse engineering the channel layout! Ran it on my 9950X3D machine with dual-channel DDR5 and saw clear spikes from 70ns to 330ns every 15us or so. The hedging technique is a cool demo too, but I’m not sure it’s practical. At a high level it’s a bit contradictory; trying to reduce the tail latency of cold reads by doubling the cache footprint makes every other read even colder. I understand the premise is “data larger than cache” given the clflush, but even then you’re spending 2x the memory bandwidth and cache pressure to shave ~250ns off spikes that only happen once every 15us. There’s just not a realistic scenario where that helps. Especially HFT is significantly more complex than a huge lookup table in DRAM. In the time you spend doing a handful of 70ns DRAM reads, your competitor has done hundreds of reads from cache and a bunch of math. It’s just far better to work with what you can fit in cache. And to shrink what doesn’t as much as possible.
- formerly_proven 7mo agoOn most RAM tREF can be increased a lot from the default, at least if kept somewhat cool.
- Lramseyer 7mo agoAnother point about HFT - They're mostly using FPGAs (some use custom silicon) which means that they have much tighter control over how DRAM is accessed and how the memory controller is configured. They could implement this in hardware if they really need to, but it wouldn't be at the OS level.
- snizovtsev 7mo agoNot really. FPGAs run some simple proven strategies, but hardware guys can't keep up with how fast quants iterate on new strategies - so CPUs stay relevant.
- josephg 7mo agoIt could be massively improved with a special CPU instruction for racing dram reads. That might make it actually useful for real applications. As it is, the threading model she used here would make it incredibly difficult to use this in a real program.
- rcbdev 7mo agoAm I the only one who feels the comments here don't sound organic at all?
- Alifatisk 7mo agoI don’t see anything unusual
- isoprophlex 7mo agoYou're absolutely right
- tredre3 7mo agoNo I felt the same way, they're exactly like the usual LLM bot comment where a LLM recap ops and ends with an platitude or witty encouragement. But all the accounts are old/legit so I think that you and me have just become paranoid...
- wkjagt 7mo agoI have become oversensitive to this, and my brain is probably generating a lot of false positives. I don't think it's necessarily the case here, but I've wondered if people who use LLMs a lot take over some of its idiosyncrasies and in a way start sounding like one a bit. A strange side effect is that I've come to appreciate text with grammatical errors, videos where people don't enunciate well etc because it's a sign that it's human created content.
- kome 7mo ago[flagged]
- deleted 7mo ago[deleted]
- perching_aix 7mo agoWhen you use LLMs all day, their writing style rubs off on you. From wording to structure. It's like when you interact with any other piece of language oriented media.
- boznz 7mo agoShould say DRAM, SRAM does not have this.
- guenthert 7mo agoIndeed. And only for certain DRAM refresh strategies. I mean, it's at least conceivable that a memory management system responsible for the refresh notices that a given memory location is requested by the cache and then fills the cache during the refresh (which afaiu reads the memory) or -- simpler to implement perhaps -- delays the refresh by a μs allowing the cache-fill to race ahead. (seems that in the earlier submission, https://news.ycombinator.com/item?id=47680023 https://news.ycombinator.com/item?id=47680023, jeffbee hinted that IBM zEnterprise is doing something to that effect) Said that, I'm not convinced that this is a big issue in practice. If you really care about performance, you got to avoid cache misses.
- namibj 7mo agoNone of the DDR2 and onwards memories have anywhere near enough bandwidth to meet refresh frequency on each bit by you even just reading it in a loop. The refresh that we do is run in parallel on the memory arrays inside the RAM chips completely bypassing any of the related IO machinery.
- guenthert 7mo agoAnd those memory arrays cannot detect access from the bus? I'm not saying that it's easy or cheap or worthwhile (I'd rather guess it's not in most cases), but I don't see why it couldn't be done.
- dang 7mo agoOk I've consed a D onto the title above.
- rkagerer 7mo agoHalfway through this great video and I have two questions: 1) Can we take this library and turn it into a a generic driver or something that applies the technique to all software (kernel and userspace) running on the system? i.e. If I want to halve my effective memory in order to completely eliminate the tail latency problem, without having to rewrite legacy software to implement this invention. 2) What model miniature smoke machine is that? I instruct volunteer firefighters and occasionally do scale model demos to teach ventilation concepts. Some research years back led me to the "Tiny FX" fogger which works great, but it's expensive and this thing looks even more convenient.
- imp0cat 7mo agoBrio 33884. It has a tiny ultrasonic humidifier in there.
- hawk_ 7mo ago> halve my effective memory in order to completely eliminate the tail latency problem, Wouldn't you have a tail latency problem on the write side though if you just blindly apply it every where? As in unless all the replicas are done writing you can't proceed.
- lauriewired 7mo ago1. not that I can think of, due to the core split. It really has to be independent cores racing independent loads. anything clever you could do with kernel modules, page-table-land, or dynamically reacting via PMU counters would likely cost microseconds...far larger than the 10s-100s of nanoseconds you gain. what I wished I had during this project is a hypothetical hedged_load ISA instruction. Issue two requests to two memory controllers and drop the loser. That would let the strategy work on a single thread! Or, even better, integrating the behavior into the memory controller itself, which would be transparent to all software without recompilation. But, you’d have to convince Intel/AMD/someone else :) 2. It’s called a “smokeninja”. Fairly popular in product photography circles, it’s quite fun!
- rkagerer 7mo ago
- dinkumthinkum 7mo agoThis is an unreasonably good video. Hopefully, it inspires others to see we can still think hard and critically about technical things.
- deathanatos 7mo agoYeah, wow, the comments weren't kidding. This'll probably be the best video I watch all month, at least, if not more. I would have said what she was trying to do was "impossible" (had I not seen the title and figured … well … she posted the video) and right about when I was thinking that she got me with: > Hold on a second. That's a really bad excuse. And technology never got anywhere by saying I accept this and it is what it is.
- deleted 7mo ago[deleted]
- josalhor 7mo agoI haven't had time to see the whole thing yet, but I'm quite surprised this yielded good results. If this works I would have expected CPU implementations to do some optimization around this by default given the memory latency bottleneck of the last 1.5 decades. What am I missing here?
- formerly_proven 7mo agoTurning on mirroring does this for the low, low price of doubling your RAM cost.
- bronlund 7mo agoShe could probably have been stinking rich on this work alone, but instead she just put it up on Github. Kudos to Laurie.
- larodi 7mo agoShe probably is already stinking rich, or at least rich enough. Beyond certain point, though, research and knowledge seems more interesting than riches, and particularly if you feel yourself a researcher. Otherwise, perhaps, she be doing the same to business and be Ellona or something. Thank God she does not, but the contrary - is an inspiration to so many people - young and adult. Kudos!
- ahoka 7mo agoCompanies are standing in line to double their RAM usage right now, right.
- gkbrk 7mo agoDepends how much total RAM your application needs and how much money RAM access tail latency costs your business.
- bronlund 7mo agoFor an HFT firm, RAM cost is a non-issue. Even the tiniest improvement in latency can result in millions of dollars of extra profit. They can octuple their RAM usage and still make a killing. I bet Citadel already has reached out to Laurie :)
- ahoka 7mo ago[flagged]
- bronlund 7mo agoThis is not a problem which needed to be fixed, it's an improvement in efficiency - though a costly one. We are talking about a company which do make their own custom microchips so you could very well be right, but it may also be that they weren't even aware this was possible. Citadel executes trades in about 10 microseconds, so a 500 nanosecond reduced execution time is a 5% improvement. For a company which executes trades for hundreds of billions a day, this translates to real money. Your sarcasm indicates that you have no clue as to how important such an improvement can be for some actors. Some do though; the repo has almost 100 forks and 2K stars after just two days.
- deleted 7mo ago[deleted]
- yalogin 7mo agoThis is a cool idea, very well put through for everyone to understand such an esoteric concept. However I wonder if the core idea itself is useful or not in practice. With modern memory there are two main aspects it makes worse. First is cost, it needs to double the memory used for the same compute. With memory costs already soaring this is not good. Then the other main issue of throughout, haven’t put enough thought into that yet but feels like it requires more orchestration and increases costs there too.
- deleted 7mo ago[deleted]
- sbiru93 7mo agoDoesn't doing this halve the computing power? I don't know this world at all, is that acceptable?
- fc417fc802 7mo agoIt halves (or thirds or quarters or etc) available CPU cores, cache space, memory bandwidth, all the critical resources. So I expect that it's only applicable for small reads that you are reasonably certain won't be in cache and that it can only be used extremely sparingly, otherwise it will be nothing but a massive drain.
- dombiscoff 7mo ago[flagged]
- tromp 7mo agoA more accurate but less inspiring title would be: RAM Has a Design Tradeoff from 1966. I made another one on top. The first tradeoff, of 6x fewer transistors for some extra latency, is immensely beneficial. The second, of reducing some of that extra latency for extra copies of static data, is beneficial only to some extremely niche application. Still a very educational video about modern memory architecture. [EDIT: accidental extra copy of this comment deleted]
- deleted 7mo ago[deleted]
- villgax 7mo ago[flagged]
- kzrdude 7mo agoDon't know her twitter, but bsky is here: https://bsky.app/profile/lauriewired.bsky.social https://bsky.app/profile/lauriewired.bsky.social Not slop, seems mildly interesting
- rachr 7mo agoAre we looking at the same account? Unless it's some project like this, she doesn't even tweet every day. Just promotion of her own content.
- t1234s 7mo ago[flagged]
- jqbd 7mo agoBeing a woman in tech seems to have some benefits at least on YouTube
- actionfromafar 7mo agoSurely, but that's the baseline for most videos regardless of topic and presenter.
- freedomben 7mo agoLaurieWired is so incredibly smart, and so incredibly nerdy :-D Really enjoyed this video, and I'm pretty picky. I learned a lot, even though I already know (or thought I knew) quite a bit about this subject as it was a particular interest of mine in Comp Sci school. I highly recommend. Skip forward through chunks of the train part though where she is messing around. It does get more informative later though so don't skip all of the train part
- drooopy 7mo agoShe and Technology Connections are two of my favourite YouTube channels. Also I love her geocities website so much: https://lauriewired.com/ https://lauriewired.com/
- dwoldrich 7mo agoVoxel Space[1] could have used this, would that multicore had been prevalent at the time. I recall being fascinated that simply facing the camera north or south would knock off 2fps from an already slow frame rate. Many of our maps' routes would be laid out in a predominately east or west-facing track to max out our staying within cache lines as we marched our rays up the screen. So, we needed as much main memory bandwidth as we could get. I remember experimenting with cache line warming to try to keep the memory controllers saturated with work with measurable success. But it would have been difficult in Voxel Space to predict which lines to warm (and when), so nothing came of it. Tailslayer would have given us an edge by just splitting up the scene with multiprocessing and with a lot more RAM usage and without any other code. Alas, hardware like that was like 15 years in the future. Le sigh. [1] https://en.wikipedia.org/wiki/Voxel_Space https://en.wikipedia.org/wiki/Voxel_Space
- sodaplayer 7mo ago>Many of our maps' routes would be laid out in a predominately east or west-facing track That's fascinating to find out! I grew up a fan of Nova Logic, so I'll have to pay attention to this the next time I revisit their games. Was this done for Comanche or did you also do this for Delta Force?
- dwoldrich 7mo agoI was just the lowly build-master/installer/utility developer, but I got tapped for testing and debugging and performance because I was just a sponge for coding knowledge because I wanted to be a game developer so baaaad at the time. I didn't get to do any of the game coding, and my experiments were just fruits of conversations with benevolent sages. The reason facing east-west (or was it north-south, now I'm unsure) made such a difference in framerate was the color and height maps were ray marched in straight lines up from the bottom of the screen to the horizon. This meant you were zipping through the color map in straight lines, wrapping around to the other side if the ray went far enough. When those straight lines lined up with the color and height map (north-south), life was good (and when a ray marched up a sheer canyon wall, life was VERY good.) But, when those straight lines went perpendicular (east-west) to the color and height map, you were blowing through the L2 cache constantly and going to main memory very often. I imagine on modern hardware these cache misses wouldn't amount to much measurable time, but on a 386dx with 8megs of RAM, the impact was very clear. Novalogic was the only programming job I ever had where I got my own office with a door. ;) When I was with them, they had a policy of one game developer per game which I never saw again. Maximum cowboy coder energy, good times.
- hpcgroup 7mo ago[flagged]
- volume_tech 7mo ago[dead]
- jayd16 7mo agoI suppose hardware folks could cook up some kind of RAID 5 style striped RAM layout and utilize a hedging strategy. Write out parity, drop a late parity read instead of using it for error correction. You get better results than double the RAM needed with a striping strategy. I guess this would have been a nice way to reduce HDD latency as a new RAID mode... oh well.
- r0yadar 7mo agoClickbait title. Resource wasting solution. It's a nice PoC, but nothing more...
- atlasagentsuite 7mo ago[dead]
- ece 7mo agoWould be cool to see the performance difference for llama2.c or see it work for gddr on gpus too with nanogpt, though I guess the latter might or might not be possible because of architecture differences.
- Wowfunhappy 7mo agoThis is so cool! One thing that occurred to me while watching the video: would it potentially make more sense to just do this in the kernel? That way, you don't have to fight virtual addressing, and I imagine (?) you could even know for sure which channel you're on instead of guessing.
- kaliszad 7mo agoIn effect if the operating system knew about the DRAM layout, it could for instance double critical data structures and race the processing. Maybe this would be helpful in the networking areas. On the other hand this can maybe get fixed in hardware by just copying the page that's being refreshed to the side somewhere, eliminating the whole waiting problem. Last but not least, AFAIK writes to a row already recharge the capacitors so there shouldn't be a need to refresh it. What am I missing?