13 ms·
I had my formative years in programming when memory usage was something you still worried about as a programmer. And then memory expanded so much that all kinds
by travisgriggs 7mo ago
I had my formative years in programming when memory usage was something you still worried about as a programmer. And then memory expanded so much that all kinds of “optimal” patterns for programming just become nearly irrelevant. Will we start to actually consider this in software solutions again as a result?
- jacquesm 7mo ago> And then memory expanded so much that all kinds of “optimal” patterns for programming just become nearly irrelevant. I don't think that ever happened. Using relatively sparse amount of memory turns into better cache management which in turn usually improves performance drastically. And in embedded stuff being good with memory management can make the difference between 'works' and 'fail'.
- zeta0134 7mo agoThe need to use optimal patterns didn't go away, but the techniques certainly did. Just as a quick example, it's usually a bad idea now to use lookup tables to accelerate small math workloads. The lookup table creates memory pressure on the cache, which ends up degrading performance on modern systems. Back in the 1980s, lookup tables were by far the dominant technique because math was *slow.*
- jacquesm 7mo agoThe way to approach this is to benchmark and then pick the best solution.
- zozbot234 7mo ago> Back in the 1980s, lookup tables were by far the dominant technique because math was slow. This actually generalizes in a rather clean way: compared to the 1980s, you now want to cheaply compress data in memory and use succinct representations as much as practicable, since the extra compute involved in translating a more succinct representation into real data is practically free compared to even one extra cacheline fetch from RAM (which is now hundreds of cycles latency, and in parallel code often has surprisingly low throughput).
- QuadmasterXLII 7mo agoIt’s a mad word where ultimate performance in one problem can require compressing data in ram and in another storing it uncompressed on disc.
- bonesss 7mo agoThe same atmosphere that makes bread hard makes crackers soft.
- yread 7mo agoWhen was the last time you used mergesort because you had to?
- jacquesm 7mo agoCoincidentially, last night, and I'm not pulling your leg! But to be fair that's the first time in much more than a decade. I don't normally work with such huge files and this was one very rare exception. I also nearly crashed my machine by triggering the OOM killer after naively typing 'vi file' without first checking how large it had become. I'm working on a project that I probably should run on a more serious machine but I don't feel like moving my whole work environment from the laptop that I normally use.
- _fizz_buzz_ 7mo agoIt obviously never became completely irrelevant. But I think programmers spend a lot less time thinking about memory than they used to. People used to do a lot of gymnastics and crazy optimizations to fit stuff into memory. I do quite a bit of embedded programming and most of the time it seems easier for me to simply upgrade the MCU and spend 10cents more (or whatever) than to make any crazy optimimzations. But of course there are still cases where it makes sense.
- II2II 7mo agoWhile thinking less about memory optimizations is possible since we have more memory, it was enabled by the languages and libraries we use. Fourty years ago, you were probably implementing your own data structures. Sure, there were plenty of languages that offered them back then (LISP was based on linked lists, and that language is from the 1960's). Chances are you weren't using such languages unless you were using big computers or writing software that didn't handle much data. These days, pretty much any language will provide at least some data structures and their related algorithms. Even systems programming languages like C++ and Rust. Of course, there are an absurd number of libraries if you need anything more specialized.
- rTX5CMRXIfFG 7mo agoI never really bought in to the anti-Leetcode crowd’s sentiment that it’s irrelevant. It has always mattered as a competitive edge, against other job candidates if you’re an employee or the competition of you’re a company. It only looked irrelevant because opportunities were everywhere during ZIRP, but good times never last.
- raw_anon_1111 7mo agoMost developers work at banks, insurance companies and other “enterprise” jobs. Even most developers at BigTech and who are working “at scale” are building on top of scalable infrastructure and aren’t worrying about reversing a btree on a whiteboard.
- deleted 7mo ago[deleted]
- AdamN 7mo agoAgree that the whiteboard thing is often not applicable but it's so nice when a developer has efficient code if only because it indicates that they know what's going on and also that there are fewer bugs and other bottlenecks in the system.
- raw_anon_1111 7mo agoThose bugs don’t come from using the wrong algorithm, they come from not understanding the business case of what you’re writing. Most performance issues in the real world for most cases don’t have anything to do with the code. It’s networking, databases, etc. Your login isn’t slow because the developer couldn’t do leetcode
- tracker1 7mo agoNo, it's because 50k reads of settings are happening with a SQL Table in memory that's queried via SQL statement instead of a key/value hashtable. (real world experience, I think it was close to 28k reads, but the point stands)
- fulafel 7mo agoYou're right in terms of fitting your program to memory, so that it can run in the first place. But in performance work, the relative speed of RAM relative to computation has dropped such that it's a common wisdom to treat today's cache as RAM of old (and today's RAM as disk of old, etc). In software performance work it's been all about hitting the cache for a long time. LLMs aren't too amenable to caching though.
- seanmcdirmid 7mo agoLLMs need memory bandwidth to stream lots of data through quickly, not so much caching. Well, this is basically the same way that a GPU uses memory.
- zozbot234 7mo agoOTOH, LLM inference tends to have very predictable memory access patterns. So well-placed prefetch instructions that can execute predictable memory fetches in parallel with expensive compute might help CPU performance quite a bit. I assume that this is done already as part of optimized numerical primitives such as GEMM, since that's where most of the gain would be.
- makapuf 7mo agoAFAIK, you can't explicitly allocate cache like you allocate RAM however. A bit like if you could only work on files and ram was used for cache. Maybe I am mistaken ? (Edit: typo)
- NooneAtAll3 7mo agomost likely in a couple years this bubble will pop, just like 8 years and 16 years ago it's just a cartel cycle of gaining profits while soon eliminating all investments into competitors when flood of cheap ram "suddenly" appears
- thfuran 7mo agoThis is coming from an insane demand spike, not some nefarious plot by the RAM manufacturers.
- cyanydeez 7mo agoYes, it's a nefarious plot of AI producers to attempt a monopoly with a product that no one seems capable of demonstrating has the exponential value they're betting on.
- adornKey 7mo agoOnce everybody has a decent amount of VRAM they can just run local AIs and the need to mess with Ad-laden search results will fizzle. So of course they are desperate to grab a new monopoly. People haven't realised yet, that local AIs are fast and produce good results - on pretty average hardware. If they don't manage to grab a new monopoly Google will be history. But it doesn't really need a nefarious plot for the price spikes. There is a serious lack of VRAM deployed out there. Filling that gap will take quite some time. Add to that the nefarious plot and the situation will most likely get even worse....
- mrob 7mo agoLLM inference is mostly read only, so high-bandwidth flash looks like it could provide huge cost savings over VRAM. It's not yet in commercial products but there are working prototypes already. Previous HN discussion: https://news.ycombinator.com/item?id=46700384 https://news.ycombinator.com/item?id=46700384
- 7mo ago
- ReedorReed 7mo agoI just heard in a podcast, they talked about how powerful our devices are today but do not feel faster than they did 15 years ago and that it's because of what you write here.
- AdamN 7mo agoA lot of that is on the OS vendors (and security requirements drive some inefficiencies that didn't used to be needed either).
- enaaem 7mo agoI have a 2020 Intel Mac (quad core, 16gb RAM) and it feels as slow as the Packard Bell from 2000 when I was a kid. The launchpad takes 1-2 seconds to show a bunch of icons. Absolutely insane!
- lmcd 7mo agoI've recently started a side project for the N64, and this is very relatable! Working within such tight constraints is most of the fun.
- cyberrock 7mo agoIt's not like most developers are wasting memory for fun by using Electron etc. It's just the simplest way to deploy applications that require frequent multiplatform changes. Until you get Apple to approve native app changes faster and Linux users to agree on framework, app distribution, etc., it's the most optimal way to ship a product and not just a program.
- close04 7mo ago> for fun Not for fun but for convenience (laziness occasionally?). Someone needed to "pay" for the app being available on all platforms. Either the programmer by coding and optimizing multiple times, or the user by using a bloated unoptimized piece of software. The choice was made to have the user pay. It's been so long I doubt recent generations of coders could even do it differently.
- hulitu 7mo ago> applications that require frequent multiplatform changes Maybe a bit of engineering and planning could help here. Shipping always half finished products is it usually not a recipe to success.
- yxhuvud 7mo agoWe would have, if the expensive memory was a long term trend. It is not - eventually the supply will expand to match demand. There is no fundamental lack of raw materials underlying the issues, it is just a demand shock.
- junon 7mo agoAlso, it's not like we have regressed in the process itself either, which was historically the limiting factor. As you said this is purely an economics thing resulting from a greedy shift in business focus by e.g. Micron.
- jooz 7mo agoWhen I train some leetcode problems, I remember the best solution was the one that optimised cpu (time) instead of memory. Meaning adding data index in memory instead of iterating on the main data structure. I thought, ok, thats fine, it's normal, you can (could) always buy more RAM, but you can't buy more time. But well, I think there is no right answer and there always be a trade off case by case depending on the context.
- dahcryn 7mo agoI've actively started to use outlook and teams through chrome to free up some of my ram, easily saves 3-4gb. It's gotten ridiculous how much ram basic tools are using, leaving nothing for doing actually real work
- christophilus 7mo agoPeople get on me all the time about not installing programs on my computer. I run everything in the browser, if I can. Partly so I can kill it properly without it misbehaving, and partly because I don't trust their software at all. Zoom, Slack, Gmail, etc-- if I can run it in the browser, then that's the only way I'll run it.
- zuhsetaqi 7mo agoSame for me on mobile. I don’t install the Amazon app I just use the browser where I can limit tracking and only log in when actually buying something.
- emeril 7mo agoive found the web versions use a similar amount of memory and have fewer features my issue is that my company won't issue laptops with more than 16 gbs of ram guess i'm not virtualizing anything...
- dgxyz 7mo agoEvery app ships with its own isolated web browser now. That idea needs to die. Back to native apps without bloated toolkits!
- zarzavat 7mo agoRAM didn't get more expensive to produce. It just got more desirable. The prices will come down again when supply responds. It may take some time, but it will happen eventually.
- zozbot234 7mo agoRAM actually got more expensive to produce in the medium term because production is bottlenecked. It takes years to expand production.
- StopDisinfo910 7mo agoRAM production is highly inelastic and controlled by an oligopoly. They have little desire to increase production considering the lead time and the risk that the AI demand might be transient. They actively prefer keeping confortable margins than competing between each other. They have already been condemned for active collusion in the past. New actors from China could shake things up a bit but the geopolitical situation makes that complicated. The market can stay broken for a long time.
- zozbot234 7mo agoThey are increasing production as fast as they can (which is not fast at all, it's more like slowly steering a huge ship towards the correct direction) because current prices are too high even when accounting for the historical oligopoly dynamics. They can easily increase their collective profits by making more.
- StopDisinfo910 7mo agoAs far as I know, they are merely shifting capacities from the customer market towards the data center market with minimal retooling. I am unaware of any of the three actively investing in new capacity. Some modest increase are planned but nowhere near what you would expect given current demand.
- toast0 7mo agoRAM manufacturers don't increase production as fast as possible, because they've been through enough boom and bust. Rapid increase in capacity leads to oversupply which leads to negative margins. They've been there before, and they don't want to go there again. RAM manufacturers do routinely setup new fabs and decommision old fabs. Maybe they're trying to hurry up new fab construction in times like these, and they would likely defer shutting down old fabs or restart them where possible. But they're less likely to build new fabs that weren't already part of their long term plans.
- throw0101a 7mo ago> I had my formative years in programming when memory usage was something you still worried about as a programmer. As 'just' a user in the 1990s and MS-DOS, fiddling with QEMM was a bit of a craft to get what you wanted to run in the memory you had. * https://en.wikipedia.org/wiki/QEMM https://en.wikipedia.org/wiki/QEMM (Also, DESQview was awesome.)
- mushufasa 7mo agoI doubt it. I predict in a few years, maybe sooner, one/some of the AI companies buying up the supply will either have achieved their goal or collapsed, and then the market will be flooded with a glut of memory driving prices low again. Or, conversely, the demand stays high for a sustained period of time and the suppliers just increase supply. There's no hard bill of materials/technical reasons for the memory prices to be this high, unlike 20+ years ago.
- Lalabadie 7mo agoAnd in the meantime, major buyers (government, big orgs) adjust by extending the planned lifespan of their computers, and upping the IT wage budget a bit to support that. That adjustment probably won't go away after supply returns.
- BunsanSpace 7mo agoThat's honestly a good thing. Computers aren't really getting faster for end users doing mundane tasks the past couple of years. Will help reduce E-Waste, and to the end user there won't be a different. A machine from 5 years ago feels just as fast as a brand new machine.
- KellyCriterion 7mo agoIm always shocked how much good IT equipment is shoved into the trashbin: At a lot of companies I could make a great deal - either for using it on my own or selling it on Ebay later on. Big Corporations offen trash IT equipment thats only 3 - 4 years old. And there is no recycling etc. Very sad.
- toast0 7mo agoBig corporations tend to send old hardware through the surplus marketplace. There's lots of 3-4 year old corporate computers for sale. Often, the company leases the computers and then the lessor will sell them when they're returned.
- eulers_secret 7mo agoDepends on the machine you’re targeting. I do embedded Linux and ram usage is a major concern, same for other embedded applications. I’m partying like it’s the 90s, on a 32-bit processor and a couple hundred MB of ram.
- nostrademons 7mo agoAndroid's investing significantly in reducing the memory usage of the next release simply because the BOM cost of RAM for their low-end partners is becoming prohibitive.
- halJordan 7mo agoBut if that new or different because of this event? No it's not, Android has had several initiatives to enable low end devices, from optimizing full fatter Android, to inventing new versions of Android.
- nostrademons 7mo agoIn this case it is explicitly because of the RAMpocalypse. The initiatives have existed forever but they've gotten a lot more funding and a lot more exec attention because of the situation in the hardware market.
- toast0 7mo agoAndroid has been talking about these kinds of things for a long time. But if they're actually meaningfully making progress on them, it's most likely because of real pressure. (He types on his phone with 6GB of ram)
- deleted 7mo ago[deleted]