9 ms·

Everything old is new again: memory optimization

231 points by ibobev 7mo ago

est 7mo ago
I think py version can be shortened as: from collections import Counter stats = Counter(x.strip() for l in open(sys.argv[1]) for x in l)
- voidUpdate 7mo ago
  Would that decrease memory usage though?
  - est 7mo ago
    see https://news.ycombinator.com/item?id=47543756 https://news.ycombinator.com/item?id=47543756
griffindor 7mo ago
Nice! > Peak memory consumption is 1.3 MB. At this point you might want to stop reading and make a guess on how much memory a native code version of the same functionality would use. I wish I knew the input size when attempting to estimate, but I suppose part of the challenge is also estimating the runtime's startup memory usage too. > Compute the result into a hash table whose keys are string views, not strings If the file is mmap'd, and the string view points into that, presumably decent performance depends on the page cache having those strings in RAM. Is that included in the memory usage figures? Nonetheless, it's a nice optimization that the kernel chooses which hash table keys to keep hot. The other perspective on this is that we sought out languages like Python/Ruby because the development cost was high, relative to the hardware. Hardware is now more expensive, but development costs are cheaper too. The take away: expect more push towards efficiency!
- pjc50 7mo ago
  >> Peak memory consumption is 1.3 MB. At this point you might want to stop reading and make a guess on how much memory a native code version of the same functionality would use. At this point I'd make two observations: - how big is the text file? I bet it's a megabyte, isn't it? Because the "naive" way to do it is to read the whole thing into memory. - all these numbers are way too small to make meaningful distinctions. Come back when you have a gigabyte. It gets more interesting when the file doesn't fit into RAM at all. The state of the art here is : https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times-by-70/ https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times... , wherein our hero finds the terrible combination of putting the whole file in a single string and then running strlen() on it for every character.
  - deleted 7mo ago
    [deleted]
  - dgb23 7mo ago
    > all these numbers are way too small to make meaningful distinctions. Come back when you have a gigabyte. I have to disagree. Bad performance is often a result of a death of a thousands cuts. This function might be one among countless similarly inefficient library calls, programs and so on.
    - rcxdude 7mo ago
      If you're not putting a representative amount of data through the test, you have no idea if the resource usage you're seeing scales with the amount of data or is just a fixed overhead if the runtime.
  - kloop 7mo ago
    > how big is the text file? I bet it's a megabyte, isn't it? The edit in the article says ~1.5kb
    - pjc50 7mo ago
      Single page on many systems, which makes using mmap() for it even funnier.
- veunes 7mo ago
  I suspect it'll be selective
- zozbot234 7mo ago
  > If the file is mmap'd, and the string view points into that, presumably decent performance depends on the page cache having those strings in RAM. Not so much, because you only need some fraction of that memory when the program is actually running; the OS is free to evict it as soon as it needs the RAM for something else. Non-file-backed memory can only be evicted by swapping it out and that's way more expensive,
  - astrange 7mo ago
    All major OSes (well Windows and macOS) do in-memory compression before swap, which is cheaper than evicting a file-backed page. But still slow, so you don't want to rely on it.
biorach 7mo ago
"copyright infringement factories"
- maipen 7mo ago
  Tells you right away where this is coming from.
  - muskstinks 7mo ago
    The critisism is valid. The problem is how you value this critism. I agree they are stealing it but I also see the benefit of it for society and for myself. Suckerberg downloaded terabytes of books for training, while people around me got sued to hell 20 years ago for downloading one mp3 file.
    - anthk 7mo ago
      Anna's Archive. Aaron Swartz. FB and so are CIA fronts and they can do anything they please. Until they hit against Disney and lobbying giants and if a CIA idiot tries to sue/bribe/blackmail them they can order Hollywood to rot their images into pieces with all the wars they promoted in Middle East and Latin America just to fill the wallets of CEO's. That among some social critique movie on FB about getting illegal user data all over the world to deny insurances and whatnot. And OFC with a clear mention of the Epstein case with related people, just in case the Americans forgot about it. Then the US industry and military complex would collapse in months with brainwashed kids running away from the army. Not to mention to the Call of Duty franchise and the like. It would be the end of Boeing and several more, of course. To hell to profit driven wars for nothing. Ah, yes, AIPAC lobbies and the like. Good luck taming right wing wackos hating the MAGA cult more than the 'woke' people themselves. These will be the first ones against you after sinking the US image for decades, even more than the illegal Iraq war with no WMD's and the Bush/Cheney mafia. The outcome of this? proper and serious engineering a la Airbus. Instant profit-driven MBA and war sickos being kicked out from the spot. OFC the AI snakeoil sellers except for the classical AI/NN against concrete cases (image detection and the like), these will survive fine, even better because these kind of jobs are highly specific and they are not statistical text parrots. They can provide granted results unlike LLM's prone to degrade because the human based content feeding needs to be continuous, while for tumour detection a big enough sample can cover a 99% of the cases. R&D on electric vehicles/energy and nuclear power like nowhere else. And, for sure, the EV equivalent of a Ford T for Americans. A cheap and reliable one, good enough for the common Joe/Mary without being a luxury item. A new Golden Age would rise, for sure. But the oil mafia will try to fight them like crazy.
tzot 7mo ago
Well, we can use memoryview for the dict generation avoiding creation of string objects until the time for the output: import re, operator def count_words(filename): with open(filename, 'rb') as fp: data= memoryview(fp.read()) word_counts= {} for match in re.finditer(br'\S+', data): word= data[match.start(): match.end()] try: word_counts[word]+= 1 except KeyError: word_counts[word]= 1 word_counts= sorted(word_counts.items(), key=operator.itemgetter(1), reverse=True) for word, count in word_counts: print(word.tobytes().decode(), count) We could also use `mmap.mmap`.
- akx 7mo ago
  This doesn't do the same thing though, since it's not Unicode aware. >>> 'x\u2009 a'.split() ['x', 'a'] # incorrect; in bytes mode, `\S` doesn't know about unicode whitespace >>> list(re.finditer(br'\S+', 'x\u2009 a'.encode())) [<re.Match object; span=(0, 4), match=b'x\xe2\x80\x89'>, <re.Match object; span=(7, 8), match=b'a'>] # correct, in unicode mode >>> list(re.finditer(r'\S+', 'x\u2009 a')) [<re.Match object; span=(0, 1), match='x'>, <re.Match object; span=(5, 6), match='a'>]
  - contravariant 7mo ago
    There's bound to be a way to turn a stream of bytes into a stream of unicode code points (at least I think that's what python is doing for strings). Though I'm explicitly not volunteering to write the code for it.
    - est 7mo ago
      import mmap, codecs from collections import Counter def word_count(filepath): freq = Counter() decode = codecs.getincrementaldecoder('utf-8')().decode with open(filepath, 'rb') as f, mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) as mm: for chunk in iter(lambda: mm.read(65536), b''): freq.update(decode(chunk).split()) freq.update(decode(b'', final=True).split()) return freq
fix4fun 7mo ago
Digression: Nowadays when RAM is expensive good old zram is gaining popularity ;) Try to check on trends.google.com . Since 2025-09 search for it doubled ;)
canpan 7mo ago
String views were a solid addition to C++. Still underutilized. It does not matter which language you are using when you make thousands of tiny memory allocations during parsing. https://en.cppreference.com/w/cpp/string/basic_string_view.html https://en.cppreference.com/w/cpp/string/basic_string_view.h...
- pjc50 7mo ago
  C# gained similar benefits with Span<>/ReadOnlySpan<>. Essential for any kind of fast parser.
  - cdcarter 7mo ago
    Swift too, in 6.3!
- VorpalWay 7mo ago
  The issue with retrofitting things to an existing well established language is that those new features will likely be underutilized. Especially in other existing parts of the standard library, since changing those would break backwards compatibly. std::optional is another example of this, which is not used much in the c++ standard library, but would be much more useful if used across the board. Contrast this with Rust, which had the benefit of being developed several decades later. Here Option and str (string views) were in the standard library from the beginning, and every library and application uses them as fundamental vocabulary types. Combined with good support for chaining and working with these types (e.g. Option has map() to replace the content if it exists and just pass it along if None). Retrofitting is hard, and I have no doubt there will be new ideas that can't really be retrofitted well into Rust in another decade or two as well. Hopefully at that point something new will come along that learned from the mistakes of the past.
  - menaerus 7mo ago
    Retrofitting new patterns or ideas is underutilized only when it is not worth the change. string_view example is trivial and anyone who cared enough about the extra allocations that could have happened already (no copy-elision taking place) rolled their own version of string_view or simply used char+len pattern. Those folks do not wait for the new standard to come along when they can already have the solution now. std::optional example OTOH is also a bad example because it is heavily opinionated, and having it baked into the API across the standard library would be a really wrong choice to do.
gostsamo 7mo ago
> how much memory a native code version of the same functionality would use. native to what? how c++ is more native than python?
- VorpalWay 7mo ago
  Native code usually refers to code which is compiled to machine code (for the CPU it will run on) ahead of time, as opposed to code running in a byte code VM (possibly with JIT). I would consider all of C, C++, Zig, Rust, Fortran etc to produce native binaries. While things like Cython exist, that wasn't what was used here (and for various reasons would likely still have more overhead than those I mentioned).
- fluoridation 7mo ago
  Native to the hardware platform.
dgb23 7mo ago
Not a C++ programmer and I think the solution is neat. But it's not necessarily an apples to apples comparison. It's not unfair to python because of the runtime overhead. It's unfair because it's a different algorithm with fundamentally different memory characteristics. A fairer comparison would be to stream the file in C++ as well and maintain internal state for the count. For most people that would be the first/naive approach as well when they programmed something like this I think. And it would showcase what the actual overhead of the python version is.
- VorpalWay 7mo ago
  > A fairer comparison would be to stream the file in C++ as well and maintain internal state for the count. Wouldn't memory mapping the data in Python be the more fair comparison? If the language doesn't support that, then this seems to absolutely be a fair comparison. > For most people that would be the first/naive approach as well when they programmed something like this I think. I disagree, my mind immediately goes to mmap when I have to deal with a single file that I have to read in it's entirety. I think the non-obvious solution here is rather io-uring (which I would expect to be faster if dealing with lots of small files, as you can load them async concurrently from the file system).
  - dgb23 7mo ago
    I'd make the bet that "most people" (who can program) would not think of mmap, but either about streaming or would even just load the whole thing into memory. Ask a bunch of coding agents and they will give you these two versions, which means it's likely that the LLMs have seen these way more often than the mmap version. Both Opus and GPT even pushed back when I asked for mmap, both said it would "add complexity".
    - Filligree 7mo ago
      It does add complexity, and the optimal solution is probably not to use it. Consider what happens if a 4kB page has only a single unique word in it—you’d still need to load it to memory to read the string, it just isn’t accounted against your process (maybe). I would have expected something like this: - Scan the file serially. - For each word, find and increment a hash table entry. - Sort and print. In theory, technically, this does require slightly more memory—but it’s a tiny amount more; just a copy of each unique word, and if this is natural language then there aren’t very many. Meanwhile, OOP’s approach massively pressures the page cache once you get to the “print” step, which is going to be the bulk of the runtime. It’s not even a full copy of each unique word, actually, because you’re trading it off against the size of the string pointers. That’s… sixteen bytes minimum. A lot of words are smaller than that.
amelius 7mo ago
> AI sociopaths have purchased all the world's RAM in order to run their copyright infringement factories at full blast The ultimate bittersweet revenge would be to run our algorithms inside the RAM owned by these cloud companies. Should be possible using free accounts.
muskstinks 7mo ago
I'm always confused as hell how little insight we have in memory consumption. I look at memory profiles of rnomal apps and often think "what is burning that memory". Modern compression works so well, whats happening? Open your taskmaster and look through apps and you might ask yourself this. For example (lets ignore chrome, ms teams and all the other bloat) sublime consumes 200mb. I have 4 text files open. What is it doing? Alone for chrome to implement tab suspend took YEARS despite everyone being aware of the issue. And addons existed which were able to do this. I bought more ram just for chrome...
- Orygin 7mo ago
  200Mb for Sublime does not seem so bad when compared to Postman using 4Gb on my machine...
- veunes 7mo ago
  Part of the problem is that modern apps aren't really "one thing" anymore
- senfiaj 7mo ago
  It's partly because there are layers of abstractions (frameworks, libraries / runtimes / VM, etc). Also, today's software often has other pressures, like development time, maintainability, security, robustness, accessibility, portability (OS / CPU architecture), etc. It's partly because the complexity / demand has increased. https://waspdev.com/articles/2025-11-04/some-software-bloat-is-ok https://waspdev.com/articles/2025-11-04/some-software-bloat-...
- gwbas1c 7mo ago
  Basically, the short answer is that most memory managers allocate more memory than a process needs, and then reuse it. IE, in a JVM (Java) or dotnet (C#) process, the garbage collector allocates some memory from the operating system and keeps reusing it as it finds free memory and the program needs it. These systems are built with the assumption that RAM is cheap and CPU cycles aren't, so they are highly optimized CPU-wise, but otherwise are RAM inefficient.
- pjc50 7mo ago
90d 7mo ago
Speaking about optimization, is Windows just too far gone at this point? It is comical the amount of resources it uses at "idle".
yieldcrv 7mo ago
as long as you know what architecture questions to ask, agentic coding can help with this next phase of optimization really quickly delaying comp sci differentiation for a few months I wonder if assembly based solutions will become in vogue
callamdelaney 7mo ago
I shove everything in memory, it's a design decision. Memory is still cheap, relatively.
veunes 7mo ago
Not "C++ everywhere again" but maybe "understanding memory again"
gwbas1c 7mo ago
A lot of frameworks that use variants of "mark and sweep" garbage collection instead of automatic reference counting are built with the assumption that RAM is cheap and CPU cycles aren't, so they are highly optimized CPU-wise, but otherwise are RAM inefficient. I wonder if frameworks like dotnet or JVM will introduce reference counting as a way to lower the RAM footprint?
- vaylian 7mo ago
  Unlikely. Maybe I'm overly optimistic, but I think it's fairly likely that the RAM situation will have sorted itself out in a few years. Adding reference counting to the JVM and .NET would also take considerable time. It makes more sense for application developers to think about the unnecessary complexity that they add to software.
- xyzzy_plugh 7mo ago
  That's not strictly true. Mark and sweep is tunable in ways ARC is not. You can increase frequency, reducing memory at the cost of increased compute, for example.
  - cogman10 7mo ago
    M&S also doesn't necessitate having a moving and compacting GC. That's the thing that actually makes the JVM's heap greedy. Go also does M&S and yet uses less memory. Why? Because go isn't compacting, it's instead calling malloc and free based on the results of each GC. This means that go has slower allocation and a bigger risk of memory fragmentation, but also it keeps the go memory usage reduced compared to the JVM.
    - astrange 7mo ago
      Compacting reduces memory usage - that's why it's called compacting. The JVM uses a lot of memory a) because it's tuned for servers and not for low memory usage and b) because Java is a poorly designed language without value types.
tombert 7mo ago
I've been rewriting a lot of my stuff in Rust to save memory. Rust is high-level enough to still be fun for me (tokio gives me most of the concurrency goodies I like), but the memory usage is often like 1/10th or less compared to what I would write in Clojure. Even though I love me some lisp, pretty much all my Clojure utilities are in Rust land now.
bcjdjsndon 7mo ago
A few things - since GC languages became prevalent, and maybe high level programming in general, coders arent as economic with their designs. Memory isn't something a coder should worry about apparently. - far more people code apps in web languages because they don't know anything else. These are anywhere from 5-10 levels of abstraction away from the metal, naturally inefficient. - increasing scope... I can only describe this one by example, web browsers must implement all manner of standards etc that it's become a mammoth task, especially compared to 90s. Same for compilers, oses, heck even computers thenselves were all one-man jobs at some point because things were simpler cos we knew less.
1vuio0pswjnm7 7mo ago
Been waiting for online commentary about programming to start acknowledging this situation as it pertains to writing programs Memory and storage are not "cheap" anymore. Power may also rise in cost Under these conditions, memory usage and binary size are irrefutably relevant^1 To some, this might feel like going backwards in time toward the mainframe era. Another current HN item with over 100 points, "Hold on to your hardware", reflects on how consumer hardware may change as a result To me, the past was a time of greater software efficiency; arguably this was necessitated by cost. Perhaps higher costs in the present and future could lead to better software quality. But whether today's programmers are up for the challenge is debatable. It's like young people in finance whose only experience is in a world with "zero" interest rates. It's easier to whine about lowering rates than to adapt With the money and poltical support available to "AI" companies, the incentive for efficiency of any kind is lacking. Perhaps their "no limits" operations, e.g., its effects on supply, may provide an incentive for others' efficiency 1. As an underpowered computer user that compiles own OS and writes own simple programs, I've always rejected large binary size and excessive memory use, even in times of "abundance"
- jasonjmcghee 7mo ago
  Idk about "memory is cheap" but storage certainly is. Whether you're talking about buying or renting in the cloud - RAM is 3 orders of magnitude more expensive
- 1vuio0pswjnm7 7mo ago
  Hard drive prices increase 40-60% in months https://storedbits.com/hard-drive-prices-increase/ https://storedbits.com/hard-drive-prices-increase/ It seems like the AI apocalypse has come for hard drives https://www.ign.com/articles/the-ai-apocalypse-has-come-for-hard-drives-now https://www.ign.com/articles/the-ai-apocalypse-has-come-for-... Expect HDD, SSD shortages as AI rewrites the rules of storage hierarchy; multiple companies announce price hikes https://www.tomshardware.com/pc-components/storage/expect-hdd-ssd-shortages-as-ai-rewrites-the-rules-of-storage-hierarchy-multiple-companies-announce-price-hikes-too https://www.tomshardware.com/pc-components/storage/expect-hd... Hard Drives Are Sold Out for 2026; AI Data Centers Are to Blame https://www.gadgetreview.com/hard-drives-are-sold-out-for-2026-and-ai-data-centers-are-to-blame https://www.gadgetreview.com/hard-drives-are-sold-out-for-20... AI Data Centers Are Causing a Surge in Hard Drive Prices https://technewsjunkies.com/fundings-and-exits/ai-data-centers-are-causing-a-surge-in-hard-drive-prices/ https://technewsjunkies.com/fundings-and-exits/ai-data-cente...
zahlman 7mo ago
> This sounds like a job for Python. Indeed, an implementation takes fewer than 30 lines of code. I don't know if the implementation is written in a "low-level" way to be more accessible to users of other programming languages, but it can certainly be done more simply leveraging the standard library: from collections import Counter import sys with open(sys.argv[1]) as f: words = Counter(word for line in f for word in line.split()) for word, count in words.most_common(): print(count, word) At the very least, manually creating a (count, word) list from the dict items and then sorting and reversing it in-place is ignoring common idioms. `sorted` creates a copy already, and it can be passed a sort key and an option to sort in reverse order. A pure dict version could be: import sys with open(sys.argv[1]) as f: counts = {} for line in f: for word in line.split(): counts[word] = counts.get(word, 0) + 1 stats = sorted(counts.items(), key=lambda item: item[1], reverse=True) for word, count in stats: print(count, word) (No, of course none of this is going to improve memory consumption meaningfully; maybe it's even worse, although intuitively I expect it to make very little difference either way. But I really feel like if you're going to pay the price for Python, you should get this kind of convenience out of it.) Anyway, none of this is exactly revelatory. I was hoping we'd see some deeper investigation of what is actually being allocated. (Although I guess really the author's goal is to promote this Pystd project. It does look pretty neat.)
yakkomajuri 7mo ago
The abrupt ending was funny and then I realized the author is Finnish and it all made sense. Nice post. (P.S. I'm also Finnish)
wbsun 7mo ago
It’s less about 'old vs. new' and more about the evolving trade-offs dictated by the constraints of the era. There have always been engineers trying to squeeze every last drop of performance out of the bits available to them.
baud9600 7mo ago
Strange days we live in. Python and C++? What about a line of bash: tr -s '[:space:]' '\n' < file.txt | sort | uniq -c | sort -rn I’d like to know the memory profile of this. The bottleneck is obviously sort which buffers everything in memory. So if we replace this with awk using a hash map to keep count of unique words, then it’s a much smaller data set in memory: tr -s '[:space:]' '\n' < file.txt | awk '{c[$0]++} END{for(w in c) print c[w], w}' | sort -rn I’m guessing this will beat Python and C++?
- pjscott 7mo ago
  > I’d like to know the memory profile of this. The bottleneck is obviously sort which buffers everything in memory. That's not obvious to me. I checked the manuals for sort(1) in GNU and FreeBSD, and neither of them buffer everything in memory by default. Instead they read chunks to an in-memory buffer, sort each chunk, and (if there are multiple chunks) use the filesystem as temporary storage for an external mergesort. This sorting program was originally developed with memory-starved computers in mind, and the legacy shows.
- knome 7mo ago
  >which buffers everything in memory gnu sort can spill to disk. it has a --buffer-size option if you want to manually control the RAM buffer size, and a --temporary-directory option for instructing it where to spill data to disk during sort if need be.
- alok-g 7mo ago
  Isn't using bash effectively saying, I have a bunch of functions already written in say C which I'll use but would not count those towards the lines of code? You could do the same in C and C++ itself too. In other words, I am not sure if the comparison you are making is a fundamental one.
kristianp 7mo ago
How much memory does the C++ compiler use when compiling the program? I wonder how that compares to the python program? Not a completely unrelated metric. Would the rust compiler use much more memory compiling a comparable program to the C++ version?
- xantronix 7mo ago
  The point is a bit moot. The compiler is the one place where it's okay to be a little spendy, because you're not compiling each time you need to execute the program.
perching_aix 7mo ago
Since we're doing this, I do wonder then: is going from a 1.3K input file to 21K peak memory usage (16x) really optimal? It's certainly a lot better than 1000x, sure, but still surprised me.
6510 7mo ago
Someone on youtube[0] suggested we use (something like) KolibriOS[1] in a vm as a kind of webbrowser. Then we can have snappy effective desktop apps for everything cross platform(!) The 12 MB OS looks surprisingly mature. We are so conditioned for bloat that with each click I'm surprised how fast it responds. I don't remember ever being surprised by the same thing twice in a row but here it stays surprising how everything opens in the next frame even on very poor hardware. Besides from a vm I read there is a synery[3] client for it[4]. I've used crappy pc's on extra screens and having a dedicated machine for a single application is fun, useful and it makes old stuff useful. You can run heavy applications on the main computer, it's unimportant the extra cant do it. [0] - https://www.youtube.com/watch?v=v3NVKOsWkQs https://www.youtube.com/watch?v=v3NVKOsWkQs [1] - https://www.kolibrios.org/en https://www.kolibrios.org/en [3] - https://www.youtube.com/watch?v=tlt7X0H5GJw https://www.youtube.com/watch?v=tlt7X0H5GJw [4] - https://board.kolibrios.org/viewtopic.php?t=2544 https://board.kolibrios.org/viewtopic.php?t=2544

231 points by ibobev 7mo ago

est 7mo ago

I think py version can be shortened as: from collections import Counter stats = Counter(x.strip() for l in open(sys.argv[1]) for x in l)

voidUpdate 7mo ago
Would that decrease memory usage though?
- est 7mo ago
  see https://news.ycombinator.com/item?id=47543756 https://news.ycombinator.com/item?id=47543756

griffindor 7mo ago

Nice! > Peak memory consumption is 1.3 MB. At this point you might want to stop reading and make a guess on how much memory a native code version of the same functionality would use. I wish I knew the input size when attempting to estimate, but I suppose part of the challenge is also estimating the runtime's startup memory usage too. > Compute the result into a hash table whose keys are string views, not strings If the file is mmap'd, and the string view points into that, presumably decent performance depends on the page cache having those strings in RAM. Is that included in the memory usage figures? Nonetheless, it's a nice optimization that the kernel chooses which hash table keys to keep hot. The other perspective on this is that we sought out languages like Python/Ruby because the development cost was high, relative to the hardware. Hardware is now more expensive, but development costs are cheaper too. The take away: expect more push towards efficiency!

pjc50 7mo ago
>> Peak memory consumption is 1.3 MB. At this point you might want to stop reading and make a guess on how much memory a native code version of the same functionality would use. At this point I'd make two observations: - how big is the text file? I bet it's a megabyte, isn't it? Because the "naive" way to do it is to read the whole thing into memory. - all these numbers are way too small to make meaningful distinctions. Come back when you have a gigabyte. It gets more interesting when the file doesn't fit into RAM at all. The state of the art here is : https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times-by-70/ https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times... , wherein our hero finds the terrible combination of putting the whole file in a single string and then running strlen() on it for every character.
- deleted 7mo ago
  [deleted]
- dgb23 7mo ago
  > all these numbers are way too small to make meaningful distinctions. Come back when you have a gigabyte. I have to disagree. Bad performance is often a result of a death of a thousands cuts. This function might be one among countless similarly inefficient library calls, programs and so on.
  - rcxdude 7mo ago
    If you're not putting a representative amount of data through the test, you have no idea if the resource usage you're seeing scales with the amount of data or is just a fixed overhead if the runtime.
- kloop 7mo ago
  > how big is the text file? I bet it's a megabyte, isn't it? The edit in the article says ~1.5kb
  - pjc50 7mo ago
    Single page on many systems, which makes using mmap() for it even funnier.
veunes 7mo ago
I suspect it'll be selective
zozbot234 7mo ago
> If the file is mmap'd, and the string view points into that, presumably decent performance depends on the page cache having those strings in RAM. Not so much, because you only need some fraction of that memory when the program is actually running; the OS is free to evict it as soon as it needs the RAM for something else. Non-file-backed memory can only be evicted by swapping it out and that's way more expensive,
- astrange 7mo ago
  All major OSes (well Windows and macOS) do in-memory compression before swap, which is cheaper than evicting a file-backed page. But still slow, so you don't want to rely on it.

biorach 7mo ago

"copyright infringement factories"

maipen 7mo ago
Tells you right away where this is coming from.
- muskstinks 7mo ago
  The critisism is valid. The problem is how you value this critism. I agree they are stealing it but I also see the benefit of it for society and for myself. Suckerberg downloaded terabytes of books for training, while people around me got sued to hell 20 years ago for downloading one mp3 file.
  - anthk 7mo ago
    Anna's Archive. Aaron Swartz. FB and so are CIA fronts and they can do anything they please. Until they hit against Disney and lobbying giants and if a CIA idiot tries to sue/bribe/blackmail them they can order Hollywood to rot their images into pieces with all the wars they promoted in Middle East and Latin America just to fill the wallets of CEO's. That among some social critique movie on FB about getting illegal user data all over the world to deny insurances and whatnot. And OFC with a clear mention of the Epstein case with related people, just in case the Americans forgot about it. Then the US industry and military complex would collapse in months with brainwashed kids running away from the army. Not to mention to the Call of Duty franchise and the like. It would be the end of Boeing and several more, of course. To hell to profit driven wars for nothing. Ah, yes, AIPAC lobbies and the like. Good luck taming right wing wackos hating the MAGA cult more than the 'woke' people themselves. These will be the first ones against you after sinking the US image for decades, even more than the illegal Iraq war with no WMD's and the Bush/Cheney mafia. The outcome of this? proper and serious engineering a la Airbus. Instant profit-driven MBA and war sickos being kicked out from the spot. OFC the AI snakeoil sellers except for the classical AI/NN against concrete cases (image detection and the like), these will survive fine, even better because these kind of jobs are highly specific and they are not statistical text parrots. They can provide granted results unlike LLM's prone to degrade because the human based content feeding needs to be continuous, while for tumour detection a big enough sample can cover a 99% of the cases. R&D on electric vehicles/energy and nuclear power like nowhere else. And, for sure, the EV equivalent of a Ford T for Americans. A cheap and reliable one, good enough for the common Joe/Mary without being a luxury item. A new Golden Age would rise, for sure. But the oil mafia will try to fight them like crazy.

tzot 7mo ago

Well, we can use memoryview for the dict generation avoiding creation of string objects until the time for the output: import re, operator def count_words(filename): with open(filename, 'rb') as fp: data= memoryview(fp.read()) word_counts= {} for match in re.finditer(br'\S+', data): word= data[match.start(): match.end()] try: word_counts[word]+= 1 except KeyError: word_counts[word]= 1 word_counts= sorted(word_counts.items(), key=operator.itemgetter(1), reverse=True) for word, count in word_counts: print(word.tobytes().decode(), count) We could also use `mmap.mmap`.

akx 7mo ago
This doesn't do the same thing though, since it's not Unicode aware. >>> 'x\u2009 a'.split() ['x', 'a'] # incorrect; in bytes mode, `\S` doesn't know about unicode whitespace >>> list(re.finditer(br'\S+', 'x\u2009 a'.encode())) [<re.Match object; span=(0, 4), match=b'x\xe2\x80\x89'>, <re.Match object; span=(7, 8), match=b'a'>] # correct, in unicode mode >>> list(re.finditer(r'\S+', 'x\u2009 a')) [<re.Match object; span=(0, 1), match='x'>, <re.Match object; span=(5, 6), match='a'>]
- contravariant 7mo ago
  There's bound to be a way to turn a stream of bytes into a stream of unicode code points (at least I think that's what python is doing for strings). Though I'm explicitly not volunteering to write the code for it.
  - est 7mo ago
    import mmap, codecs from collections import Counter def word_count(filepath): freq = Counter() decode = codecs.getincrementaldecoder('utf-8')().decode with open(filepath, 'rb') as f, mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) as mm: for chunk in iter(lambda: mm.read(65536), b''): freq.update(decode(chunk).split()) freq.update(decode(b'', final=True).split()) return freq

fix4fun 7mo ago

Digression: Nowadays when RAM is expensive good old zram is gaining popularity ;) Try to check on trends.google.com . Since 2025-09 search for it doubled ;)

canpan 7mo ago

String views were a solid addition to C++. Still underutilized. It does not matter which language you are using when you make thousands of tiny memory allocations during parsing. https://en.cppreference.com/w/cpp/string/basic_string_view.html https://en.cppreference.com/w/cpp/string/basic_string_view.h...

pjc50 7mo ago
C# gained similar benefits with Span<>/ReadOnlySpan<>. Essential for any kind of fast parser.
- cdcarter 7mo ago
  Swift too, in 6.3!
VorpalWay 7mo ago
The issue with retrofitting things to an existing well established language is that those new features will likely be underutilized. Especially in other existing parts of the standard library, since changing those would break backwards compatibly. std::optional is another example of this, which is not used much in the c++ standard library, but would be much more useful if used across the board. Contrast this with Rust, which had the benefit of being developed several decades later. Here Option and str (string views) were in the standard library from the beginning, and every library and application uses them as fundamental vocabulary types. Combined with good support for chaining and working with these types (e.g. Option has map() to replace the content if it exists and just pass it along if None). Retrofitting is hard, and I have no doubt there will be new ideas that can't really be retrofitted well into Rust in another decade or two as well. Hopefully at that point something new will come along that learned from the mistakes of the past.
- menaerus 7mo ago
  Retrofitting new patterns or ideas is underutilized only when it is not worth the change. string_view example is trivial and anyone who cared enough about the extra allocations that could have happened already (no copy-elision taking place) rolled their own version of string_view or simply used char+len pattern. Those folks do not wait for the new standard to come along when they can already have the solution now. std::optional example OTOH is also a bad example because it is heavily opinionated, and having it baked into the API across the standard library would be a really wrong choice to do.

gostsamo 7mo ago

> how much memory a native code version of the same functionality would use. native to what? how c++ is more native than python?

VorpalWay 7mo ago
Native code usually refers to code which is compiled to machine code (for the CPU it will run on) ahead of time, as opposed to code running in a byte code VM (possibly with JIT). I would consider all of C, C++, Zig, Rust, Fortran etc to produce native binaries. While things like Cython exist, that wasn't what was used here (and for various reasons would likely still have more overhead than those I mentioned).
fluoridation 7mo ago
Native to the hardware platform.

dgb23 7mo ago

Not a C++ programmer and I think the solution is neat. But it's not necessarily an apples to apples comparison. It's not unfair to python because of the runtime overhead. It's unfair because it's a different algorithm with fundamentally different memory characteristics. A fairer comparison would be to stream the file in C++ as well and maintain internal state for the count. For most people that would be the first/naive approach as well when they programmed something like this I think. And it would showcase what the actual overhead of the python version is.

VorpalWay 7mo ago
> A fairer comparison would be to stream the file in C++ as well and maintain internal state for the count. Wouldn't memory mapping the data in Python be the more fair comparison? If the language doesn't support that, then this seems to absolutely be a fair comparison. > For most people that would be the first/naive approach as well when they programmed something like this I think. I disagree, my mind immediately goes to mmap when I have to deal with a single file that I have to read in it's entirety. I think the non-obvious solution here is rather io-uring (which I would expect to be faster if dealing with lots of small files, as you can load them async concurrently from the file system).
- dgb23 7mo ago
  I'd make the bet that "most people" (who can program) would not think of mmap, but either about streaming or would even just load the whole thing into memory. Ask a bunch of coding agents and they will give you these two versions, which means it's likely that the LLMs have seen these way more often than the mmap version. Both Opus and GPT even pushed back when I asked for mmap, both said it would "add complexity".
  - Filligree 7mo ago
    It does add complexity, and the optimal solution is probably not to use it. Consider what happens if a 4kB page has only a single unique word in it—you’d still need to load it to memory to read the string, it just isn’t accounted against your process (maybe). I would have expected something like this: - Scan the file serially. - For each word, find and increment a hash table entry. - Sort and print. In theory, technically, this does require slightly more memory—but it’s a tiny amount more; just a copy of each unique word, and if this is natural language then there aren’t very many. Meanwhile, OOP’s approach massively pressures the page cache once you get to the “print” step, which is going to be the bulk of the runtime. It’s not even a full copy of each unique word, actually, because you’re trading it off against the size of the string pointers. That’s… sixteen bytes minimum. A lot of words are smaller than that.

amelius 7mo ago

> AI sociopaths have purchased all the world's RAM in order to run their copyright infringement factories at full blast The ultimate bittersweet revenge would be to run our algorithms inside the RAM owned by these cloud companies. Should be possible using free accounts.

muskstinks 7mo ago

I'm always confused as hell how little insight we have in memory consumption. I look at memory profiles of rnomal apps and often think "what is burning that memory". Modern compression works so well, whats happening? Open your taskmaster and look through apps and you might ask yourself this. For example (lets ignore chrome, ms teams and all the other bloat) sublime consumes 200mb. I have 4 text files open. What is it doing? Alone for chrome to implement tab suspend took YEARS despite everyone being aware of the issue. And addons existed which were able to do this. I bought more ram just for chrome...

Orygin 7mo ago
200Mb for Sublime does not seem so bad when compared to Postman using 4Gb on my machine...
veunes 7mo ago
Part of the problem is that modern apps aren't really "one thing" anymore
senfiaj 7mo ago
It's partly because there are layers of abstractions (frameworks, libraries / runtimes / VM, etc). Also, today's software often has other pressures, like development time, maintainability, security, robustness, accessibility, portability (OS / CPU architecture), etc. It's partly because the complexity / demand has increased. https://waspdev.com/articles/2025-11-04/some-software-bloat-is-ok https://waspdev.com/articles/2025-11-04/some-software-bloat-...
gwbas1c 7mo ago
Basically, the short answer is that most memory managers allocate more memory than a process needs, and then reuse it. IE, in a JVM (Java) or dotnet (C#) process, the garbage collector allocates some memory from the operating system and keeps reusing it as it finds free memory and the program needs it. These systems are built with the assumption that RAM is cheap and CPU cycles aren't, so they are highly optimized CPU-wise, but otherwise are RAM inefficient.
pjc50 7mo ago

90d 7mo ago

Speaking about optimization, is Windows just too far gone at this point? It is comical the amount of resources it uses at "idle".

yieldcrv 7mo ago

as long as you know what architecture questions to ask, agentic coding can help with this next phase of optimization really quickly delaying comp sci differentiation for a few months I wonder if assembly based solutions will become in vogue

callamdelaney 7mo ago

I shove everything in memory, it's a design decision. Memory is still cheap, relatively.

veunes 7mo ago

Not "C++ everywhere again" but maybe "understanding memory again"

gwbas1c 7mo ago

A lot of frameworks that use variants of "mark and sweep" garbage collection instead of automatic reference counting are built with the assumption that RAM is cheap and CPU cycles aren't, so they are highly optimized CPU-wise, but otherwise are RAM inefficient. I wonder if frameworks like dotnet or JVM will introduce reference counting as a way to lower the RAM footprint?

vaylian 7mo ago
Unlikely. Maybe I'm overly optimistic, but I think it's fairly likely that the RAM situation will have sorted itself out in a few years. Adding reference counting to the JVM and .NET would also take considerable time. It makes more sense for application developers to think about the unnecessary complexity that they add to software.
xyzzy_plugh 7mo ago
That's not strictly true. Mark and sweep is tunable in ways ARC is not. You can increase frequency, reducing memory at the cost of increased compute, for example.
- cogman10 7mo ago
  M&S also doesn't necessitate having a moving and compacting GC. That's the thing that actually makes the JVM's heap greedy. Go also does M&S and yet uses less memory. Why? Because go isn't compacting, it's instead calling malloc and free based on the results of each GC. This means that go has slower allocation and a bigger risk of memory fragmentation, but also it keeps the go memory usage reduced compared to the JVM.
  - astrange 7mo ago
    Compacting reduces memory usage - that's why it's called compacting. The JVM uses a lot of memory a) because it's tuned for servers and not for low memory usage and b) because Java is a poorly designed language without value types.

tombert 7mo ago

I've been rewriting a lot of my stuff in Rust to save memory. Rust is high-level enough to still be fun for me (tokio gives me most of the concurrency goodies I like), but the memory usage is often like 1/10th or less compared to what I would write in Clojure. Even though I love me some lisp, pretty much all my Clojure utilities are in Rust land now.

bcjdjsndon 7mo ago

A few things - since GC languages became prevalent, and maybe high level programming in general, coders arent as economic with their designs. Memory isn't something a coder should worry about apparently. - far more people code apps in web languages because they don't know anything else. These are anywhere from 5-10 levels of abstraction away from the metal, naturally inefficient. - increasing scope... I can only describe this one by example, web browsers must implement all manner of standards etc that it's become a mammoth task, especially compared to 90s. Same for compilers, oses, heck even computers thenselves were all one-man jobs at some point because things were simpler cos we knew less.

1vuio0pswjnm7 7mo ago

Been waiting for online commentary about programming to start acknowledging this situation as it pertains to writing programs Memory and storage are not "cheap" anymore. Power may also rise in cost Under these conditions, memory usage and binary size are irrefutably relevant^1 To some, this might feel like going backwards in time toward the mainframe era. Another current HN item with over 100 points, "Hold on to your hardware", reflects on how consumer hardware may change as a result To me, the past was a time of greater software efficiency; arguably this was necessitated by cost. Perhaps higher costs in the present and future could lead to better software quality. But whether today's programmers are up for the challenge is debatable. It's like young people in finance whose only experience is in a world with "zero" interest rates. It's easier to whine about lowering rates than to adapt With the money and poltical support available to "AI" companies, the incentive for efficiency of any kind is lacking. Perhaps their "no limits" operations, e.g., its effects on supply, may provide an incentive for others' efficiency 1. As an underpowered computer user that compiles own OS and writes own simple programs, I've always rejected large binary size and excessive memory use, even in times of "abundance"

jasonjmcghee 7mo ago
Idk about "memory is cheap" but storage certainly is. Whether you're talking about buying or renting in the cloud - RAM is 3 orders of magnitude more expensive
1vuio0pswjnm7 7mo ago
Hard drive prices increase 40-60% in months https://storedbits.com/hard-drive-prices-increase/ https://storedbits.com/hard-drive-prices-increase/ It seems like the AI apocalypse has come for hard drives https://www.ign.com/articles/the-ai-apocalypse-has-come-for-hard-drives-now https://www.ign.com/articles/the-ai-apocalypse-has-come-for-... Expect HDD, SSD shortages as AI rewrites the rules of storage hierarchy; multiple companies announce price hikes https://www.tomshardware.com/pc-components/storage/expect-hdd-ssd-shortages-as-ai-rewrites-the-rules-of-storage-hierarchy-multiple-companies-announce-price-hikes-too https://www.tomshardware.com/pc-components/storage/expect-hd... Hard Drives Are Sold Out for 2026; AI Data Centers Are to Blame https://www.gadgetreview.com/hard-drives-are-sold-out-for-2026-and-ai-data-centers-are-to-blame https://www.gadgetreview.com/hard-drives-are-sold-out-for-20... AI Data Centers Are Causing a Surge in Hard Drive Prices https://technewsjunkies.com/fundings-and-exits/ai-data-centers-are-causing-a-surge-in-hard-drive-prices/ https://technewsjunkies.com/fundings-and-exits/ai-data-cente...

zahlman 7mo ago

> This sounds like a job for Python. Indeed, an implementation takes fewer than 30 lines of code. I don't know if the implementation is written in a "low-level" way to be more accessible to users of other programming languages, but it can certainly be done more simply leveraging the standard library: from collections import Counter import sys with open(sys.argv[1]) as f: words = Counter(word for line in f for word in line.split()) for word, count in words.most_common(): print(count, word) At the very least, manually creating a (count, word) list from the dict items and then sorting and reversing it in-place is ignoring common idioms. `sorted` creates a copy already, and it can be passed a sort key and an option to sort in reverse order. A pure dict version could be: import sys with open(sys.argv[1]) as f: counts = {} for line in f: for word in line.split(): counts[word] = counts.get(word, 0) + 1 stats = sorted(counts.items(), key=lambda item: item[1], reverse=True) for word, count in stats: print(count, word) (No, of course none of this is going to improve memory consumption meaningfully; maybe it's even worse, although intuitively I expect it to make very little difference either way. But I really feel like if you're going to pay the price for Python, you should get this kind of convenience out of it.) Anyway, none of this is exactly revelatory. I was hoping we'd see some deeper investigation of what is actually being allocated. (Although I guess really the author's goal is to promote this Pystd project. It does look pretty neat.)

yakkomajuri 7mo ago

The abrupt ending was funny and then I realized the author is Finnish and it all made sense. Nice post. (P.S. I'm also Finnish)

wbsun 7mo ago

It’s less about 'old vs. new' and more about the evolving trade-offs dictated by the constraints of the era. There have always been engineers trying to squeeze every last drop of performance out of the bits available to them.

baud9600 7mo ago

Strange days we live in. Python and C++? What about a line of bash: tr -s '[:space:]' '\n' < file.txt | sort | uniq -c | sort -rn I’d like to know the memory profile of this. The bottleneck is obviously sort which buffers everything in memory. So if we replace this with awk using a hash map to keep count of unique words, then it’s a much smaller data set in memory: tr -s '[:space:]' '\n' < file.txt | awk '{c[$0]++} END{for(w in c) print c[w], w}' | sort -rn I’m guessing this will beat Python and C++?

pjscott 7mo ago
> I’d like to know the memory profile of this. The bottleneck is obviously sort which buffers everything in memory. That's not obvious to me. I checked the manuals for sort(1) in GNU and FreeBSD, and neither of them buffer everything in memory by default. Instead they read chunks to an in-memory buffer, sort each chunk, and (if there are multiple chunks) use the filesystem as temporary storage for an external mergesort. This sorting program was originally developed with memory-starved computers in mind, and the legacy shows.
knome 7mo ago
>which buffers everything in memory gnu sort can spill to disk. it has a --buffer-size option if you want to manually control the RAM buffer size, and a --temporary-directory option for instructing it where to spill data to disk during sort if need be.
alok-g 7mo ago
Isn't using bash effectively saying, I have a bunch of functions already written in say C which I'll use but would not count those towards the lines of code? You could do the same in C and C++ itself too. In other words, I am not sure if the comparison you are making is a fundamental one.

kristianp 7mo ago

How much memory does the C++ compiler use when compiling the program? I wonder how that compares to the python program? Not a completely unrelated metric. Would the rust compiler use much more memory compiling a comparable program to the C++ version?

xantronix 7mo ago
The point is a bit moot. The compiler is the one place where it's okay to be a little spendy, because you're not compiling each time you need to execute the program.

perching_aix 7mo ago

Since we're doing this, I do wonder then: is going from a 1.3K input file to 21K peak memory usage (16x) really optimal? It's certainly a lot better than 1000x, sure, but still surprised me.

6510 7mo ago

Someone on youtube[0] suggested we use (something like) KolibriOS[1] in a vm as a kind of webbrowser. Then we can have snappy effective desktop apps for everything cross platform(!) The 12 MB OS looks surprisingly mature. We are so conditioned for bloat that with each click I'm surprised how fast it responds. I don't remember ever being surprised by the same thing twice in a row but here it stays surprising how everything opens in the next frame even on very poor hardware. Besides from a vm I read there is a synery[3] client for it[4]. I've used crappy pc's on extra screens and having a dedicated machine for a single application is fun, useful and it makes old stuff useful. You can run heavy applications on the main computer, it's unimportant the extra cant do it. [0] - https://www.youtube.com/watch?v=v3NVKOsWkQs https://www.youtube.com/watch?v=v3NVKOsWkQs [1] - https://www.kolibrios.org/en https://www.kolibrios.org/en [3] - https://www.youtube.com/watch?v=tlt7X0H5GJw https://www.youtube.com/watch?v=tlt7X0H5GJw [4] - https://board.kolibrios.org/viewtopic.php?t=2544 https://board.kolibrios.org/viewtopic.php?t=2544