14 ms·
How do functions like alloca allocate memory from the stack?
- kjellsbells 2mo agoOne thing that scares me a little is whether there are younger developers, say, 25-40, who can and want to pick up the mantle of Windows internals gurus. I mean, Chen has decades of winternals in his head. Microsoft has been gutting their staff for years now. When the Petzold/Chen generation hang up their spurs, does Microsoft still have a critical mass of people who understand Windows from the metal up?
- majorchord 2mo agoMaybe they'll just train Copilot on their code to help them.
- im3w1l 2mo agoOne trend to watch is AI cheat devices. Instead of running detectable software they have a fully separate device that uses AI for object detection and aimbotting. If cheaters move to using those, then the argument for kernel mode anticheat weakens. And that is the cornerstone keeping gamers on windows.
- not_a9 2mo agoWhile I’m not sure if this is a bot (where did vidya enter the convo?), game hacking on both cheat and anticheat side has genuinely deep Windows internals knowledge (admittedly somewhat lopsided, but deep nonetheless)
- im3w1l 2mo agoProducer and consumer sides are tied. If there is strong demand for windows development then there will be money sloshing around which will attract devs. I predict demand will decrease, at least for this particular niche. To go even further off topic, being called a bot is certainly a wake up call for me that the internet is dying, and I'm not ready for it, and need to reposition myself asap somehow.
- pjc50 2mo agoWindows demand is overwhelmingly corporate. Just like Nvidia is barely bothering with gamers at the moment, Windows is not being kept alive for games.
- matheusmoreira 2mo agoNow I'm looking forward to these AI cheat devices. It's going to be hilarious if the kernel anticheat malware finally gets killed by AI aimbots of all things.
- inigyou 2mo agoThe term "AI" used in video games is not really related to the current LLM craze.
- matheusmoreira 2mo agoIn what other ways could "AI cheat devices" and "uses AI for object detection and aimbotting" possibly be interpreted?
- inigyou 2mo agoAs a misunderstanding. The curenet high-end cheat devices use the PCIe bus to read main memory without relying on any AI. Since they also plug in to the HDMI port to overlay their information on top of the game screen, this might be misunderstood as them detecting objects in the video output. But detecting objects in video also doesn't fall under the new definition of AI, which means LLMs and image diffusers.
- not_a9 2mo agoThe significantly increased capabilities of cheats if you can read/write memory and the dogshit that AI-oriented anticheat approaches continue producing (look at VACnet, though I am sure there are other companies producing better things like Anybrain - the thing is, 99% of the time the products of said companies are integrated along with a proper strong anticheat like EAC) will ensure kernel mode anticheats will be alive and well.
- LatencyKills 2mo agoI was a dev on the Visual Studio and Windows teams in the 90s. I’m retired but mentor CS students at two local universities. I haven’t had a student in two years that was even remotely interested in ring-0, internals, or really understanding a debugger. I’m not being critical; they are just focused on higher level abstractions.
- arjvik 2mo agoArguably the lower level abstractions are more interesting too! how exactly Windows does ring-0 is less interesting than writing your own ring-0! And unless you care about writing driver-level software for Windows or contributing to the kernel, learning this is also less useful. I'm essentially arguing that unless you work at MSFT, there's next to no reason to learn that specific abstraction layer.
- LatencyKills 2mo ago> I'm essentially arguing that unless you work at MSFT, there's next to no reason to learn that specific abstraction layer. Really? Understanding the cost of ring transitions is incredibly useful. I recently consulted with a company that was having horrible performance issues, and it came down to the fact that the primary developer didn't know that certain Win32 calls forced ring transitions. The entire fix was switching from a mutex to a critical section (one causes a ring transition, the other doesn't). Treating the OS like an impenetrable black box will bite upcoming engineers/companies... eventually.
- arjvik 2mo agoThis makes sense, and I guess as someone who’s never written Windows software I didn’t realize it was a leaky abstraction!
- AdieuToLogic 2mo agoSpeaking of debuggers... I still twitch whenever someone says "use ddd" and they are not referring to Evans' seminal work. :-D
- dataflow 2mo ago> who can Probably enough to keep Windows going, at least. > and want to Not if the pay or location is uncompetitive.
- mrheosuper 2mo agoObviously it's Copilot /s
- AdieuToLogic 2mo ago> One thing that scares me a little is whether there are younger developers, say, 25-40, who can and want to pick up the mantle of Windows internals gurus. A similar "brain drain" has occurred in macOS (formerly known as OS-X) over the years, as evident in man page documentation for "newer" daemons shipped. An easy way to verify this is to run: ps -A | awk '{ print $4 }' | grep 'libexec/.*[a-z]d$' And compare the man pages for the daemons running with the man page for `launchd`. While this exercise is illuminating, it is also depressing IMHO.
- pjmlp 2mo agoAnd the documentation, now it is mostly generated, the famous Apple books are now gone, at least the archive is still available.
- masklinn 2mo ago> formerly known as OS-X It was never OS-X, it was OS X, and originally Mac OS X, as in the one after Mac OS 9. The Mac prefix was dropped with Lion (10.7). The Mac OS lingo having itself been introduced with 7.6, before that the OS core was called System.
- AdieuToLogic 2mo ago>> formerly known as OS-X > It was never OS-X, it was OS X ... If my worst sin is an introduction of a hyphen, then I can live with that.
- charcircuit 2mo agoWhy do you think that is not already happening within Microsoft?
- pjmlp 2mo agoWhy do you think Windows is currently such a mess? Microsoft new blood has been educated on Macs and ChromeOS, even if they do games it is most likely consoles. On WinUI community calls you usually would get puzzled faces when the Q&A touched when would WinUI be able to do "insert basic Win32/Forms/WPF" feature. Management apparently doesn't care they actually understand Windows, or get the required trainings to meet the quality of their predecessors. That is how you get Webview2 all over the place.
- wolfi1 2mo agoi fear the times when the chip makers change their architectures and the OS makers have to change the inner working of their OSes but then again, I guess there is an equivalent in the chip-making industry as well
- mesrik 2mo ago>Why do you think Windows is currently such a mess? There have been some writings and posts here about Microsoft. Here is one from last spring, from a guy that was long time Windows core developer and moved to Azure group. It's well worth reading, what he writes about challenges they have had and most likely still have if not even worse now. https://isolveproblems.substack.com/p/how-microsoft-vaporized-a-trillion https://isolveproblems.substack.com/p/how-microsoft-vaporize... and the related HN thread https://news.ycombinator.com/item?id=47616242 https://news.ycombinator.com/item?id=47616242 And from what I've understood old chaps like Dave Cutler are involved much less than they were for a very long time.
- pjmlp 2mo agoI have read that, yeah it also shows a lot how things changed. As per his interview on Dave's Garage, besides being of an age where he really doesn't need to work, at the time he was involved on getting Linux running on XBox on Azure, apparently Microsoft uses idle consoles from XBox Cloud for AI. https://youtu.be/xi1Lq79mLeE?t=10743 https://youtu.be/xi1Lq79mLeE?t=10743
- delta_p_delta_x 2mo agoAs someone within that age group who moved back to Windows for development and entertainment, I find systems programming on Windows more fun and engaging than on competition OSs. Oddly enough the open-source nature of the latter kind of takes away some of the thrill. Eerything is just... there, whereas with Windows there's always quite a bit of digging and investigation involved. Or maybe this is Stockholm syndrome; I dunno.
- stelonix 2mo agoFunny feelings in my tummy reading this! I worked with winapi back in late 2000s to early 2010s and I remember having some great fun with it. Although there was MFC and WPF I didn't want to learn them because I wanted the fastest & leanest (catch the reference :) executable I could get; I'd then run gnu strip over the .exe too. Stack Overflow was essential to figure out arcane flags that could solve my issues (when even MSDN, another great site with its examples, couldn't) and Raymond was very present in SO at that time, iirc he replied to one of my questions too. That's when I found his blog, always great reads! Now I've been a 14-year Linux user and none of the toolkits and libraries give anything close to the winapi experience.
- delta_p_delta_x 2mo ago> fastest & leanest Surely you mean leanest and meanest :P I'd also say that tooling on Windows is simultaneously better and easier to use than on Linux; the noob case of green play button in an IDE is taken care of, but if you want detailed performance and memory profiling, record-replay debugging, hot-reload, all of this is straightforwardly available on Windows.
- ferrow 2mo ago[flagged]
- jacknews 2mo agoOnly passingly related, some fun rust stack-allocation insanity by my 17yo son: https://ogghostjelly.github.io/slog/alloca.html https://ogghostjelly.github.io/slog/alloca.html
- jjice 2mo ago17? You should be very proud. This is good work for anyone, but especially at his age!
- anitil 2mo agoVery impressive for a 17yo!
- matheusmoreira 2mo agoThis is great. I'm learning Rust myself and your son's article contributed to my knowledge. I'm also very impressed by part 2. I have my own lisp but I haven't managed to implement a compiler or code generation yet. Really enjoyed reading about the hashmap too. The textbook solution to collisions is probing and comparison. It never occurred to me that I could just resize the underlying array until the collisions disappear altogether.
- jacknews 2mo agoIt's perhaps the least-efficient method of dealing with collisions, in keeping with the rest of his project, lol, but less code. He researched different methods. My favourite for elegance is to recursively hash the hash n times.
- rramadass 2mo agoRelated to the above, two important concepts to know w.r.t a stack are "Red Zone" and "Guard Pages". Raymond Chen again; Why do we even need to define a red zone? Can’t I just use my stack for anything? - https://devblogs.microsoft.com/oldnewthing/20190111-00/?p=100685 https://devblogs.microsoft.com/oldnewthing/20190111-00/?p=10... A closer look at the stack guard page - https://devblogs.microsoft.com/oldnewthing/20220203-00/?p=106215 https://devblogs.microsoft.com/oldnewthing/20220203-00/?p=10...
- Joker_vD 2mo agoI wonder how Linux manages without explicit _chkstk? In my experience, it feels like MAP_GROWSDOWN regions have way more than 1 guard page below its start — I can poke like a megabyte lower than its start, and the kernel will grow the memory region into there just fine.
- inigyou 2mo agoIt doesn't. This causes the StackClash vulnerability.
- rramadass 2mo agohttps://news.ycombinator.com/item?id=49343032 https://news.ycombinator.com/item?id=49343032
- rramadass 2mo agoPreventing stack guard-page hopping - https://lwn.net/Articles/725832/ https://lwn.net/Articles/725832/ According to this article, allocation in page sizes with implicit probing is used; Stack clash mitigation in GCC, Part 3 (-fstack-clash-protection option) - https://developers.redhat.com/blog/2020/05/22/stack-clash-mitigation-in-gcc-part-3# https://developers.redhat.com/blog/2020/05/22/stack-clash-mi...
- avadodin 2mo agoThank you for the context. But still. You have a desirable performance optimization feature —used in every Linux program— that happens to interfere with a lousy exploit mitigation. No one should ever need more than 64kBs for a stack anyways.
- eska 2mo agoRecommended related reading: https://nullprogram.com/blog/2024/02/05/ https://nullprogram.com/blog/2024/02/05/
- pjmlp 2mo agoOne of those functions that isn't really implementable in standard C, requiring either compiler support, or being written in straight Assembly for stack registers manipulation, one of those "micro runtime" features for C. From UNIX 7th edition all the way up to C99, when VLAs where introduced, only to be made optional in C11, and the C23 update still doesn't support automatic VLAs, only for function parameters, thus the point stands.
- stkdump 2mo agoSo what if several functions that use less than 4KB each call each other before using the stack variables in a way that the first access skips over one page?
- st_goliath 2mo agoA function call causes the return address to be pushed onto the stack, thus accessing the stack below the adjusted stack pointer address. On compiler generated x86 code, the base pointer register will quickly follow when entering the target function.
- stkdump 2mo agoMakes sense. The only exception - tail call optimization where no (new) return address is pushed - relies on cleaning up the stack before executing the tail call.
- uecker 2mo agoNobody should use alloca. If you must allocate a buffer on the stack, use a VLA, which is standardized, has proper scope-bound lifetimes, and a type that remembers the exact size. Yes, I know MSVC does not upport it. Don't use this compiler. (where credit is due: MSVC had stack probing a lot ealier than GCC and clang, and clang was very late). With gcc, you get stack probing with -fstack-clash-protection, which is similar to _chkstk but GCC inlines the stack probes. VLA got a bad name because of stack clash attacks, but without stack clash protection these attacks can appear also without VLAs (and the first such attacks actually exploited fixed-size arrays), and if you activate this protection there is IMHO not much reason to avoid VLAs. If you need a small variably-sized buffers, VLAs are almost always superior to any alternative. alloca is worse in every way (see above), a regular array with worst-case bound increases stack use relative to a VLA and does not encode the correct dynamic size which makes bounds checking weaker, and moving the buffer to the heap is slower and complicates the code. If you can not properly account for the sizes of the things you put on your stack and worry about VLAs exceeding the limit (but again, regular arrays with worst-case size increase stack usage compared to VLAs), on GCC you can use -Wvla-larger-than to make sure the size of each VLA stays bounded.
- rurban 2mo agoOn the contrary, nobody should use VLA, rather use alloca, if small enough. VLA is very new, compiler horror, poorly supported and on the chopping block. alloca is supported for decades already
- OCTAGRAM 2mo agoDevelopers should know that built-in stack is something like arena, aka dynamic region. And alloca() or VLA is using built-in stack as arena. If neither alloca() nor VLA is available, then additional arena can do the trick. Portable programs should have such fallback. Requires additional management for cleaning on exit, but that's it. I have seen malloc fallback in portable programs, but arena can be better.