21 ms·
Learning to Read X86 Assembly Language
- adamnemecek 10y agoAlso Matt Godbolt's gcc explorer is the the bee's knees for understanding assembly https://godbolt.org/ https://godbolt.org/ I think that playing around with it for 2 hours will teach you more than most classes on the topic. It really drives home why interactivity is such a bit deal in education. You should also try writing a script for counting instructions in binaries. It's pretty illuminating. Here are some sample statistics https://webcache.googleusercontent.com/search?q=cache:j0gebKGwf5YJ:https://www.strchr.com/x86_machine_code_statistics+&cd=1&hl=en&ct=clnk&gl=us https://webcache.googleusercontent.com/search?q=cache:j0gebK...
- makmanalp 10y agoThis is amazing. After a few minutes I learned so much already - try with an empty function that returns 0, then return an argument, then return an argument +1, argument^2, etc. As an aside, I was looking at the header code generated by gcc to handle the initial function call. What's the convention on how we assign parameters to registers? I'm trying stuff out, and the first paramter always seems to be edi, then esi, eax. Except when I change the types, and it turns into xmm0 and the like. And the result returned is eax too, except when it's a float, then it's xmm0. How do things know where to look?
- adamnemecek 10y ago> How do things know where to look? The compiler generates code that uses the correct register. So the compiler picks a register into which it will put the result and then generates code after the calling location that gets the result from the correct register. And yeah, there's quite a bit of surprises. E.g. I found out that gcc is smart enough to perform tail call optimizations https://godbolt.org/g/MZDmwP https://godbolt.org/g/MZDmwP
- makmanalp 10y agoOK, so it's not a magical convention - it works it out bottom-up. First decide on registers for each parameter when you generate the code for the function, then based on that generate specific code for the instances where you call that function. Cool, thank you. Also that example, heh. I tried to go back gcc versions to see if there was a case where it didn't do TCO - nope. Also, I like how returning 0 is "xor eax, eax".
- MichaelBurge 10y agoSuppose you're right and the registers are arbitrary. Then how would foreign function calls work? If you're compiling Rust code that calls into a C library, how does it know what registers to use? So the choice of registers cannot be arbitrary, unless the compiler knows the function is only used within an object file. The registers are predetermined by a convention unless you use the 'static' keyword to signal that the function is only used internally to a module, so the compiler has complete freedom to choose registers.
- pjmlp 10y agoThat is the role of operating system ABI, which definines the calling conventions between the programming languages on the OS.
- makmanalp 10y ago> If you're compiling Rust code that calls into a C library, how does it know what registers to use? Ah, interesting, I figured that the generated object files would just store some metadata on that basically.
- MichaelBurge 10y agoThat can work for statically-linked object files, but what about dynamically linked? You can load one with a function call, get back a function pointer, and invoke it like any other function. Trying to use some metadata would slow down the function call significantly, even if you tried to cache it somewhere.
- Merad 10y agoThis is determined by the calling convention [0]. If you're interacting with code that you didn't write, you need to know the calling convention that they use in order to set up arguments correctly. 0: https://en.wikipedia.org/wiki/X86_calling_conventions https://en.wikipedia.org/wiki/X86_calling_conventions
- richchurcher 10y agoThis is such gold: thanks!
- JoshTriplett 10y agoOne great example to try: add -O3 to the compiler options, then write a function returning its argument multiplied by 2, 3, 4, ... You'll see the various ways the compiler avoids emitting an actual multiplication instruction. The smallest multiplier I found that actually produced an imul: 46.
- boterock 10y agoInteresting example, I came up with clang 3.9 producing an imul with 11 though.
- JoshTriplett 10y agoWith optimization turned on? GCC produces an lea that multiplies by 5 into %rax for the return value, then another lea that multiplies that by 2 and adds the original.
- MichaelMoser123 10y agoYou still need an overview of the instructions and registers. Knowing where the operator and operand of the instruction is also helpful. (i always mix up intel and at&t notations)
- jwr 10y agoWow, thanks for posting this — it is a fantastic resource! I love it even more for using Intel x86 assembly syntax, instead of the horrible AT&T syntax.
- Graziano_M 10y agoAT&T is still an option. Which is fantastic because it's a quick way to see how the two differ.
- pmoriarty 10y agoOne of the most useful tools that helped me learn assembly was the Ketman Assembly Language Tutorial.[1] As I step through an assembly language program, it gives me instant visual feedback of the contents of each register, flag, and memory. I'm really surprised that there's nothing remotely like this on the web yet, and that I have to resort to running dosbox or freedos to have access to this super useful tool. [1] - http://www.oocities.org/siliconvalley/office/6208/ http://www.oocities.org/siliconvalley/office/6208/
- umbs 10y agoUnfortunately, the tool linked from that page (it's archived) can't be downloaded.
- pmoriarty 10y agoI think you can still find it hosted at other sites by doing a web search for "Ketman Assembly Language Tutorial" or "asmket16.zip". I also just created a .tar.xz file of my own copy of the tutorial and uploaded it to: https://www.datafilehost.com/d/6d9086c8 https://www.datafilehost.com/d/6d9086c8 I also did a base64 encoding of the same .tar.xz file and pasted it here: https://bpaste.net/raw/2521d4e2a230 https://bpaste.net/raw/2521d4e2a230 The sha256 checksum of the .tar.xz file is: 41e5d16b2e0d15f67535d725baa2fa2fec00f65003e4745b60a3fb082111f5ff Please note that this is not an official version of this tutorial. It's just what I had lying around on my disk after who knows how many years. Use at own risk. If you want an official copy, you could try contacting the author at btketman@btinternet.com
- Jugurtha 10y agoYou might like this: https://news.ycombinator.com/item?id=9322723 https://news.ycombinator.com/item?id=9322723
- qwertyuiop924 10y agox86 is the worst ISA. If you want to play with assembler without feeling a desire to stab yourself and end it all, I recommend ARM. Or go learn Z80, x86's weird, 8-bit cousin (it had a 16-bit version, but it sold poorly), which had a greater emphasis on backwards compatability (you can run code from the original 8080 on a Z80, unchanged), and is nicer to work with (because it wasn't extended in unticipated directions far beyond its original capabilities, while keeping fetishistic backwards compatability by stacking hack on top of hack on top hack. It also didn't have memory segmentation, otherwise known as The Worst Thing.) There are only two common reasons to learn Z80 assembler, though: to program the Gameboy (which runs on a modified Z80 with all the cool instructions removed), and to program a TI calculator, thus making all highschoolers in your area immensely happy. TI calculators are a comically overpriced scam, that have only survived because of the College Board, but that's another story.
- umanwizard 10y agoThis is about learning to read, not learning to write. If you're ever in a situation where you need to read assembly, it's typically not up to you what ISA it'll be in. There are two situations where I've had to read assembly: either reverse-engineering a compiled binary or trying to understand the compiler output for a small piece of a program I'm working on in a higher-level language.
- qwertyuiop924 10y agoOh. Well then, yeah. Besides, you don't have to worry about the worst of x86 in those cases. I'm pretty sure everyone has to learn at least that much x86 at some point, like it or not.
- toast0 10y agoI'm not sure everyone has to learn even that much. I've managed to avoid it until very recently and I'm more willing to dive into underlying code than most people I know (I've contributed to a FreeBSD kernel patch in TCP and an OpenSSL key exchange interoperability fix)
- pselbert 10y agoExcellent article that has context around the how and why, which I appreciated. Pat has a great "explorer" writing style. I was going to post this last night but figured it must have been posted already. I was wrong! Nice to see it ok the front page.
- pat_shaughnessy 10y agoThank you! Knowing that people enjoy reading this sort of thing makes it worthwhile.
- iamahacker2 10y agoExcellent article.
- JoeAltmaier 10y agoThe examples seem odd to me: the argument order is reversed from every Intel disassembly (or assembler) I've ever used. Addi edi, 43 is the normal way to say "Add 43 to edi". The destination register is normally first; the source register 2nd in the disassembly, right?
- ycmbntrthrwaway 10y agogdb uses AT&T syntax by default. [0] [0] https://en.wikipedia.org/wiki/X86_assembly_language#Syntax https://en.wikipedia.org/wiki/X86_assembly_language#Syntax
- sigjuice 10y agoThis article uses lldb, and not gdb. Both default to AT&T syntax, I guess.
- wolfgke 10y agoIn the examples given in the article it is written addl $42, %edi which is correct in AT&T syntax. In Intel syntax it would be add edi, 42
- raimue 10y agoThe examples in the article are in AT&T syntax, you seem to be used to Intel syntax. Just keep on reading, the article will discuss the differences between AT&T and Intel syntax.
- Gruselbauer 10y ago> It turns out x86 assembly is much simpler than Hungarian Well, what isn't? I have a knack for languages but that beautiful beast seems like an almost impenetrable fortress of strangeness. On the other hand, that encourages me to learn to read basic assembly.
- satysin 10y agoA video I like to send friends asking about how to understanding x86 assembly is https://www.youtube.com/watch?v=yOyaJXpAYZQ https://www.youtube.com/watch?v=yOyaJXpAYZQ I think the video maker does a good job of mapping a simple C program to its disassembly.
- gnuvince 10y agoNice article! It take a subject that is scary for many and does a great explaining a little bit very clearly using good visual aids. I look forward to the next articles!
- pat_shaughnessy 10y agoThanks a lot. I'm looking forward to writing it, I think it will be interesting.
- RandomInteger4 10y agoThis is the first time I actually read all the way through an article on assembly. It was nice and concise. Granted, I'll probably forget this until the next article (due to focusing on other studies), but thank you none the less.
- pat_shaughnessy 10y agoThanks! Glad to hear you made it to the bottom :) I always worry that I tend to write on and on for too long.
- ivanhoe 10y agoTo understand assembly it really helps to know at least something about how computers work on a low level. When I first tried learning it (long time ago, in a high school) I had no idea how computers really work on such a low level, how CPU's addressing registers and that kind of stuff, and while I managed to learn the syntax, even write some asm code, it was all really confusing to me. And only few years later on University, after I've learned in details about the cpu architecture, registers, buses, DMA, etc, it suddenly all started to make perfect sense and became 100x more clear and easier. So if you're interested in this, it will save you a lot of effort to invest some time first to learn the computer architecture basics, and then from there go to learn the assembly lang. Just my $0.02
- theparanoid 10y agoIt's helpful to realize x86 assembly is not what's executed by the machine; machine code is. One assembly instruction, e.g. ADDL, is translated to several different machine code instructions depending on the destination, source, and addressing mode.
- amw-zero 10y agoCan you point to a source for this? All x86 assemblers that I know of map one assembly instruction to one machine instruction.
- anarazel 10y agoI suspect the instructions they are thinking about are cpu-level uops.
- spc476 10y agoI'm looking at the Microsoft Macro Assembler 5.1 Reference manual (it was nearby and easily accessible to me; yes it's old (from the very late 80s or early 90s) but it covers the 32 bit 80386, which is still valid. Anyway, it shows three different encodings for the ADD instruction. The first: 000000dw mod,reg,r/m This adds register to register, or memory to register (either direction, the d above) using either 8 or 16/32 bits (the w above [1]). The second form: 100000sw mod,000,r/m This adds an immediate value (8 or 16 bits, w again) to a register. The s bit is used to sign extend the data (s=1; otherwise, 0-extend it) if required [2]. The final form: 0000010w data This adds an immediate value to the accumulator register (EAX, AX, AL) [1]. That's three different encodings for the "same" instruction. The MOV instruction (and again, I'm only talking about the 80386 here) has 8 different encodings, depending upon registers used. [1] If the current code segment is designated as a 16-bit segment, then the w means 16 bits, unless a size override byte (an opcode prefix byte) is present, in which case it means 32-bits. If the current code segment is designated as a 32-bit segment, then the w means 32 bits, again unless a size override byte is present, in which case it means 16-bits. [2] It seems to me that if w=1, then the s bit is extraneous and thus could be used to encode other instructions. I'm not sure if that is the case but it's common to use otherwise nonsensical instruction encoding to do something useful.
- Grom_PE 10y agoAs AT&T syntax is still being used, at this point I'm willing to believe it's to purposely make x86 assembly hard and unpleasant to read and write. Perhaps so people will want to stay away from it, and in a way, to reduce the amount of code that is tied to the x86 platform. It spreads the thinking x86 assembly is terrible and ugly. Intel syntax is much cleaner, in particular, Intel Ideal (as opposed to MASM), and specifically, FASM (flat assembler). FASM makes it as clean as possible and turns writing assembly into a joy. Compare: movl %fs:-10(%ebp), %eax ; AT&T mov eax, dword ptr fs:[ebp-10] ; MASM mov eax, [fs:ebp-10] ; FASM
- spc476 10y agoAs I recall, the "dword ptr" stuff was only necessary if the instruction was otherwise ambiguous. Using EAX means you are using a 32-bit destination. But something like: move fs:[ebp-10],5 is ambiguous. Is that an 8-bit constant? 16 bits? 32-bits?
- Grom_PE 10y agoI believe MASM-style assemblers require "dword ptr" at all times and many disassemblers keep outputting that in unambiguous situations. Your example won't assemble because size can't be guessed, but this will: mov dword [fs:ebp-10], 5
- burfog 10y agoAll of those are misleading because the FS segment override isn't specific to an operand. It applies to the whole instruction, which commonly has one place (a memory reference) for the override to take effect. You can have more than one override, but only the last one remains active. Normally you can have an override even if it isn't used. There are a few instructions with more than one memory access; the override only applies to one access and you don't get to choose which one.
- Grom_PE 10y agoNo. This is misleading: fs movs byte [edi], [esi] This is a syntax error: movs byte [fs:edi], [esi] And this is a valid override with sensible syntax: movs byte [edi], [fs:esi]
- userbinator 10y agoWhat a train wreck! It’s hard to imagine a more confusing state of affairs. I'd say that's more attributed to someone many many years ago deciding they would not follow the official Intel syntax (for what reason I do not know), and somehow convincing the rest of the community to follow them. That's actually one of the things that could make for a very interesting article: how one processor family got two different and incompatible Asm syntaxes. The fact that the mnemonics and syntax don't correspond to those found in the manufacturer's datasheets and manuals just increases the barrier to understanding. As far as I know, the same didn't happen to ARM, MIPS, SPARC, and the others. Especially when the sense of the comparisons/conditional jumps is reversed, and some of the more advanced addressing modes look less-than-obvious, it's hard to imagine why anyone would adopt such a syntax: http://x86asm.net/articles/what-i-dislike-about-gas/ http://x86asm.net/articles/what-i-dislike-about-gas/ Note that the GNU tools have option to use Intel syntax too, so you can avoid some of the confusion (in the DOS/Windows and embedded world at least until recently, Intel syntax is overwhelmingly the norm.)
- pcwalton 10y agoAnd Golang uses a separate syntax from Intel syntax and AT&T syntax, so now there are three incompatible syntaxes in common use. What a mess. :( In Go's case it's because they wanted to have the syntax reflect the MachineInstr-like abstraction they have in place. I wouldn't be surprised if something similar was responsible for the AT&T syntax as well. (In case it isn't clear, my preference is to do what LLVM does and to treat MachineInstr and machine code as distinct objects, and to use the vendor syntax for the human readable representation of the latter.)
- matchagaucho 10y agoWhen I worked at Intel in the Server BIOS group, the development process involved several iterations of developing macro abstractions until the x86 ASM code became more readable and maintainable. No one particularly enjoyed working in raw ASM 100% of the time.
- tptacek 10y agoSo, this is fantastic, but I want to make an appeal for the most important thing to understand about any assembly language, even before you work out the individual instructions: Break your programs into basic blocks! Reverse engineers never read assembly in a straight line. Instead, they read the control flow graphs of subroutines, which is the graph where nodes are runs of instructions ending in jumps and branches, and the edges are the jump targets. I hope this doesn't sound complicated, because it isn't: it's literally just what I wrote in this paragraph. It takes about 15 minutes for most platforms to learn enough to recover CFGs from subroutines by hand. To get a decent understanding of what a chunk of assembly code is doing, all you really need is: * The code broken into subroutines (this is usually your starting point) and then CFGs (good disassemblers do this for you, but it's easy to do by hand as a first pass) * The CALLs (CALLs don't end basic blocks!) * The platform's calling convention (how are arguments passed and return values returned from subroutines) There are two tricks to reading large amounts of assembly: 1. Most of the code does not matter, and you won't be much better off for painstakingly grokking it. 2. Virtually all the assembly you'll see is produced by compilers, and compilers spit out assembly in patterns. Like the dude in The Matrix, after an hour or so of reading the CFGs of programs from a particular compiler, you'll stop having to read all the instructions and start seeing the "ifs" and "whiles" and variable assignments.
- kitsuac 10y agoIf you want to learn how to read X86, don't choose this AT&T style syntax as it's a monstrosity.
- bogomipz 10y agoI thought this sentence was curious: "To write code that runs directly on your microprocessor you need to know how memory segmentation works" Although you can't completely ignore segments, in practice at least on Linux the only segments in use are user code/data and kernel code/data segments. Does anyone know why the author might suggest that understanding segmentation is necessary to write Assembly code?
- ktRolster 10y agoProbably because you need to deal with the MMU. You can't just write raw assembly and expect it to work (ignoring the MMU), but the kernel takes care of that for you.
- bogomipz 10y agoI am not following - what is raw assembly? I can write a complete userland program using only Assembly and it will run just fine. When do I need to deal with the MMU exactly?
- wolfgke 10y ago> When do I need to deal with the MMU exactly? When writing programs for user mode (ring 3 on x86), you hardly need to care (except sometimes use some segment override prefixes (cf. https://news.ycombinator.com/item?id=13052076 https://news.ycombinator.com/item?id=13052076), which pedantically is "dealing with the MMU", since because of the MMU this works; but in my opinion it is not necessary to understand the technical details behind it, why this works). On the other hand, if you are an OS ("operating system", here I don't mean "open source") developer, you probably better know the details of the MMU. Concerning https://news.ycombinator.com/item?id=13052892 https://news.ycombinator.com/item?id=13052892: I also consider the author's statement as misleading that one has to know how segmentation works. The knowledge of about segmentation is absolutely necessary for x86-16 (real mode), which many people tend to associate with assembly (because there seem to be many more assembly tutorials available for DOS/x86-16 than for x86-32 or even x86-64), but hardly relevent for people who just write user mode code.
- ndesaulniers 10y agoNice illustrations. It was a good idea to get the prologue/epilogue out of the way. Note that for optimized code, it's usually unnecessary. For more info: https://nickdesaulniers.github.io/blog/2014/04/18/lets-write-some-x86-64/ https://nickdesaulniers.github.io/blog/2014/04/18/lets-write...
- Vera527 10y agoWhat a great read. Thanks for sharing op.
- Annatar 10y agoAnd, of course, modern compilers will usually produce faster, more optimized code than you ever could, without making any mistakes. This assertion comes up over and over again in the last 30 years. Every time I've had it asserted to me, it always came from non-assembler programmers, who always wrote in a high level language. I have yet to see evidence of optimizing compilers generating code even remotely close in efficiency to what we would code directly in assembler. A coder would never write all those extra frame pointer setup code, nor would they waste encoding and tact cycles shuffling values from one register to another. For example, a human might write the code from the article thus: Add42: addb $42, %al ret and that's it. No frame pointer or stack setup, that's all unnecessary overhead because compiler algorithms can't reliably make such contextual decisions.
- aap_ 10y agoWhy couldn't they though? Doesn't sound very hard to only generate the wordy prologues and epilogues when necessary (i.e. when you have to save any registers). Why they apparently don't do this is another question then.
- Annatar 10y agoI'm not aware of a general algorithm which is capable of deciding whether and how many processor's registers to use instead of setting up frame and stack pointers, and pushing a variable number of arguments on the stack, are you? A human will know while coding in assembler, at any given time, how many variables are in the game; and will almost always manage to fit them all within processor's registers; There were only two times in my life where I actually had more than eight variables within a subroutine and had to use the stack, and even then, I didn't push everything, but only as many registers as I was actually coming up short, and the rest I still stuffed in the available registers. The other time, I figured out a more efficient algorithm where I could fit everything within the seven general purpose address registers (a0 - a6, since a7 is the stack pointer). A human will also know whether the expected result is within a byte, word, longword, or quadword range, and will only use those instruction and register sizes; a compiler has no chance to figure that out. It's trivial for humans, but as far as I'm aware, impossible for a general compiler algorithm. In fact, even the best optimizing compilers are so dumb, that one is not allowed to mix and match 8-, 16-, 32- or 64-bit code; one must either compile everything 32- or 64-bit (the linker won't let one link 32- and 64-bit object code together). A human could easily write correct assembler code using all of those instruction / register sizes at once, and we often do. I have yet to see a compiler capable of inferring that. If you know of one, please show me the generated code. I'd love to use such a compiler.
- RX14 10y agoIf you're experimenting with asm in crystal, you might want to use --prelude=empty to remove the standard library to make the asm output cleaner. You can then then require lib_c and use that directly.
- pat_shaughnessy 10y agoI tried that while researching the article, but found the call to "puts" doesn't link without the standard library code. And without a "puts" or similar call to produce output LLVM optimized the entire program away :) I suppose this could work if, as you suggest, I manually called out to a lib_c function like printf instead.
- RX14 10y agoYeah, puts is part of the part of the standard library, and uses crystal's evented io framework, fiber scheduler and libevent. This is what most of the extra code in the asm output will be doing.
- kruhft 10y agoReading assembly language is about having the computer in your head, just like regular programming. You read and execute the instructions just like any other language, just the operations are that much smaller and less abstract. Each instruction is a 'function call'; just like any low level language you leverage abstraction to build up these operations into greater pieces of execution knowledge using macros and functions in logical ways to get the outcome. It's not magic. The best way to learn assembly is to program in it. I learned on the Gameboy by getting a job and programming 2 games in it. Fun as hell, especially when the machine is small enough to really fit in your head and clock cycles count at 4Mhz.
- dman 10y agoBtw if someone has a local copy of intels intrinsics tool before they converted it into a web version - could you please share it?