7 ms·
Reverse Engineering for Everyone
- andrewmcwatters 2y agoHmm, yeah I don't know. This reads like a lot of fluff or immediately unimportant stuff. Reverse engineering in the real world takes a few forms, some of which the write takes on too briefly towards the end of the material. Applied reverse engineer is usually modifying an existing piece of software so: * .dll/lib injection * signature scanning and patching * packet interception and rewriting * mitm HTTP(s) calls These are just a few places where you see reverse engineering used, usually to modify existing software. I'm curious if there's any reading out there that covers this stuff from the meat and potatoes and less of this CS 101 stuff. I've done all of the above, and you can usually learn about this stuff from some different forums on the web, but I don't know of any good bibles on the subject matter.
- darby_nine 2y ago> I'm curious if there's any reading out there that covers this stuff from the meat and potatoes In my experience using radare2 to peek at the code is pretty much the meat and potatoes of reverse engineering binaries and far from "CS 101 stuff". You certainly don't need to modify a binary to MITM an API or inspect/alter packets or inject code via dynamic loading; nor is it the most convenient or clean or easy to maintain way to do so. Secondly, this is a shockingly dismissive attitude for such a large resource. It took me a few minutes to just read through the table of contents.
- andrewmcwatters 2y agoJust because it's large doesn't mean it's relevant: using radare2, IDA Pro, or some other tool doesn't mean you're going to be able to do anything besides look at a binary. I mean, you said you read the table of contents, yeah? Doing the same thing across different CPU architectures isn't doing something at length, it's just doing the same thing over and over again in rhymes. In practice, yeah, people in the wild are absolutely modifying binaries, injecting, stubbing .dlls and redirecting calls, or creating proxy servers that alter payloads, for sure. Learning how to compile a program isn't exactly reverse engineering worthy content to write about.
- darby_nine 2y ago> Just because it's large doesn't mean it's relevant: using radare2, IDA Pro, or some other tool doesn't mean you're going to be able to do anything besides look at a binary. Looking at a binary is like 99% of the work, though. Or at least looking at some secondary form of it (e.g. assembly, decompilation, etc). Tools are absolutely critical to the work. > people in the wild are absolutely modifying binaries, injecting, stubbing .dlls and redirecting calls, or creating proxy servers that alter payloads, for sure I would call modifying a binary "cracking" it but it's been a few decades since I was involved in that scene. I also think that the topic is large enough to warrant multiple focuses—to me, at least, writing a MITM server is much more trivial than extracting a private key from a binary (or a running process) that makes that MITM server functionally useful. > Learning how to compile a program isn't exactly reverse engineering worthy content to write about. That's a disingenuous characterization of most of the content here. Coding at the instruction level requires a different way of reading and writing code than you're otherwise exposed to. Most programmers aren't used to handling bits directly, and certainly not to the extent that it rewards you at the instruction level for learning and knowing. With the tools here you can, in fact, sit down and inspect the license verification function of a piece of software (although I'm not sure how much that's true or beneficial these days with code-signing etc). EDIT: Or you could do what I did and work with as, `otool`, and a hex editor, and learn extremely slowly & painfully why custom-built reverse engineering tools are so valuable to learn. There's always more to learn, of course, but that's no reason to belittle what you've already learned and other people still have yet to learn.
- andrewmcwatters 2y agoYeah, I'm sure what I'm saying probably comes off as belittling, but that's not my intent. It's just more productive to understand who the audience is. The author write "free PDF" content with Guy Fawkes mask header images in the README.mds. If you're going to target script kiddies, at least show them how to Hello, World! from a DLL_PROCESS_ATTACH, and then teach them sigscanning.
- acureau 2y agoI disagree, learning how to compile a program is a prime example of something you'd want in a book about reverse engineering "for everyone". A book which focuses only on specific methods of changing software behavior would be useful only to those who know how to understand said software. In fact the term "reverse engineering" itself does not imply modification at all.
- jonpalmisc 2y agoResources exist, but are only so helpful IMO. One can't necessarily build an airplane after watching a documentary on it. Even if there was some "bible" on it, reverse engineering is one of those things that you have to put the reps in for to get good at it and actually develop understanding. The "bible" is tackling reverse-engineering related projects independently over the course of months/years and picking up knowledge along the way. Starting with something like cracking software (and making increasingly-advanced cracks) is always my advice for beginners.
- boricj 2y agoApplied reverse-engineering is all about bending the rules of engineering. Because of this, I think it can be learned through experience, but I doubt it can be taught through theory (or at least not in an effective manner). At its core, it's about spotting metapatterns to gain an understanding of a program and applying leverage to affect it. That's more art than science, no matter how much tooling you throw at it. Honestly, I think the most effective way to learn about how to reverse-engineer something is to learn engineering at the same layer first and then start tinkering. If you want to binary patch a program, learn assembly. If you want to inject a .dll, learn how to write and use dynamic libraries. If you want to MITM a REST API, learn how to call a REST API. Because once you know the rules well, you can start breaking them and see exactly how much you can get away with. I wrote a series of articles on reverse-engineering on my blog, about studying and modifying a program that outputs an ASCII table, mostly because I needed a way to introduce delinking as a technique. I would not say it's good, but it starts with how to build the case study and then it handholds the reader through the meat and potatoes.
- palata 2y ago> I wrote a series of articles on reverse-engineering on my blog, about studying and modifying a program that outputs an ASCII table, Would you mind sharing the links? I would be interested!
- boricj 2y agoYou can find the table of contents for the series there: https://boricj.net/reverse-engineering/2023/05/01/introduction.html https://boricj.net/reverse-engineering/2023/05/01/introducti... I expect that you'll be mostly interested in parts 2 through 6. Part 1 explains how a toolchain works in general (so mostly CS 101 stuff as the OP put it). Parts 7 to 10 demonstrates the delinking technique by easing into it, a technique which is as powerful as it is esoteric, but probably not what you're looking for in a beginner's guide.
- PennRobotics 2y agoThis. There's a lot to be said about understanding registers and assembly and different languages and how a USB packet is constructed, but efficiency in reverse engineering comes down to effective pattern recognition. A binary is likely to have a reasonable amount of often-called code for memory operations (memset, memcpy, strcat, strlen, sscanf, log) and a lot of library code (Flexcomm_Init, Clock_AttachClk, SPI1_Handler, NVIC_EnableIRQ) and then probably fairly little actual application code. For Ghidra users, being able to ignore the boilerplate (mem and BSP code) and quickly find and analyze the application code saves a TON of time. (Conversely, if I know a binary is written using FreeRTOS, finding the task creation function would be my first step, as this reveals nearly all of the application code.) There are techniques to help (setting a flash memory region as non-write so string references are recognized and disassembled correctly, loading a chip SVD so all the library code is more obvious) but those come with experience or a good hands-on tutorial, and they still won't tell you everything about the application code. In my own breakdown of one Cortex-M binary (bare metal, no objects known) the only reason I was able to get the firmware in the first place was by noticing and decoding a base64 string in an unpacked Electron app used for USB communication with the device. This ended up holding plaintext credentials for their update server which had two channels: one for encrypted production binaries and the other for unencrypted development binaries. In this specific case, it helped to know what base64 looks like, but that's like how knowing different methods of slicing onions might help you figure out a recipe by tasting a cooked meal. Very often such background knowledge is irrelevant. Once in a while it will be the only realistic way forward.
- xkcd-sucks 2y ago> places where you see reverse engineering used, usually to modify existing software. funnily enough I have a team reverse engineering binary data formats, which is often more easily accomplished by other means + only dropping down to the disassembly/decompilation where absolutely necessary. and which as far as I am aware never involves binary patching but yeah about the article it seems like if you know this much about assembly / chips etc. to be able to read it, then general problem solving ability should be able to cover most of the article's content
- deleted 2y ago[deleted]
- bitwrangler 2y agoIt seems like a high-level overview, good for somebody new to the topic. It also linked to this resource, which was more in depth... https://github.com/mytechnotalent/Reverse-Engineering https://github.com/mytechnotalent/Reverse-Engineering EDIT: Whoops... it looks like it mostly links back to the original article.
- amiga386 2y agoI'd like to add that reverse engineering can also be done without any peeking at the thing you're trying to reverse-engineer. Andrew Tridgell explaining how he reverse engineered Microsoft's SMB protocol with the "French cafe technique": https://www.samba.org/ftp/tridge/misc/french_cafe.txt https://www.samba.org/ftp/tridge/misc/french_cafe.txt Tridge also reverse engineered BitKeeper, the proprietary software that Linus foolishly used to host Linux kernel development for a while. He noticed that if you telnet to the BitKeeper address:port rather than use its proprietary client, you can type "help" and it then spits out a list of commands to try... You can then interrogate the repository with these commands and get a complete understanding of all the internal data structures, without ever using the proprietary software, let alone having to disassemble it. The fact that Tridge did this reverse-engineering led BitKeeper's owner, Larry McVoy, to rescind the Linux community's use of his software, so Linus wrote git.
- actionfromafar 2y agoSo Andrew saved us not only once, but twice! It goes to show that yes, you need someone to score, but you also need someone to make that critical pass of the ball.
- tsujamin 2y agoAn undergrad highlight for me was hearing the bitkeeper/git story from Tridge one afternoon that he happened to be in the faculty lunch room :)
- brcmthrowaway 2y agoI bet Larry Mcavoy highly regrets his actions.
- executesorder66 2y agoYes, he is still butthurt about it. This is from 20 days ago : https://news.ycombinator.com/item?id=40887244 https://news.ycombinator.com/item?id=40887244
- 2y ago
- llmblockchain 2y agoI never thought of reversing as something you pick up a book for. Everything I learned was through application from a young age. 1. Learning how to use Cheat Engine to scan video game process memory and modify games. 2. Learned how to read/replay packets in an MMO to try an cheat. 3. Learned how to craft DLLs, hooks and inject them in processes. 4. Learned how create patches for executables to solve some crackme challenges. 5. Mess with real world software that requires a license key, to suddenly not require a license key (or accept any key). 6. Mess with binary formats to try an reverse how game saves worked to.. you guessed it, cheat. 7. Get a real job and make money with the skills and knowledge I acquired.
- solidalbo12 2y agoLmao we must be related. U explained my childhood
- boricj 2y agoGoing straight for reverse-engineering is doable, but it's significantly harder without some engineering background, either formal or self-taught. I have an ongoing reverse-engineering project for a video game and I ended up getting in contact with a self-taught modder of the game, who doesn't know how to program. He learned more in a couple of evening Discord calls with me showing him around the reverse-engineered Ghidra project, explaining the basics of computer program engineering as we went, than he did flipping bits with Cheat Engine. He then proceeded to recreate a fairly ambitious mod that was showcased in a Youtube video 15 years ago but never released, something that was bugging him for years but was unable to recreate. I steered him throughout, but by seeing how the pieces fit together he then managed to do the same mod on the sequel (which was never done before) all by himself. Experience with engineering gives you perspective when reverse-engineering.
- deleted 2y ago[deleted]
- chc4 2y agoI don't think this is true, or at least I'm not convinced by a single anecdote. The majority of good reverse engineers I know picked up reverse engineering first and programming second (and a lot of them are still frankly not great programmers), and likewise I know plenty of good programmers who would be completely lost reverse engineering. Reverse engineering is a very different skillset than programming.
- tonetegeatinst 2y agoBook links are not working for me, did you get the HN hug of death?
- armitron 2y agoI found so many mistakes, badly understood concepts and entirely wrong explanations just reading for 5 minutes, that I can't possibly recommend this. It's obviously written by amateurs / people with minimal experience of the domain. Much better resources are Eldad Eilam's "Secrets of Reverse Engineering", for Windows "Practical Reverse Engineering" and for the absolute basics, Patterson's "Computer Organization and Design".
- optymizer 2y agoSecrets of Reverse Engineering is from 2005. Is there a more recent book you would recommend?
- deleted 2y ago[deleted]
- imhoguy 2y agoReverse engineering Java is cool as well, especially fishy Android apps which control some appliances via byzantine Bluetooth protocols.
- raggles 2y agoIf only there was an equivalent to DnSpyEx for Java. Can't wait for Recaf4 to be ready.
- svag 2y agoThis brought back memories when I was reading reversing tutorials from searchlores.org and fravia.com... It's in web archive now, https://web.archive.org/web/20191201105759/http://search.lores.eu/ https://web.archive.org/web/20191201105759/http://search.lor...
- parsonch 2y agoNeat, I know the guy that wrote this guide! Glad to see it made it onto HN, if you have any specific feedback I can pass it along.
- fercircularbuf 2y agoThis is way too short on engagement and visuals. Way too much telling and walls of text. For these reasons alone this is not for "everyone".
- gouggoug 2y agoProps to the author for writing this – that being said, I felt the same way. Very long, windy and hard to parse sentences. For example, Part 2 > There are two basic techniques that you can employ when analyzing malware. The first being static analysis and the other being dynamic analysis. > Static analysis uses software tools to examine the executable without running the actual decompiled instructions in Assembly. We will not focus on this type of analysis here as we are going to focus on actual disassembled binaries instead however in future courses we will. > Dynamic analysis uses disassemblers and debuggers to analyze malware binaries while actually running them. The most popular tool in the market today is called IDA which is a multi-platform, multi-processor disassembler and debugger. There are other disassembler/debugger tools as well on the market today such as Hopper Disassembler, OllyDbg and many more. > A disassembler will convert an executable binary written in Assembly, C, C++, etc into Assembly Language instructions that you can debug and manipulate. > Reverse engineering is much more than just malware analysis. At the end of our series, our capstone tutorial will utilize IDA as we will create a real-world scenario where you will be tasked by the CEO of ABC Biochemicals to secretly try to ethically hack his companies software that controls a bullet-proof door in a very sensitive Bio-Chemical lab in order to test how well the software works against real threats. The project will be very basic however it will ultimately showcase the power of Assembly Language and how one can use it to reverse engineer and ultimately provide solutions on how to better design the code to make it safer. > In our next lesson we will discuss various types of malware. could be written: > There are two basic techniques that you can employ when analyzing malware: static analysis and dynamic analysis. > Static analysis examines the executable without running it. We will not focus on this type of analysis here, however in future courses we will. > Dynamic analysis uses disassemblers and debuggers to analyze malware binaries while running them. > A disassembler converts an executable binary into Assembly Language instructions that you can debug and manipulate. There are many disassembler/debugger tools available such as Hopper, OllyDbg, IDA and many more. The most popular being IDA, a multi-platform, multi-processor disassembler and debugger. > Reverse engineering is much more than just malware analysis. > At the end of our series, we will use IDA in a fictional scenario where you will be tasked by the CEO of ABC Biochemicals – a very sensitive Bio-Chemical lab – to ethically hack his company’s bullet-proof door control-system. > The project, while basic, will showcase the power of Assembly Language and how one can use it to reverse engineer black-box binaries and ultimately find solutions to make the code safer. > In our next lesson we will discuss various types of malware.
- sva_ 2y ago> The x64 Architecture Ahh yes, the famous 8064 computer. Just kidding, looks like a great work.