27 ms·

Ghidra, NSA's reverse-engineering tool

1283 points by twodayslate 8y ago

kevinchen 8y ago
This is an unusually large open source project, especially for NSA. I wonder whether they were motivated to release this tool because of their recent brain drain / hiring problems.
- maerF0x0 8y ago
  Here's a potential angle: If you're going to use a tool internally it's in your best interest to be able hire people w/ experience with that tool. (ie, people learn the tool for free on their own time)
openloop 8y ago
China? Russia? Etc? Really I think this is a good thing.
mrmuagi 8y ago
I'm definitely excited for this, considering I couldn't fork out the thousand of dollars needed for using IDA. I can't really justify that on a small hobby project (reverse engineering games).
- snazz 8y ago
  The free version (https://www.hex-rays.com/products/ida/support/download_freeware.shtml https://www.hex-rays.com/products/ida/support/download_freew...) would likely meet your needs, assuming the games are for x86.
  - convery 8y ago
    The freeware version supports x64 as well.
- kabwj 8y ago
  If it’s a hobby project why don’t you just pirate it? Honest question.
  - tptacek 8y ago
    Somebody sort of casually pirated a copy of IDA Pro back in the mid-2000s (IIRC, he shared his copy on a public server). The IDA people (DataRescue, at the time, but from what I recall the page survived the move to Hex-Rays) found out, banned him from using IDA, and then put a page on their website threatening to rescind IDA licenses from any company that employed him. The IDA team is pretty aggro.
    - jchw 8y ago
      To be fair, there's definitely a difference between downloading a pirated version of software, and actually leaking copies of software for others to pirate. The DataRescue and now Hex-Rays folks seem particularly sensitive to leaked copies and I imagine leaked copies genuinely do affect their bottom line of sales given the kinds of markets they're in.
      - philpem 8y ago
        They certainly seemed to be very much against selling it to private individuals (or were when I asked many years ago). I guess unless you've got a CV which says "presented at Defcon and Blackhat, five times" or "currently work at {big infosec company}", even if you can afford it the answer will be "nope". The end result for me was that I bought a Mac Mini and a copy of Hopper and Synalyze It. My entire reverse-engineering of the Polaroid film recorder driver (and the resulting Linux port) was done by reversing the driver DLL in Hopper and shimming the driver and ASPI calls with PyDbg. I keep looking away for a month or so and finding a new version of Hopper with shiny new features to play with...
        sdwisely 8y ago
        I really really liked hopper. it's probably the one app I missed most switching to windows. x64dbg eventually grew on me and I like it nearly as much.
poojapanwar27 8y ago
sdfsdf
twodayslate 8y ago
Download: https://ghidra-sre.org/ https://ghidra-sre.org/ GitHub: https://github.com/NationalSecurityAgency/ghidra https://github.com/NationalSecurityAgency/ghidra
- WrtCdEvrydy 8y ago
  Download from the NSA without open source software... anyone else virtualizing three layers deep to get to this?
  - aboutruby 8y ago
    I don't think they would burn some 0-days for this, one container should be enough.
    - omeid2 8y ago
      I think you're underestimating the value of intel on people who know how to get intel or at least interested in the topic.
      - est31 8y ago
        Fortunately, a lot of people who are very knowledgeable about reverse engineering are downloading & opening it as we speak. They will point out any flaws/viruses found. Seems one has already been found [1]. All you need to do is to wait. [1]: https://news.ycombinator.com/item?id=19315515 https://news.ycombinator.com/item?id=19315515
      - strictnein 8y ago
        It's part of the NSA's recruiting push. "If you are interested in projects like this ... consider applying" is even mentioned in the README. There's zero chance there's some secret trojan, because the people who are interested in this type of software are the exact people who would be able to find it.
        8y ago
Foxboron 8y ago
Also found this community edition linked on twitter: https://ghidrace.github.io/ https://ghidrace.github.io/
- deleted 8y ago
  [deleted]
- _wmd 8y ago
  This link is third party junk, don't touch it. A "fork" existing before the original software release, yeah right
- WrtCdEvrydy 8y ago
  That's coming from someone else (https://github.com/withzombies?tab=following https://github.com/withzombies?tab=following)
- duskwuff 8y ago
  There's literally nothing there. The project doesn't even exist yet.
purpleidea 8y ago
Risky click of the day ;)
maxfan8 8y ago
It appears that it isn't actually available as of now. Apparently, the NSA is going to release it at RSA Conference 2019, so it'll probably actually be published within the next couple of days.
- arthurcolle 8y ago
  It is available here: https://ghidra-sre.org/ https://ghidra-sre.org/
  - rock_artist 8y ago
    maybe the NSA don't like non-US citizens using open-sourced software :) this is what I get - 403 ERROR The request could not be satisfied. Request blocked. Generated by cloudfront (CloudFront) Request ID: i_Sa-b1Fj2C4ZAUtBsmAp-7mVg9LerDTlD9t1_00ex4fQyCUhrYvdw==
    - arthurcolle 8y ago
      I have the downloaded files if you want to reach me at my email address in my profile, I can upload them to mega or something
- twodayslate 8y ago
  Try clearing your cache.
- pacificmint 8y ago
  The RSA conference is currently ongoing and the NSA talk was today. Actually, it should be over just about now.
  - pkaye 8y ago
    Is the NSA talk about this available online?
    - 8y ago
nyrulez 8y ago
I am going to sound pessimistic here, but isn't there a real danger of having this technology available to bad actors and is there any value to keeping such things confidential if it plays a role in national security? If someone was releasing malicious software to hijack the power grid as an example, wouldn't they be first able to use this to try to improve the robustness and invisibility of their attack ? Or is the functionality here common place enough that it doesn't tilt the axis of power in an unfavorable way?
- mparlane 8y ago
  Bad actors have been using IDA this entire time. So no, not really.
  - nyrulez 8y ago
    You mean there is nothing new here? Then why is this news? I am not wondering about the concept of reverse engineering but the specific (and hopefully novel) feature set that this may enable.
    - saagarjha 8y ago
      It's a competitor to IDA's monopoly, basically. It might be better in certain aspects.
      - tptacek 8y ago
        IDA has a bunch of competitors --- Hopper, Binja, and all the Capstone interfaces.
        saagarjha 8y ago
        Yup, well aware of them (I think Hopper uses Capstone, FWIW). I'm sure you agree that they're not quite at the level of IDA, though ;)
baby 8y ago
Annndddd, it's ugly as fuck. Well, I'm not going to install something made by the NSA on my machine but I'd be interested in feedback.
- tptacek 8y ago
  It's not as if IDA is a work of art. Arguably, this is a cleaner, more usable interface.
  - baby 8y ago
    Totally, that's why I was hopping for something better looking. Looks like Binary Ninja is the only one that cares about that.
    - saagarjha 8y ago
      And Hopper!
- Arbalest 8y ago
  So no SELinux for you?
sscarduzio 8y ago
Does it work for non windows binaries?
- saagarjha 8y ago
  Seems to work on a Mach-O and ELF files as well; at least, it loaded those two just fine.
- jukeboxbandit 8y ago
  >includes a suite of software analysis tools for analyzing compiled code on a variety of platforms including Windows, Mac OS, and Linux From the site, so yes it works on non-windows binaries. It also runs on Linux, Mac and Windows. This is the list of file formats I found in the docs that are supported by Ghidra * Common Object File Format (COFF) * Debug Symbols (DBG) * Executable and Linking Format (ELF) * Ghidra Data Type Archive Format * GZF Input Format * Intel Hex * Mac OS X Mach-O * Module Definition (DEF) * Motorola Hex * New Executable (NE) * Old-style DOS Executable (MZ) * Portable Executable (PE) * Preferred Executable Format (PEF) * Program Mapfile (MAP) * Raw Binary * XML Input Format
slimsag 8y ago
Why this is important (for those uninitiated): - Ghidra is basically the first real competitor to IDA Pro, the extremely expensive and often pirated state-of-the-art software for reverse engineering. Nothing else has come close to IDA Pro. - Ghidra is open-source, IDA Pro is not. - Ghidra has a lot of really cool features that IDA Pro doesn't, such as decompiling binaries to pseudo-C code. - It's also collaborative, which is interesting because multiple people can reverse engineer the same binary at the same time -- something IDA only got VERY recently.
- saagarjha 8y ago
  > Ghidra has a lot of really cool features that IDA Pro doesn't, such as decompiling binaries to pseudo-C code. To be fair, IDA Pro has a decompiler plugin to do this.
  - slimsag 8y ago
    It does, but it costs an extra $1.5k on top of the main software, per architecture.
    - Avery3R 8y ago
      $2.6k+ you mean
  - _wmd 8y ago
    For which they charge a per-CPU fortune https://www.hex-rays.com/cgi-bin/quote.cgi https://www.hex-rays.com/cgi-bin/quote.cgi
    - tptacek 8y ago
      It's a funny situation, though: decompilation probably should cost a small fortune. If you're in a line of work that needs it, the quality of your decompiler is probably a huge factor in how valuable an hour of your time is, and many [most?] fields where people routinely decompile stuff are very highly compensated. IDA has always had a weirdly low price point given the bill rates of people who use it, and it's interesting to see that price being competed all the way down to free.
stargazing 8y ago
Aaaaaand: https://twitter.com/hackerfantastic/status/1103087869063704576 https://twitter.com/hackerfantastic/status/11030878690637045...
- meowface 8y ago
  Though the obvious explanation for that is that it was an intentional backdoor, that honestly looks more to me like a legitimate oversight than a backdoor. I think an actual backdoor would be a lot more subtle and clever than that. Especially since this way, absolutely anyone could exploit it (it's just Java Debug Wire Protocol). Also, you have to explicitly run it in debug mode for this to happen, which probably only a small percentage of end users will do. Kind of seems like the equivalent of running Flask apps in debug mode, which by default will handle exceptions by showing a traceback with an interactive debugger that can be used to execute arbitrary code. There could be some backdoors in it, but I'm leaning towards that not being an intentional one. (But I definitely could be totally wrong; you never know when it comes to intelligence agencies.)
  - earenndil 8y ago
    I also don't think it's a backdoor, but the best way to hide a backdoor is to make it look like a mistake.
    - meowface 8y ago
      >but the best way to hide a backdoor is to make it look like a mistake It is, but usually the best way to do is to make it look like a mistake that's very subtle and difficult to notice without careful testing and analysis, kind of like Apple's infamous SSL "goto fail". That's a classic example of a vulnerability that really could be either an honest mistake or a very insidious backdoor. This is more like leaving the house's sliding glass door to the backyard wide open for everyone to see.
    - hiccuphippo 8y ago
      Then again, maybe the best way to hide a backdoor is to have another very obvious one so people look the other way.
souprock 8y ago
It's not the first real competitor available to the public. Hopper Disassembler and Binary Ninja are both capable. They have been available for a few years. Binary Ninja is also collaborative if you get the enterprise edition: https://binary.ninja/purchase/ https://binary.ninja/purchase/
- comex 8y ago
  They're arguably competitors if you don't care about decompilation. But Binary Ninja has no decompiler and Hopper's was awful last I checked. Ghidra's decompiler seems as competent as Hex-Rays.
  - souprock 8y ago
    Binary Ninja has most of a decompiler and is expected to get the rest soon. Binary Ninja offers multiple views of the code, each with an API that gives you the same access that the GUI has. The different views vary in how much they are like assembly or C. Only that last step, real C code, is still missing. Those other views are quite good if your goal is to understand things, but less good if you were hoping to throw the results into a C compiler.
    - comex 8y ago
      Ah, I hadn't heard of the IL functionality. From a quick test of the Binary Ninja demo, it looks like an approach that could become a viable competitor to a decompiler in the future, but isn't a good one in its current state. For instance, one of the most useful aspects of a decompiler for me is the ability to recover high-level control flow, which Binary Ninja apparently doesn't support. Instead it gives you an IDA-like graph view (but with IL instead of assembly in the graph nodes); but at least in my experience using IDA without a decompiler, even moderately large functions tend to result in a spiderweb of a graph, and recovering the control flow by hand ends up feeling like a pointless brain teaser. (This condition being true short-circuits this set of comparisons, so it must be an ||... but it still ends up doing this other set of comparisons, which you can also get to from... wait, where was I again?) It also doesn't seem to allow eliding temporary assignments. Here's a short sample from some random function (retyped by hand since I don't see a way to copy and paste): int32_t edx = arg4 int32_t eax = esi eax_1, ecx_2 = sub_3c670(eax, edx) It does do a sort of SSA transformation and assign unique variable names (like eax_1 instaed of eax) to the same register based on the location in the program, so that's nice. But what I really want is eax_1, ecx_2 = sub_3c670(esi, arg4) I may be missing some option to do this manually, but it should be automatic.
openloop 8y ago
Holy shit.
andrewcchen 8y ago
There's a typo in launch.sh that breaks when you try launch in debug mode > "{$DEBUG_PORT}"
snazz 8y ago
The intro video is pretty good, if you want to see a screen recording: https://ghidra-sre.org/GhidraGettingStartedVideo/GhidraGettingStartedVideo_player.html?embedIFrameId=embeddedSmartPlayerInstance&theme=dusk https://ghidra-sre.org/GhidraGettingStartedVideo/GhidraGetti...
yifanlu 8y ago
From someone who does binary reverse engineering full time, in my experience, BinaryNinja, Hopper, radare2, etc are toys compared to IDA Pro + Hex Rays Decompiler. The quality of the results and the features supported are unmatched... until now. I haven’t spent too much time with ghidra yet but it’s the real deal. The output of the decompiler looks alright (not complete garbage like I’ve seen with other tools). Even if everything else sucks, the decompiler by itself makes it outrank every other tool aside from IDA. And it costs $10k less! The fact that it’ll be open source is just icing on the cake.
- tptacek 8y ago
  Leaving the decompiler aside, for core disassembly features, in what ways is modern IDA far ahead of its competitors?
  - XMPPwocky 8y ago
    To be fair, without undo, Hex-Rays can only move forwards. This explains their advantage.
  - simcop2387 8y ago
    Not sure about everything, but last i looked IDA had a lot more support for different architectures and file formats compared to most of the open source stuff (not sure about other proprietary ones).
  - yifanlu 8y ago
    Auto analysis when you have barely any information. Any tool can make nice output if you feed it nice input. Try a partial dump from an exotic device and then you’ll see IDA shine.
    - XMPPwocky 8y ago
      Definitely- it's all the years of tweaking and the massive numbers of heuristics to handle, i don't know, code emitted by Microsoft Visual FORTRAN from 1972- that's IDA's moat. Screw the decompiler. If GHIDRA can match that, it's a huge step forwards.
Defcon6 8y ago
https://twitter.com/hackerfantastic/status/1103087869063704576?s=21 https://twitter.com/hackerfantastic/status/11030878690637045...
vasilia 8y ago
Are they serious? They are banning Russian IPs with decompiler source code. Hmm, I know ARM and x86 assembly. Of course, I don't know how to download these sources :)
- empath75 8y ago
  Legal reasons.
  - vasilia 8y ago
    As I see it's licensed under Apache 2.0. I don't know about any regional restrictions for this type of license. But it's could be a real reason because of our stupid government.
keypnchr 8y ago
Started to watch the video, then remembered Snow Crash.
ramon 8y ago
does Ghidra work for APK files, it would be cool if it also worked with mobile projects.
- kdbg 8y ago
  Yes it does work with APKs. Well, it should. It kept having issues with the only APK I had on hand, but when I just pulled some DEX files out and loaded them it handled them just fine (including decompilation)
- tatoalo 8y ago
  Direct APK doesn't work at the moment (I've tried with 3 apks), just unzip it and work with the DEX file.
alexozer 8y ago
I wonder how this compares to retdec, an open source cross-architecture decompiler by Avast. https://github.com/avast-tl/retdec https://github.com/avast-tl/retdec
- snazz 8y ago
  I'm wondering this too. I haven't heard of retdec being used too much, but it looks very cool. I'll guess that Hex-Rays is better, but I still am interested in the opinion of someone more experienced who has tried retdec.
  - DannyBee 8y ago
    I'll go one better: I've contributed patches to retdec. Retdec is ... okay. On small binaries it's usable. On even average sized windows binaries (a few meg), not really. Like on things that IDA takes 10-15 minutes and a reasonable amount of memory (like a 7 meg windows binary), retdec can take forever and unlimited amounts of memory. I started fixing a lot of the memory issues (completely recursive CFG traversal, etc), but there are also very serious algorithmic issues (N^3/N^4 algorithms in the optimizers). If i disable a lot of the backend optimizers, i can make it work okay. But then the output is also a lot larger/worse. To be fair: It used to be about 50x bigger than similar IDA output. The latest development version of retdec now has a new backend IR converter, and the output is only 5x-10x bigger than IDA output. So as a TL;DR: retdec in its default state is unusable for anything but small binaries. If you understand what is going on, you can get it to work on a lot of binaries as long as you have a ton of memory and time to spare.
- tptacek 8y ago
  Retdec is based in part on Capstone, like Hopper.
  - alexozer 8y ago
    I believe Capstone is merely a disassembly framework, and retdec's decompilation process is custom implementation which works on LLVM IR.
ryanmarsh 8y ago
The logo reminds me of 8 chan's logo.
- 5mu3eke56mu 8y ago
  my first thought
- ecommerceguy 8y ago
  indeed.
imjustsaying 8y ago
>clicking the link before reading the domain regret
deleted 8y ago
[deleted]
zelon88 8y ago
I'm curious what feature specifically prompted the NSA to develop their own IDA Pro alternative. I mean, someone somewhere at the NSA must have been trying to do something with IDA Pro only to repeatedly fail before the decision was made that whatever the NSA was trying to do warranted developing their own IDA Pro... right? Or perhaps they used IDA Pro so often and grew so frustrated by it that they started their own?
- mox1 8y ago
  Is it possible that maybe this pre-dates IDA Pro? Ever tried to use IDA Pro on the same project with a co-worker...at the same time?
  - souprock 8y ago
    I believe the included decompiler predates the one that goes with IDA Pro, so that could be a motivation. IDA Pro itself is really really old, having once been a 16-bit DOS shareware program, which explains why IDA Pro doesn't use the GUI shortcuts that were standardized much later. IDA Pro still doesn't support collaboration, although there are very broken hacks that attempt to add it. Binary Ninja supports collaboration if you buy the enterprise edition.
- tptacek 8y ago
  Hex-Rays can be hard to deal with, and the IC deals pretty extensively with large federal contractors like Raytheon, so it's possible they just needed something as capable as IDA that they could roll out across all their suppliers to use as a common toolchain and interchange format. But it's also possible this is just sort of a labor of love type thing.
  - comex 8y ago
    There’s a third possibility: they wanted a piece of software they could customize to meet their needs. Admittedly, for simple bugfixes and the like, Hex-Rays’ support is known for being quite responsive (as suits the small number of customers). There’s also a quite large (albeit poorly-documented and crash-prone) SDK, which can handle a wide variety of needs and has gained functionality over time. But if you want to add new functionality that can’t be implemented using the SDK? (That includes just about anything related to the decompiler.) You’re at the mercy of Ilfak’s priority list. Well, mere mortals are, at least; the NSA has enough money that it might be able to set up some sort of special contract with Hex-Rays, but I suspect that’s easier said than done. From that perspective, the ideal is what the NSA ended up with, a codebase whose development is fully in-house. Notably, though, second best would be to just have access to the source code of an existing tool, so you can at least make your own patches if necessary, even if you’re not in control of the codebase’s overall direction. Did the NSA ever seek that in IDA’s case, and could they have obtained it if they did? I don’t know the answer to either question… but source access certainly isn’t offered to typical customers. In general I’m surprised that “paid + source access for customers” isn’t a more popular model of software development. From my perspective, which admittedly is very different from the NSA’s, I was never very interested in low-cost IDA competitors like Hopper or Binary Ninja, but I’m very excited about Ghidra. Why? Partly because it’s a more full-fledged competitor in terms of feature set, I admit – but the competitors I mentioned are bound to narrow the gap over time. Partly because of cost: I myself am at a point where I could justify the $600/y for Binary Ninja’s commercial edition, or even the order-of-magnitude-higher cost of the Hex-Rays decompilers, without wincing too badly. but I believe that reverse engineering should be accessible to beginners and amateurs. (Piracy is a partial solution, including in IDA’s case, but some people don’t like to do that.). But the main reason I’m excited about Ghidra is that I have the source code. As a concrete example, I’ve spent a good amount of time reverse engineering software for the Nintendo Wii and Wii U. Both consoles have a main CPU based on the PowerPC architecture, but with a custom ISA extension for an extremely barebones version of SIMD. Well, both Hex-Rays and Ghidra support PowerPC decompilation (although that’s a relatively recent development), but unsurprisingly, neither of them have full support for that ISA extension. IDA actually does have built-in support for disassembling it, but AFAIK not for decompiling; Ghidra doesn’t seem to support it at all (but I may just need to configure it properly). What can I do? Well, in practice, nothing, because I don’t care about the Wii U anymore. But if Ghidra had been released a few years ago, I’m pretty sure I would have gone and implemented support for the extension myself; I haven’t looked at Ghidra’s source yet, but since it already supports other vector ISAs, it probably wouldn’t be that hard. With IDA, I was stuck. The SDK supports adding custom instruction sets for disassembly, but the decompiler SDK is so limited that supporting them there is either impossible or at least would be a huge hack. And that’s just one of many customizations I‘ve wanted over the years. Some of them are probably easier said than implemented, but at least now I can put that to the test!
xxpor 8y ago
Why do they have a LICENCE file with the Apache Licence in it? As a work of the federal government this is public domain by definition, isn't it? edit: Oh, outside contributors of course retain their own copyright. That's what's licenced. https://github.com/NationalSecurityAgency/ghidra/blob/master/NOTICE https://github.com/NationalSecurityAgency/ghidra/blob/master...
- paxys 8y ago
  They can also enforce copyright in other countries.
  - pbhjpbhj 8y ago
    Can they? How's that work, surely a resident in USA could take the work to the other country and make it available, if they can't then it wasn't public domain in the first place? Can you point me to any caselaw/ analysis please?
    - MileyCyrax 8y ago
      I don't quite get it either, but one of their documentation files seems to imply that they can: >In countries where copyright protection is available (which does not include the U.S.), contributions made by U.S. Federal Government employees are released under the License. Merged contributions from private contributors are released under the License. https://github.com/NationalSecurityAgency/ghidra/blob/master/INTENT.md https://github.com/NationalSecurityAgency/ghidra/blob/master...
    - webninja 8y ago
      Guns
- vharuck 8y ago
  A common work-around is for a federal agency to hire a contractor to write software and require the copyright be transferred as part of the contract. The federal government can hold copyrights, so it's legally kosher.
m0zg 8y ago
Yeah, I think I'm gonna pass on this strictly binary release and wait until the code is released, reviewed, and independently compiled.
poojapanwar27 8y ago
test
yalogin 8y ago
You are the leader in your segment of the market one day and the undisputed leader. You wake up and the NSA decides to send a free competitor out with better or matching functionality. Tough blow. But good for us.
- Tepix 8y ago
  I'm surprised that noone is yelling "socialism" yet. Tax paid competition for existing commercial products. Isn't that considered evil/wrong by pure capitalists?
  - dang 8y ago
    "Eschew flamebait. Don't introduce flamewar topics unless you have something genuinely new to say. Avoid unrelated controversies and generic tangents." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
    - ngold 8y ago
      Thanks for the link. That's a great reference point I did not know about.
    - ddtaylor 8y ago
      OP does have a point that this software was subsidized by tax payers. One could argue the NSA needs advanced tools and that the costs of IDA Pro add up.
      - woodman 8y ago
        This, unfortunately, occurs so infrequently that it can safely be ignored by 99.9% of the economy. Businesses have really enjoyed having their cake and eating it too with the transition away from a highly involved acquisition process that generally resulted in a tailored solution that the USG owned, to the present COTS policy that allows them to then go on to sell software to people that have already effectively paid for it through taxes. While there was an impressive amount of bureaucracy and an infinitely self referential system of standards in the old method, it did lead to some pretty interesting side effects: Ada[0], IDEF[1], MIL-STD-498[2], etc. The most recent liberation of useful taxpayer funded software that I can think of was over ten years ago, when NIST released NFIS2 - the fingerprint software that the FBI relied on. They of course had to be crappy about it and wrap it in export controls that limited its utility, but it was interesting to see all the work that internal development had done - very polished, with man pages going back to '97. Ah the memories: software classified as munitions, the clipper chip... [0] http://archive.adaic.com/pol-hist/policy/naig94-1.txt http://archive.adaic.com/pol-hist/policy/naig94-1.txt [1] https://en.wikipedia.org/wiki/IDEF#The_IDEF_modeling_languages https://en.wikipedia.org/wiki/IDEF#The_IDEF_modeling_languag... [2] https://en.wikipedia.org/wiki/MIL-STD-498 https://en.wikipedia.org/wiki/MIL-STD-498 [3] https://web.archive.org/web/20041206072946/http://fingerprint.nist.gov/NFIS/index.html https://web.archive.org/web/20041206072946/http://fingerprin...
subjectsigma 8y ago
I just don't understand the doubt and hate. It's perfectly reasonable to distrust the NSA in most cases, but look at the context - the NSA has a huge brain drain and PR problem. They desperately need qualified people to start trusting and applying to them again. Does anyone seriously think they would try to backdoor security researchers in such a stupidly obvious way? I was actually at the RSA talk where they released the tool - the presenter was very open in saying that this is a recruiting tool. They want college kids just getting into RE to learn their tools and have their name in the back of their mind so they apply for internships and jobs, and are trained for those roles from day zero. There are other benefits to releasing the tool, like free labor and testing from people submitting patches and bug reports, but the real value is in making the NSA appear like the good guys and getting people on their side. It seems pretty obvious to me that this gives the NSA more benefit than trying (and probably failing) to hack random people. And yet the dude sitting next to me was shaking his head and saying he would only ever run it in a VM. Irrational as hell.
- iup 8y ago
  Well, I would apply if they allowed non-US citizens to do so. I have no ties to this shitty post-Soviet country, just let me apply for god's sake.
- stebann 8y ago
  Well, they shouln't have involve in "hacking random people". Then we would trust them. They didn't and they still have surveillance and hacking programs. Why would I expose myself and become a target for the next years? Are they trying to know where are the new targets?
  - subjectsigma 8y ago
    Your comment makes no sense whatsoever. Let's say you're an NSA target. You're probably already hacked. If not, then you are very smart or you haven't been an NSA target for long. Let's assume you're a very smart malware researcher - that means you 1. Already have tools like IDA and don't need this, 2. Have an in depth knowledge of how to acquire and run potentially malicious code safely, 3. Have experience figuring out if that code is malicious. Do you think the winning strategy for the NSA here is to attack you in a way that you're perfectly equipped to deal with?
J253 8y ago
Can anyone speculate as to why the NSA decided to release this? Have they released any OSS in the past?
- noidea_ 8y ago
  Yes they have. Their biggest effort to date is SELinux: https://github.com/SELinuxProject https://github.com/SELinuxProject If you run Linux, you likely have NSA OSS on your machine. You can find their public GitHub profile at: https://github.com/nationalsecurityagency https://github.com/nationalsecurityagency
- pcnix 8y ago
  The seem to do a good amount of OSS work, and from looking at their github page, they have pretty active development on some projects going on.
- MileyCyrax 8y ago
  From https://published-prd.lanyonevents.com/published/rsaus19/sessionsFiles/13678/PNG-T09-Come-Get-Your-Free-NSA-Reverse-Engineering-Tool%21.pdf https://published-prd.lanyonevents.com/published/rsaus19/ses... (page 7): >Why Did We Release Ghidra? > * Improve cybersecurity tools > * Build a community > * Educational Use > * Your tax dollars at work
- whiskeykilo 8y ago
  I suspect they're just trying to expand US cyber capabilities and recruiting. “If I go to the next capture-the-flag contest and I see some college students using Ghidra, I will be really excited” - Rob Joyce, senior cybersecurity adviser at NSA Source: https://www.cyberscoop.com/ghidra-nsa-tool-public/ https://www.cyberscoop.com/ghidra-nsa-tool-public/
cryptokernel 8y ago
This is interesting. Well played NSA, Well played. Just leaving this here: https://coincircle.com/l/50VVxbObg3 https://coincircle.com/l/50VVxbObg3
JoachimS 8y ago
Here is the presentation slides from the Ghidra presentation at the RSA conference: https://published-prd.lanyonevents.com/published/rsaus19/sessionsFiles/13678/PNG-T09-Come-Get-Your-Free-NSA-Reverse-Engineering-Tool%21.pdf https://published-prd.lanyonevents.com/published/rsaus19/ses...
an-allen 8y ago
Nice try NSA.
hatsunearu 8y ago
Oh yeah, for those who are wondering; there's another NSA project where they made a tool that's a direct competitor with a product that's "out there": https://github.com/redhawksdr https://github.com/redhawksdr The competitor in question is GNU Radio.
savgeborn 8y ago
This is Sanskrit name for Vulture Ghid = Vulture Ra = In Sanskrit RA is the acoustic root of fire. RA also connotes with light or spiritual light.
- z3phyr 8y ago
  No. This is clear apophenia. Ghidra is a reference to the Japanese video game boss of the same name, which was supposed to be called Hydra, but due to mis-translation, came as Ghidra In Sanskrit, a vulture is vocally spoken aloud like [Giddh], emphasis on the end. > Ra is the acoustic root of fire Any source for that?
  - unpixer 8y ago
    Funny, I'd assumed it was a reference to the movie monster.
kuroguro 8y ago
I know what I'm doing this weekend :)
z3phyr 8y ago
Just used it to solve the 2015 flare-on challenge #1. Rudimentary, but I am blown away. The interface feels better than IDA, I was able to write a python script straight away! 10/10 recommended. The python interpreter attached with it is aware of the state. Where is my cursor, what memory module I have selected etc. Easy to write scripts for
- Nanocurrency 8y ago
  Any chance you could do a few videos tutorials on using it in conjunction with Python?
  - z3phyr 8y ago
    That's a great idea!! I can try.
    - canada_dry 8y ago
      Remind me! 7 days. Shit... wrong site.
      - z3phyr 8y ago
        Haven't streamed before. Need some time to prepare
jamisteven 8y ago
mother of zeus
_bashskids 8y ago
wish it would be helpful for reversing drivers etc.. for oss community
Semaphor 8y ago
RCE included ;) https://github.com/NationalSecurityAgency/ghidra/issues/6 https://github.com/NationalSecurityAgency/ghidra/issues/6
- freeflight 8y ago
  This not being the top comment kinda scares me. Sure, it might be a great tool for free, but who knows what else might be hidden in there?
  - syn0byte 8y ago
    Likely nothing, it's the source code for an RE toolkit with an NSA sticker right on the box. There is literally no worse place to try to hide back doors. At worst they will know how to mask their real malware from analysis with their own tools.
noodlesUK 8y ago
I’m really hoping this release will improve the situation with learning RE in universities etc. The free version of IDA is very limiting, and few people use the open source and cheaper alternatives (radare2/cutter, binary ninja, hopper). I’m also hoping I can get that decompiler (or something similar) in cutter at some point, but with the source not yet available we’ll have to wait.
bluedino 8y ago
Is it odd that this is written in Java? What advantages does this have?
- AsyncAwait 8y ago
  A lot of people learn Java as part of their CS degree :-)
- HelloNurse 8y ago
  Fairly portable and future-proof, and very easy to compile (just collect dependencies; you need to really go out of your way to compile and "link" Java applications wrong)
karthickgururaj 8y ago
radare2 is an other open source alternative
hendi_ 8y ago
Probably not run that on networks you care about: https://twitter.com/hackerfantastic/status/1103087869063704576 https://twitter.com/hackerfantastic/status/11030878690637045...
- acdha 8y ago
  … in debug mode. Normally it doesn't load the debugging tools.
marrowgari 8y ago
If you run in debug mode it listens on port 18001 allowing for RCE on host machine
AnIdiotOnTheNet 8y ago
Oh cool, this even supports old DOS MZ exes. And it easy to make portable. Edit: no, spoke too soon. It acts like it supports MZ exes but consistently fails to import them.
megous 8y ago
So, I've tried it on some mips binaries I've been reverse engineering on and off last 7 years from assembly, for various reasons. I'm completely blown away by the quality of the decompiler output. The binaries include symbols, so everything global is named correctly, which helps. Anyway, nothing I've tried over the years comes even close to the clean output I'm seeing from Ghidra. It's great.
Sreyanth 8y ago
Am I the only one or is anyone else thinking about what the angle here could be? Pretty impressive software though. Finally one strong open-source alternative for reverse engineering.
morenoh149 8y ago
How did they use jenkins to do this exactly? I'm curious
ru999gol 8y ago
honest question: should I trust the binary they provided? What reason should I have trusting it? Because I don't.
- webninja 8y ago
  Virtualize, containerize, or sandbox it.
the_librarian 8y ago
Discord created for this tool here: (Help with mods is needed!) https://discord.gg/RcSBc6 https://discord.gg/RcSBc6
deleted 8y ago
[deleted]
kuwze 8y ago
This is awesome!
sexpower 8y ago
지렸다..

1283 points by twodayslate 8y ago

This is an unusually large open source project, especially for NSA. I wonder whether they were motivated to release this tool because of their recent brain drain / hiring problems.

maerF0x0 8y ago
Here's a potential angle: If you're going to use a tool internally it's in your best interest to be able hire people w/ experience with that tool. (ie, people learn the tool for free on their own time)

openloop 8y ago

China? Russia? Etc? Really I think this is a good thing.

mrmuagi 8y ago

I'm definitely excited for this, considering I couldn't fork out the thousand of dollars needed for using IDA. I can't really justify that on a small hobby project (reverse engineering games).

snazz 8y ago
The free version (https://www.hex-rays.com/products/ida/support/download_freeware.shtml https://www.hex-rays.com/products/ida/support/download_freew...) would likely meet your needs, assuming the games are for x86.
- convery 8y ago
  The freeware version supports x64 as well.
kabwj 8y ago
If it’s a hobby project why don’t you just pirate it? Honest question.
- tptacek 8y ago
  Somebody sort of casually pirated a copy of IDA Pro back in the mid-2000s (IIRC, he shared his copy on a public server). The IDA people (DataRescue, at the time, but from what I recall the page survived the move to Hex-Rays) found out, banned him from using IDA, and then put a page on their website threatening to rescind IDA licenses from any company that employed him. The IDA team is pretty aggro.
  - jchw 8y ago
    To be fair, there's definitely a difference between downloading a pirated version of software, and actually leaking copies of software for others to pirate. The DataRescue and now Hex-Rays folks seem particularly sensitive to leaked copies and I imagine leaked copies genuinely do affect their bottom line of sales given the kinds of markets they're in.
    - philpem 8y ago
      They certainly seemed to be very much against selling it to private individuals (or were when I asked many years ago). I guess unless you've got a CV which says "presented at Defcon and Blackhat, five times" or "currently work at {big infosec company}", even if you can afford it the answer will be "nope". The end result for me was that I bought a Mac Mini and a copy of Hopper and Synalyze It. My entire reverse-engineering of the Polaroid film recorder driver (and the resulting Linux port) was done by reversing the driver DLL in Hopper and shimming the driver and ASPI calls with PyDbg. I keep looking away for a month or so and finding a new version of Hopper with shiny new features to play with...
      - sdwisely 8y ago
        I really really liked hopper. it's probably the one app I missed most switching to windows. x64dbg eventually grew on me and I like it nearly as much.

poojapanwar27 8y ago

sdfsdf

twodayslate 8y ago

Download: https://ghidra-sre.org/ https://ghidra-sre.org/ GitHub: https://github.com/NationalSecurityAgency/ghidra https://github.com/NationalSecurityAgency/ghidra

WrtCdEvrydy 8y ago
Download from the NSA without open source software... anyone else virtualizing three layers deep to get to this?
- aboutruby 8y ago
  I don't think they would burn some 0-days for this, one container should be enough.
  - omeid2 8y ago
    I think you're underestimating the value of intel on people who know how to get intel or at least interested in the topic.
    - est31 8y ago
      Fortunately, a lot of people who are very knowledgeable about reverse engineering are downloading & opening it as we speak. They will point out any flaws/viruses found. Seems one has already been found [1]. All you need to do is to wait. [1]: https://news.ycombinator.com/item?id=19315515 https://news.ycombinator.com/item?id=19315515
    - strictnein 8y ago
      It's part of the NSA's recruiting push. "If you are interested in projects like this ... consider applying" is even mentioned in the README. There's zero chance there's some secret trojan, because the people who are interested in this type of software are the exact people who would be able to find it.
      - 8y ago

Foxboron 8y ago

Also found this community edition linked on twitter: https://ghidrace.github.io/ https://ghidrace.github.io/

deleted 8y ago
[deleted]
_wmd 8y ago
This link is third party junk, don't touch it. A "fork" existing before the original software release, yeah right
WrtCdEvrydy 8y ago
That's coming from someone else (https://github.com/withzombies?tab=following https://github.com/withzombies?tab=following)
duskwuff 8y ago
There's literally nothing there. The project doesn't even exist yet.

purpleidea 8y ago

Risky click of the day ;)

maxfan8 8y ago

It appears that it isn't actually available as of now. Apparently, the NSA is going to release it at RSA Conference 2019, so it'll probably actually be published within the next couple of days.

arthurcolle 8y ago
It is available here: https://ghidra-sre.org/ https://ghidra-sre.org/
- rock_artist 8y ago
  maybe the NSA don't like non-US citizens using open-sourced software :) this is what I get - 403 ERROR The request could not be satisfied. Request blocked. Generated by cloudfront (CloudFront) Request ID: i_Sa-b1Fj2C4ZAUtBsmAp-7mVg9LerDTlD9t1_00ex4fQyCUhrYvdw==
  - arthurcolle 8y ago
    I have the downloaded files if you want to reach me at my email address in my profile, I can upload them to mega or something
twodayslate 8y ago
Try clearing your cache.
pacificmint 8y ago
The RSA conference is currently ongoing and the NSA talk was today. Actually, it should be over just about now.
- pkaye 8y ago
  Is the NSA talk about this available online?
  - 8y ago

nyrulez 8y ago

I am going to sound pessimistic here, but isn't there a real danger of having this technology available to bad actors and is there any value to keeping such things confidential if it plays a role in national security? If someone was releasing malicious software to hijack the power grid as an example, wouldn't they be first able to use this to try to improve the robustness and invisibility of their attack ? Or is the functionality here common place enough that it doesn't tilt the axis of power in an unfavorable way?

mparlane 8y ago
Bad actors have been using IDA this entire time. So no, not really.
- nyrulez 8y ago
  You mean there is nothing new here? Then why is this news? I am not wondering about the concept of reverse engineering but the specific (and hopefully novel) feature set that this may enable.
  - saagarjha 8y ago
    It's a competitor to IDA's monopoly, basically. It might be better in certain aspects.
    - tptacek 8y ago
      IDA has a bunch of competitors --- Hopper, Binja, and all the Capstone interfaces.
      - saagarjha 8y ago
        Yup, well aware of them (I think Hopper uses Capstone, FWIW). I'm sure you agree that they're not quite at the level of IDA, though ;)

baby 8y ago

Annndddd, it's ugly as fuck. Well, I'm not going to install something made by the NSA on my machine but I'd be interested in feedback.

tptacek 8y ago
It's not as if IDA is a work of art. Arguably, this is a cleaner, more usable interface.
- baby 8y ago
  Totally, that's why I was hopping for something better looking. Looks like Binary Ninja is the only one that cares about that.
  - saagarjha 8y ago
    And Hopper!
Arbalest 8y ago
So no SELinux for you?

sscarduzio 8y ago

Does it work for non windows binaries?

saagarjha 8y ago
Seems to work on a Mach-O and ELF files as well; at least, it loaded those two just fine.
jukeboxbandit 8y ago
>includes a suite of software analysis tools for analyzing compiled code on a variety of platforms including Windows, Mac OS, and Linux From the site, so yes it works on non-windows binaries. It also runs on Linux, Mac and Windows. This is the list of file formats I found in the docs that are supported by Ghidra * Common Object File Format (COFF) * Debug Symbols (DBG) * Executable and Linking Format (ELF) * Ghidra Data Type Archive Format * GZF Input Format * Intel Hex * Mac OS X Mach-O * Module Definition (DEF) * Motorola Hex * New Executable (NE) * Old-style DOS Executable (MZ) * Portable Executable (PE) * Preferred Executable Format (PEF) * Program Mapfile (MAP) * Raw Binary * XML Input Format

slimsag 8y ago

Why this is important (for those uninitiated): - Ghidra is basically the first real competitor to IDA Pro, the extremely expensive and often pirated state-of-the-art software for reverse engineering. Nothing else has come close to IDA Pro. - Ghidra is open-source, IDA Pro is not. - Ghidra has a lot of really cool features that IDA Pro doesn't, such as decompiling binaries to pseudo-C code. - It's also collaborative, which is interesting because multiple people can reverse engineer the same binary at the same time -- something IDA only got VERY recently.

saagarjha 8y ago
> Ghidra has a lot of really cool features that IDA Pro doesn't, such as decompiling binaries to pseudo-C code. To be fair, IDA Pro has a decompiler plugin to do this.
- slimsag 8y ago
  It does, but it costs an extra $1.5k on top of the main software, per architecture.
  - Avery3R 8y ago
    $2.6k+ you mean
- _wmd 8y ago
  For which they charge a per-CPU fortune https://www.hex-rays.com/cgi-bin/quote.cgi https://www.hex-rays.com/cgi-bin/quote.cgi
  - tptacek 8y ago
    It's a funny situation, though: decompilation probably should cost a small fortune. If you're in a line of work that needs it, the quality of your decompiler is probably a huge factor in how valuable an hour of your time is, and many [most?] fields where people routinely decompile stuff are very highly compensated. IDA has always had a weirdly low price point given the bill rates of people who use it, and it's interesting to see that price being competed all the way down to free.

stargazing 8y ago

Aaaaaand: https://twitter.com/hackerfantastic/status/1103087869063704576 https://twitter.com/hackerfantastic/status/11030878690637045...

meowface 8y ago
Though the obvious explanation for that is that it was an intentional backdoor, that honestly looks more to me like a legitimate oversight than a backdoor. I think an actual backdoor would be a lot more subtle and clever than that. Especially since this way, absolutely anyone could exploit it (it's just Java Debug Wire Protocol). Also, you have to explicitly run it in debug mode for this to happen, which probably only a small percentage of end users will do. Kind of seems like the equivalent of running Flask apps in debug mode, which by default will handle exceptions by showing a traceback with an interactive debugger that can be used to execute arbitrary code. There could be some backdoors in it, but I'm leaning towards that not being an intentional one. (But I definitely could be totally wrong; you never know when it comes to intelligence agencies.)
- earenndil 8y ago
  I also don't think it's a backdoor, but the best way to hide a backdoor is to make it look like a mistake.
  - meowface 8y ago
    >but the best way to hide a backdoor is to make it look like a mistake It is, but usually the best way to do is to make it look like a mistake that's very subtle and difficult to notice without careful testing and analysis, kind of like Apple's infamous SSL "goto fail". That's a classic example of a vulnerability that really could be either an honest mistake or a very insidious backdoor. This is more like leaving the house's sliding glass door to the backyard wide open for everyone to see.
  - hiccuphippo 8y ago
    Then again, maybe the best way to hide a backdoor is to have another very obvious one so people look the other way.

souprock 8y ago

It's not the first real competitor available to the public. Hopper Disassembler and Binary Ninja are both capable. They have been available for a few years. Binary Ninja is also collaborative if you get the enterprise edition: https://binary.ninja/purchase/ https://binary.ninja/purchase/

comex 8y ago
They're arguably competitors if you don't care about decompilation. But Binary Ninja has no decompiler and Hopper's was awful last I checked. Ghidra's decompiler seems as competent as Hex-Rays.
- souprock 8y ago
  Binary Ninja has most of a decompiler and is expected to get the rest soon. Binary Ninja offers multiple views of the code, each with an API that gives you the same access that the GUI has. The different views vary in how much they are like assembly or C. Only that last step, real C code, is still missing. Those other views are quite good if your goal is to understand things, but less good if you were hoping to throw the results into a C compiler.
  - comex 8y ago
    Ah, I hadn't heard of the IL functionality. From a quick test of the Binary Ninja demo, it looks like an approach that could become a viable competitor to a decompiler in the future, but isn't a good one in its current state. For instance, one of the most useful aspects of a decompiler for me is the ability to recover high-level control flow, which Binary Ninja apparently doesn't support. Instead it gives you an IDA-like graph view (but with IL instead of assembly in the graph nodes); but at least in my experience using IDA without a decompiler, even moderately large functions tend to result in a spiderweb of a graph, and recovering the control flow by hand ends up feeling like a pointless brain teaser. (This condition being true short-circuits this set of comparisons, so it must be an ||... but it still ends up doing this other set of comparisons, which you can also get to from... wait, where was I again?) It also doesn't seem to allow eliding temporary assignments. Here's a short sample from some random function (retyped by hand since I don't see a way to copy and paste): int32_t edx = arg4 int32_t eax = esi eax_1, ecx_2 = sub_3c670(eax, edx) It does do a sort of SSA transformation and assign unique variable names (like eax_1 instaed of eax) to the same register based on the location in the program, so that's nice. But what I really want is eax_1, ecx_2 = sub_3c670(esi, arg4) I may be missing some option to do this manually, but it should be automatic.

openloop 8y ago

Holy shit.

andrewcchen 8y ago

There's a typo in launch.sh that breaks when you try launch in debug mode > "{$DEBUG_PORT}"

snazz 8y ago

The intro video is pretty good, if you want to see a screen recording: https://ghidra-sre.org/GhidraGettingStartedVideo/GhidraGettingStartedVideo_player.html?embedIFrameId=embeddedSmartPlayerInstance&theme=dusk https://ghidra-sre.org/GhidraGettingStartedVideo/GhidraGetti...

yifanlu 8y ago

From someone who does binary reverse engineering full time, in my experience, BinaryNinja, Hopper, radare2, etc are toys compared to IDA Pro + Hex Rays Decompiler. The quality of the results and the features supported are unmatched... until now. I haven’t spent too much time with ghidra yet but it’s the real deal. The output of the decompiler looks alright (not complete garbage like I’ve seen with other tools). Even if everything else sucks, the decompiler by itself makes it outrank every other tool aside from IDA. And it costs $10k less! The fact that it’ll be open source is just icing on the cake.

tptacek 8y ago
Leaving the decompiler aside, for core disassembly features, in what ways is modern IDA far ahead of its competitors?
- XMPPwocky 8y ago
  To be fair, without undo, Hex-Rays can only move forwards. This explains their advantage.
- simcop2387 8y ago
  Not sure about everything, but last i looked IDA had a lot more support for different architectures and file formats compared to most of the open source stuff (not sure about other proprietary ones).
- yifanlu 8y ago
  Auto analysis when you have barely any information. Any tool can make nice output if you feed it nice input. Try a partial dump from an exotic device and then you’ll see IDA shine.
  - XMPPwocky 8y ago
    Definitely- it's all the years of tweaking and the massive numbers of heuristics to handle, i don't know, code emitted by Microsoft Visual FORTRAN from 1972- that's IDA's moat. Screw the decompiler. If GHIDRA can match that, it's a huge step forwards.

Defcon6 8y ago

https://twitter.com/hackerfantastic/status/1103087869063704576?s=21 https://twitter.com/hackerfantastic/status/11030878690637045...

vasilia 8y ago

Are they serious? They are banning Russian IPs with decompiler source code. Hmm, I know ARM and x86 assembly. Of course, I don't know how to download these sources :)

empath75 8y ago
Legal reasons.
- vasilia 8y ago
  As I see it's licensed under Apache 2.0. I don't know about any regional restrictions for this type of license. But it's could be a real reason because of our stupid government.

keypnchr 8y ago

Started to watch the video, then remembered Snow Crash.

ramon 8y ago

does Ghidra work for APK files, it would be cool if it also worked with mobile projects.

kdbg 8y ago
Yes it does work with APKs. Well, it should. It kept having issues with the only APK I had on hand, but when I just pulled some DEX files out and loaded them it handled them just fine (including decompilation)
tatoalo 8y ago
Direct APK doesn't work at the moment (I've tried with 3 apks), just unzip it and work with the DEX file.

alexozer 8y ago

I wonder how this compares to retdec, an open source cross-architecture decompiler by Avast. https://github.com/avast-tl/retdec https://github.com/avast-tl/retdec

snazz 8y ago
I'm wondering this too. I haven't heard of retdec being used too much, but it looks very cool. I'll guess that Hex-Rays is better, but I still am interested in the opinion of someone more experienced who has tried retdec.
- DannyBee 8y ago
  I'll go one better: I've contributed patches to retdec. Retdec is ... okay. On small binaries it's usable. On even average sized windows binaries (a few meg), not really. Like on things that IDA takes 10-15 minutes and a reasonable amount of memory (like a 7 meg windows binary), retdec can take forever and unlimited amounts of memory. I started fixing a lot of the memory issues (completely recursive CFG traversal, etc), but there are also very serious algorithmic issues (N^3/N^4 algorithms in the optimizers). If i disable a lot of the backend optimizers, i can make it work okay. But then the output is also a lot larger/worse. To be fair: It used to be about 50x bigger than similar IDA output. The latest development version of retdec now has a new backend IR converter, and the output is only 5x-10x bigger than IDA output. So as a TL;DR: retdec in its default state is unusable for anything but small binaries. If you understand what is going on, you can get it to work on a lot of binaries as long as you have a ton of memory and time to spare.
tptacek 8y ago
Retdec is based in part on Capstone, like Hopper.
- alexozer 8y ago
  I believe Capstone is merely a disassembly framework, and retdec's decompilation process is custom implementation which works on LLVM IR.

ryanmarsh 8y ago

The logo reminds me of 8 chan's logo.

5mu3eke56mu 8y ago
my first thought
ecommerceguy 8y ago
indeed.

imjustsaying 8y ago

>clicking the link before reading the domain regret

deleted 8y ago

[deleted]

zelon88 8y ago

I'm curious what feature specifically prompted the NSA to develop their own IDA Pro alternative. I mean, someone somewhere at the NSA must have been trying to do something with IDA Pro only to repeatedly fail before the decision was made that whatever the NSA was trying to do warranted developing their own IDA Pro... right? Or perhaps they used IDA Pro so often and grew so frustrated by it that they started their own?

mox1 8y ago
Is it possible that maybe this pre-dates IDA Pro? Ever tried to use IDA Pro on the same project with a co-worker...at the same time?
- souprock 8y ago
  I believe the included decompiler predates the one that goes with IDA Pro, so that could be a motivation. IDA Pro itself is really really old, having once been a 16-bit DOS shareware program, which explains why IDA Pro doesn't use the GUI shortcuts that were standardized much later. IDA Pro still doesn't support collaboration, although there are very broken hacks that attempt to add it. Binary Ninja supports collaboration if you buy the enterprise edition.
tptacek 8y ago
Hex-Rays can be hard to deal with, and the IC deals pretty extensively with large federal contractors like Raytheon, so it's possible they just needed something as capable as IDA that they could roll out across all their suppliers to use as a common toolchain and interchange format. But it's also possible this is just sort of a labor of love type thing.
- comex 8y ago
  There’s a third possibility: they wanted a piece of software they could customize to meet their needs. Admittedly, for simple bugfixes and the like, Hex-Rays’ support is known for being quite responsive (as suits the small number of customers). There’s also a quite large (albeit poorly-documented and crash-prone) SDK, which can handle a wide variety of needs and has gained functionality over time. But if you want to add new functionality that can’t be implemented using the SDK? (That includes just about anything related to the decompiler.) You’re at the mercy of Ilfak’s priority list. Well, mere mortals are, at least; the NSA has enough money that it might be able to set up some sort of special contract with Hex-Rays, but I suspect that’s easier said than done. From that perspective, the ideal is what the NSA ended up with, a codebase whose development is fully in-house. Notably, though, second best would be to just have access to the source code of an existing tool, so you can at least make your own patches if necessary, even if you’re not in control of the codebase’s overall direction. Did the NSA ever seek that in IDA’s case, and could they have obtained it if they did? I don’t know the answer to either question… but source access certainly isn’t offered to typical customers. In general I’m surprised that “paid + source access for customers” isn’t a more popular model of software development. From my perspective, which admittedly is very different from the NSA’s, I was never very interested in low-cost IDA competitors like Hopper or Binary Ninja, but I’m very excited about Ghidra. Why? Partly because it’s a more full-fledged competitor in terms of feature set, I admit – but the competitors I mentioned are bound to narrow the gap over time. Partly because of cost: I myself am at a point where I could justify the $600/y for Binary Ninja’s commercial edition, or even the order-of-magnitude-higher cost of the Hex-Rays decompilers, without wincing too badly. but I believe that reverse engineering should be accessible to beginners and amateurs. (Piracy is a partial solution, including in IDA’s case, but some people don’t like to do that.). But the main reason I’m excited about Ghidra is that I have the source code. As a concrete example, I’ve spent a good amount of time reverse engineering software for the Nintendo Wii and Wii U. Both consoles have a main CPU based on the PowerPC architecture, but with a custom ISA extension for an extremely barebones version of SIMD. Well, both Hex-Rays and Ghidra support PowerPC decompilation (although that’s a relatively recent development), but unsurprisingly, neither of them have full support for that ISA extension. IDA actually does have built-in support for disassembling it, but AFAIK not for decompiling; Ghidra doesn’t seem to support it at all (but I may just need to configure it properly). What can I do? Well, in practice, nothing, because I don’t care about the Wii U anymore. But if Ghidra had been released a few years ago, I’m pretty sure I would have gone and implemented support for the extension myself; I haven’t looked at Ghidra’s source yet, but since it already supports other vector ISAs, it probably wouldn’t be that hard. With IDA, I was stuck. The SDK supports adding custom instruction sets for disassembly, but the decompiler SDK is so limited that supporting them there is either impossible or at least would be a huge hack. And that’s just one of many customizations I‘ve wanted over the years. Some of them are probably easier said than implemented, but at least now I can put that to the test!

xxpor 8y ago

Why do they have a LICENCE file with the Apache Licence in it? As a work of the federal government this is public domain by definition, isn't it? edit: Oh, outside contributors of course retain their own copyright. That's what's licenced. https://github.com/NationalSecurityAgency/ghidra/blob/master/NOTICE https://github.com/NationalSecurityAgency/ghidra/blob/master...

paxys 8y ago
They can also enforce copyright in other countries.
- pbhjpbhj 8y ago
  Can they? How's that work, surely a resident in USA could take the work to the other country and make it available, if they can't then it wasn't public domain in the first place? Can you point me to any caselaw/ analysis please?
  - MileyCyrax 8y ago
    I don't quite get it either, but one of their documentation files seems to imply that they can: >In countries where copyright protection is available (which does not include the U.S.), contributions made by U.S. Federal Government employees are released under the License. Merged contributions from private contributors are released under the License. https://github.com/NationalSecurityAgency/ghidra/blob/master/INTENT.md https://github.com/NationalSecurityAgency/ghidra/blob/master...
  - webninja 8y ago
    Guns
vharuck 8y ago
A common work-around is for a federal agency to hire a contractor to write software and require the copyright be transferred as part of the contract. The federal government can hold copyrights, so it's legally kosher.

m0zg 8y ago

Yeah, I think I'm gonna pass on this strictly binary release and wait until the code is released, reviewed, and independently compiled.

poojapanwar27 8y ago

test

yalogin 8y ago

You are the leader in your segment of the market one day and the undisputed leader. You wake up and the NSA decides to send a free competitor out with better or matching functionality. Tough blow. But good for us.

Tepix 8y ago
I'm surprised that noone is yelling "socialism" yet. Tax paid competition for existing commercial products. Isn't that considered evil/wrong by pure capitalists?
- dang 8y ago
  "Eschew flamebait. Don't introduce flamewar topics unless you have something genuinely new to say. Avoid unrelated controversies and generic tangents." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
  - ngold 8y ago
    Thanks for the link. That's a great reference point I did not know about.
  - ddtaylor 8y ago
    OP does have a point that this software was subsidized by tax payers. One could argue the NSA needs advanced tools and that the costs of IDA Pro add up.
    - woodman 8y ago
      This, unfortunately, occurs so infrequently that it can safely be ignored by 99.9% of the economy. Businesses have really enjoyed having their cake and eating it too with the transition away from a highly involved acquisition process that generally resulted in a tailored solution that the USG owned, to the present COTS policy that allows them to then go on to sell software to people that have already effectively paid for it through taxes. While there was an impressive amount of bureaucracy and an infinitely self referential system of standards in the old method, it did lead to some pretty interesting side effects: Ada[0], IDEF[1], MIL-STD-498[2], etc. The most recent liberation of useful taxpayer funded software that I can think of was over ten years ago, when NIST released NFIS2 - the fingerprint software that the FBI relied on. They of course had to be crappy about it and wrap it in export controls that limited its utility, but it was interesting to see all the work that internal development had done - very polished, with man pages going back to '97. Ah the memories: software classified as munitions, the clipper chip... [0] http://archive.adaic.com/pol-hist/policy/naig94-1.txt http://archive.adaic.com/pol-hist/policy/naig94-1.txt [1] https://en.wikipedia.org/wiki/IDEF#The_IDEF_modeling_languages https://en.wikipedia.org/wiki/IDEF#The_IDEF_modeling_languag... [2] https://en.wikipedia.org/wiki/MIL-STD-498 https://en.wikipedia.org/wiki/MIL-STD-498 [3] https://web.archive.org/web/20041206072946/http://fingerprint.nist.gov/NFIS/index.html https://web.archive.org/web/20041206072946/http://fingerprin...

subjectsigma 8y ago

I just don't understand the doubt and hate. It's perfectly reasonable to distrust the NSA in most cases, but look at the context - the NSA has a huge brain drain and PR problem. They desperately need qualified people to start trusting and applying to them again. Does anyone seriously think they would try to backdoor security researchers in such a stupidly obvious way? I was actually at the RSA talk where they released the tool - the presenter was very open in saying that this is a recruiting tool. They want college kids just getting into RE to learn their tools and have their name in the back of their mind so they apply for internships and jobs, and are trained for those roles from day zero. There are other benefits to releasing the tool, like free labor and testing from people submitting patches and bug reports, but the real value is in making the NSA appear like the good guys and getting people on their side. It seems pretty obvious to me that this gives the NSA more benefit than trying (and probably failing) to hack random people. And yet the dude sitting next to me was shaking his head and saying he would only ever run it in a VM. Irrational as hell.

iup 8y ago
Well, I would apply if they allowed non-US citizens to do so. I have no ties to this shitty post-Soviet country, just let me apply for god's sake.
stebann 8y ago
Well, they shouln't have involve in "hacking random people". Then we would trust them. They didn't and they still have surveillance and hacking programs. Why would I expose myself and become a target for the next years? Are they trying to know where are the new targets?
- subjectsigma 8y ago
  Your comment makes no sense whatsoever. Let's say you're an NSA target. You're probably already hacked. If not, then you are very smart or you haven't been an NSA target for long. Let's assume you're a very smart malware researcher - that means you 1. Already have tools like IDA and don't need this, 2. Have an in depth knowledge of how to acquire and run potentially malicious code safely, 3. Have experience figuring out if that code is malicious. Do you think the winning strategy for the NSA here is to attack you in a way that you're perfectly equipped to deal with?

J253 8y ago

Can anyone speculate as to why the NSA decided to release this? Have they released any OSS in the past?

noidea_ 8y ago
Yes they have. Their biggest effort to date is SELinux: https://github.com/SELinuxProject https://github.com/SELinuxProject If you run Linux, you likely have NSA OSS on your machine. You can find their public GitHub profile at: https://github.com/nationalsecurityagency https://github.com/nationalsecurityagency
pcnix 8y ago
The seem to do a good amount of OSS work, and from looking at their github page, they have pretty active development on some projects going on.
MileyCyrax 8y ago
From https://published-prd.lanyonevents.com/published/rsaus19/sessionsFiles/13678/PNG-T09-Come-Get-Your-Free-NSA-Reverse-Engineering-Tool%21.pdf https://published-prd.lanyonevents.com/published/rsaus19/ses... (page 7): >Why Did We Release Ghidra? > * Improve cybersecurity tools > * Build a community > * Educational Use > * Your tax dollars at work
whiskeykilo 8y ago
I suspect they're just trying to expand US cyber capabilities and recruiting. “If I go to the next capture-the-flag contest and I see some college students using Ghidra, I will be really excited” - Rob Joyce, senior cybersecurity adviser at NSA Source: https://www.cyberscoop.com/ghidra-nsa-tool-public/ https://www.cyberscoop.com/ghidra-nsa-tool-public/

cryptokernel 8y ago

This is interesting. Well played NSA, Well played. Just leaving this here: https://coincircle.com/l/50VVxbObg3 https://coincircle.com/l/50VVxbObg3

JoachimS 8y ago

Here is the presentation slides from the Ghidra presentation at the RSA conference: https://published-prd.lanyonevents.com/published/rsaus19/sessionsFiles/13678/PNG-T09-Come-Get-Your-Free-NSA-Reverse-Engineering-Tool%21.pdf https://published-prd.lanyonevents.com/published/rsaus19/ses...

an-allen 8y ago

Nice try NSA.

hatsunearu 8y ago

Oh yeah, for those who are wondering; there's another NSA project where they made a tool that's a direct competitor with a product that's "out there": https://github.com/redhawksdr https://github.com/redhawksdr The competitor in question is GNU Radio.

savgeborn 8y ago

This is Sanskrit name for Vulture Ghid = Vulture Ra = In Sanskrit RA is the acoustic root of fire. RA also connotes with light or spiritual light.

z3phyr 8y ago
No. This is clear apophenia. Ghidra is a reference to the Japanese video game boss of the same name, which was supposed to be called Hydra, but due to mis-translation, came as Ghidra In Sanskrit, a vulture is vocally spoken aloud like [Giddh], emphasis on the end. > Ra is the acoustic root of fire Any source for that?
- unpixer 8y ago
  Funny, I'd assumed it was a reference to the movie monster.

kuroguro 8y ago

I know what I'm doing this weekend :)

z3phyr 8y ago

Just used it to solve the 2015 flare-on challenge #1. Rudimentary, but I am blown away. The interface feels better than IDA, I was able to write a python script straight away! 10/10 recommended. The python interpreter attached with it is aware of the state. Where is my cursor, what memory module I have selected etc. Easy to write scripts for

Nanocurrency 8y ago
Any chance you could do a few videos tutorials on using it in conjunction with Python?
- z3phyr 8y ago
  That's a great idea!! I can try.
  - canada_dry 8y ago
    Remind me! 7 days. Shit... wrong site.
    - z3phyr 8y ago
      Haven't streamed before. Need some time to prepare

jamisteven 8y ago

mother of zeus

_bashskids 8y ago

wish it would be helpful for reversing drivers etc.. for oss community

Semaphor 8y ago

RCE included ;) https://github.com/NationalSecurityAgency/ghidra/issues/6 https://github.com/NationalSecurityAgency/ghidra/issues/6

freeflight 8y ago
This not being the top comment kinda scares me. Sure, it might be a great tool for free, but who knows what else might be hidden in there?
- syn0byte 8y ago
  Likely nothing, it's the source code for an RE toolkit with an NSA sticker right on the box. There is literally no worse place to try to hide back doors. At worst they will know how to mask their real malware from analysis with their own tools.

noodlesUK 8y ago

I’m really hoping this release will improve the situation with learning RE in universities etc. The free version of IDA is very limiting, and few people use the open source and cheaper alternatives (radare2/cutter, binary ninja, hopper). I’m also hoping I can get that decompiler (or something similar) in cutter at some point, but with the source not yet available we’ll have to wait.

bluedino 8y ago

Is it odd that this is written in Java? What advantages does this have?

AsyncAwait 8y ago
A lot of people learn Java as part of their CS degree :-)
HelloNurse 8y ago
Fairly portable and future-proof, and very easy to compile (just collect dependencies; you need to really go out of your way to compile and "link" Java applications wrong)

karthickgururaj 8y ago

radare2 is an other open source alternative

hendi_ 8y ago

Probably not run that on networks you care about: https://twitter.com/hackerfantastic/status/1103087869063704576 https://twitter.com/hackerfantastic/status/11030878690637045...

acdha 8y ago
… in debug mode. Normally it doesn't load the debugging tools.

marrowgari 8y ago

If you run in debug mode it listens on port 18001 allowing for RCE on host machine

AnIdiotOnTheNet 8y ago

Oh cool, this even supports old DOS MZ exes. And it easy to make portable. Edit: no, spoke too soon. It acts like it supports MZ exes but consistently fails to import them.

megous 8y ago

So, I've tried it on some mips binaries I've been reverse engineering on and off last 7 years from assembly, for various reasons. I'm completely blown away by the quality of the decompiler output. The binaries include symbols, so everything global is named correctly, which helps. Anyway, nothing I've tried over the years comes even close to the clean output I'm seeing from Ghidra. It's great.

Sreyanth 8y ago

Am I the only one or is anyone else thinking about what the angle here could be? Pretty impressive software though. Finally one strong open-source alternative for reverse engineering.

morenoh149 8y ago

How did they use jenkins to do this exactly? I'm curious

ru999gol 8y ago

honest question: should I trust the binary they provided? What reason should I have trusting it? Because I don't.

webninja 8y ago
Virtualize, containerize, or sandbox it.

the_librarian 8y ago

Discord created for this tool here: (Help with mods is needed!) https://discord.gg/RcSBc6 https://discord.gg/RcSBc6

deleted 8y ago

[deleted]

kuwze 8y ago

This is awesome!

sexpower 8y ago

지렸다..