8 ms·

I found a bug in the .NET framework and fixed it by hand-altering the DLL

332 points by antics 12y ago

rubyrescue 12y ago
My knowledge of the GAC is out of date but isn't a bit of a security hole that you can replace that DLL?
- deleted 12y ago
  [deleted]
- tomasr 12y ago
  You need to be a local administrator to write to the GAC by default. See http://msdn.microsoft.com/en-us/library/yf1d93sz(v=vs.110).aspx http://msdn.microsoft.com/en-us/library/yf1d93sz(v=vs.110).a... So if you can modify the files in the GAC, you're already compromised at that point.
  - bmm6o 12y ago
    Writing to the GAC is easy, but assemblies in the GAC are strongly-named. Actually replacing one without the private key would require forging a signature, which is what parent was probably referring to. You can give an assembly the same name, but if it's signed with a different key it gets a different strong name (and therefore goes in a slightly different directory).
- jameshart 12y ago
  They didn't replace it - it looks like they created a new assembly with its own strong name, registered it alongside the 'official' assembly, and referenced their assembly in place of the .NET native assembly at compile time. What's maybe surprising is that the C# compiler generated code 'adopts' the types in their hacked expressions library. But the C# compiler, wherever it depends on particular types that need to exist for it to do its thing, has always generally relied on finding those types by -name-, not by -name and assembly strong name-. For example, the C# 3 compiler understands the calling and declaration syntax of extension methods, but to compile an extension method, it requires an Attribute type - System.Runtime.CompilerServices.ExtensionAttribute - which was only introduced into .NET in v3.5. But if you make an attribute class with that name available to the C# compiler, you can compile code using C#3 extension method syntax against the .NET 2.0 framework libraries.
- overgard 12y ago
  I don't think it's a replacement, since signing the new DLL would require a new key. (GAC DLL identity basically comes down to name + signing key. So even if you call it System.Whatever, it has a unique identity)
breischl 12y ago
Wow, that is some fancy work. Though I shudder to think about deploying and maintaining that fix, and bringing new developers up to speed on why and how it was done. I was also about to tell him to skip all the ildasm stuff and just use the online reference source (referencesource.microsoft.com) ... but that assembly isn't in there. So I guess ildasm was the best option.
skrebbel 12y ago
The real news here is that this is considered special. In most other languages, they'd have forked the source, fixed it, recompiled it for their own uses and submitted a pull request or patch. I'm a big .NET fan, but the fact that we have to jump through such hoops to find and fix a bug, and then still have near certainty that we're going to have to reapply the patch for many updates to come, well, that's just a bit sad. It feels rather last-century, to be honest. Microsoft could save a lot of double work if they'd just open source .NET and attach a decent process to it. They can still be the Linus. Just consider my patches.
- toomuchtodo 12y ago
  I work at a startup that uses almost all open source tools for our stack. To not be able to fork, fix, pull request a bug in software seems...barbaric to me.
- antics 12y ago
  ---Author here--- I should disclose that I actually work for Microsoft, and still chose the hard way. Fuck the police!
  - Alupis 12y ago
    > and still chose the hard way Chose? Unless you work on the team that made that DLL, it's likely you'd be jumping through hoops even internally.
    - wwwwwwwwww 12y ago
      i'm glad someone who works at the company like you was able to enlighten us as to microsoft's internal practices oh wait
      - Scuttles 12y ago
        If you pay any attention to Microsoft, their business practices, news, reports, blogs, etc... then you would know the parent comment is pretty much spot on.
DanielBMarkham 12y ago
Sidebar: I haven't done deep C# in several years, since moving to F#. This code is getting to look butt-ugly. It is not a good thing if it continues like this. We already have C++. Don't need another one.
- _random_ 12y ago
  Don't worry, each new version of C# gets more features from F# and Scala, this year we will get more syntax sugar and an extensible compiler (with proper IDE support - unlike F#): https://roslyn.codeplex.com/wikipage?title=Language%20Feature%20Status&referringTitle=Home https://roslyn.codeplex.com/wikipage?title=Language%20Featur... . They will add pattern matching and maybe something more interesting in the version after that. There aren't too many practical reasons to pick F# over C# today and soon there will be even less. Lesson to learn: don't pick *ML language as a foundation. I am glad that Scala is evading F#'s academic fate.
  - codygman 12y ago
    F#'s academic fate? I'm curious as to why you think this. I'm guessing you are of the opinion Haskell is forever trapped in academia?
- noblethrasher 12y ago
  I do both C# and F#. But, at this point I think in ML no matter what which language I'm using. That said, though I'm delighted by F#, there are things that I really miss about C#: * Haskell's type classes are awesome, and neither F# nor C# have them. But, you can realize them with a really simple pattern in C# whereas in F# you have to resort to reflection or generic hacks[1] (but inline functions usually make such hacks nicer in F#). * Sum types are awesome. F# realizes them in two ways: with discriminated unions and pattern matching (mainly), or with abstract classes and inheritance. Both are powerful approaches, but the latter is particularly useful when you need unbounded sum types. Unfortunately, inheritance is somewhat crippled in F# because you can't define protected members (though you can override them). I find myself needing this when I'm designing a solution rather than merely implementing one. For instance, I find that designing a language/compiler is much easier with C# than with idiomatic F#. * This is kind of a summary of the first two points: idiomatic C# is a better dynamic language, and I'm not referring to the `dynamic` keyword, though it certainly helps. I've created some really nice and secure APIs by making use of user-defined conversions. System.Linq.Xml is a good example of what I'm talking about. * Finally, idiomatic C# is a nicer classical OO language than idiomatic F#. This is kind of obvious since F# is a "functional language" with OO features whereas C# is an OO language with functional features. But, when I say classical OO, I mean the Smalltalk style OO that came before C++. Alan Kay defined that kind of OOP as (1) Encapsulation, (2) Message Passing, and (3) extreme late binding. Neither C# nor F# quite has (2) and (3), but C# gets me the closest without coloring too far outside of the lines. Of course, this begs the question that OO is better than functional. Having done both for a long time, I'm convinced that it is the case (though I do think learning functional programming is the fastest path to getting good at OO). Lamentably, I can't yet defend this point too vigorously because I've only just discovered the joy or Smalltalk. But, here is a preview of sorts: Imagine being able to update a database schema and having your statically-typed code continue to run without needing to recompile. [1] http://stackoverflow.com/questions/9868327/f-type-constraints-and-overloading-resolution http://stackoverflow.com/questions/9868327/f-type-constraint...
colanderman 12y ago
I did this once with GCC. "But GCC's open source!" you say. Well, GCC is next-to-impossible to compile for a target other than the host, especially if the target isn't x86 or ARM; and GCC maintainers insist on precise test cases to vet a bug, even if the issue is immediately obvious from reading the source code and the bug only occurs in certain very complex situations. (/me looks forward to the day Clang/LLVM becomes the default on Linux…)
- ikonst 12y ago
  Just recently I wrote https://github.com/ikonst/LongPathFix https://github.com/ikonst/LongPathFix to work around a clang/gcc issue, cause it seemed easier than going through the hoops of building clang for Win32, understanding its code and writing a patch good enough to be accepted.
- zvrba 12y ago
  > Well, GCC is next-to-impossible to compile for a target other than the host, especially if the target isn't x86 or ARM; You're exaggerating. I had no problems compiling gcc 3.something targeting MIPS-I on an x86 Linux host.
  - colanderman 12y ago
    I can't tell if you're being sarcastic. Generally, compiling GCC to target another architecture involves not only recreating most of the root directory structure (notably include files) of the target on the host, but also compiling libc, bintools, and a few other more obscure libraries. And this all needs to be done in the correct order (which involves something like compiling half of libc before compiling GCC, and compiling the rest of libc afterward). Maybe in 3.x and MIPS it "just worked". My experience is 4.x and Tilera. Myself and another engineer dumped a week into that sinkhole before giving up. Clang/LLVM on the other hand… ./configure && make && make install. No other wacky dependencies or build steps. And it generates better code in many cases (particularly when dealing with structures).
    - microcolonel 12y ago
nope_42 12y ago
Should have just used dotPeek instead of ilspy and writing IL code by hand. Recompiling would have certainly been easier. http://www.jetbrains.com/decompiler/ http://www.jetbrains.com/decompiler/
- antics 12y ago
  ---Author here--- AWESOME TIP, stoked to try it out, thanks!
  - nervousvarun 12y ago
    This kinda goes against a popular theory that Microsoft mine JetBrains for Visual Studio functionality improvement ideas (or more likely the blog author just doesn't work on VS or w/ the VS dev team all that much) Awesome read btw!
cornholio 12y ago
Dude, that's not a DLL, it's a pile of half-digested scripting language. Real men patch in assembly.
stusmith1977 12y ago
In situations like this, you can report the bug to Microsoft Connect: 1. Submit bug report. 2. Wait six months. 3. MS tech will post a comment, "this will be fixed in the next release". 4. Wait two more years. 5. Bug report will be closed as "won't fix".
- allegory 12y ago
  This. One bug in IE9 clickOnce launching thanks to them changing how download prompting works. 1. Reported to connect whilst in preview release status. Closed. Reported again. Closed. FULL test cases provided. 2. We're a gold partner with a £500k spend a year on licenses. Partner support. 19 hours on the phone over 6 months, blame shifting between the IE and .net teams and a daily call to get the case closed without resolution. Got a registry patch from ass-end support after 4 months that we have to ship to 2000 users at 200 different companies rather than an upstream fix. This checks a check box in the security settings. They broke their own product and won't fix it. Basically you can't use JS to redirect to a clickonce URL. Now today, IIS just stopped serving shit with no errors, nothing. Can't get anything out of minidumps+windbg. Just stops. None of our code is running. Who am I going to call? Redhat that's who.
  - Pxtl 12y ago
    Yup. Its funny as a dot net dev you learn to really love a few ms things and really hate the others. C# is a brilliant language. Entity framework is an awesome ORM. But anything to do with deployment? IIS? So very much rage. I'm in charge of the build process on my team and I'm quickly becoming "that guy in the corner who swears all the time".
    - locusm 12y ago
      Your still not cursing as much as the guy building msi installers though...
      - allegory 12y ago
        Again, this. Even with WiX it's painful. HKLM vs HKCU + corporate deployment hell + why the fuck do I need two MSIs to bootstrap the CLR and VSTO. Argh. I deal with VSTO, WiX, ClickOnce, IIS, COM, MSMQ and the usual bits. Pays well but it made my hair fall out and has taken a couple of years off my life at least. I long for the gong to ring so I can go home to my MacBook and OpenBSD (where I truly belong).
batmansbelt 12y ago
Fuck bro, that's hard-core.
b0b0b0b 12y ago
Why wasn't the DLL signed? Is this not a thing?
- Permit 12y ago
  DLLs are signed. But they aren't checked every time a DLL is loaded. It absolutely obliterates load time if you have to go through every byte of all your DLLs and hash them. (You have the option to turn this on within the registry, though) Signing is almost useless in .Net. And it's certainly not in place for security purposes.
- kevingadd 12y ago
  Signing wouldn't have addressed this because they're creating a new, modified executable. So they just make that one unsigned.
euroclydon 12y ago
I'm sad to report that I tried to replicate what the author did here and it didn't work... I went to the weekly tech lead meeting, and when it was my turn to talk, I said: "I think we should increase out internal NuGet package release interval to every 18 hours. F--k the Police!" People just looked at me weird.
jmesserly 12y ago
Funny, I had something to do with this code back in the day! I'm guessing it was a copy+paste bug and they copied from the LambdaCompiler, which uses StrongBox<T> for its closed-over parameters[1], since StrongBox<T>.Value is a field. The idea was to have the closures be really fast. The history of ET compiler: it started with LINQ in .NET 3.5. Originally it was pretty simple and just handled expressions. In .NET 4.0 we merged the entire codebase with the IronPython/IronRuby compiler trees, expanding the "expression trees" to handle statements. IIRC, it can generate almost any IL construct that you might need, and is usually a lot easier to work with. But we found .NET's runtime compiler (DynamicMethod) was a bit too slow for a lot of use cases. It also wasn't supported on some CLR configurations. To address this we wrote an interpreter and some heuristics to switch from interpreted to compiled. But the actual System.Linq.Expressions.Interpreter must have happened after 4.0, because I don't remember that at all. Instead we just shipped it as a shared library used by IronPython and IronRuby. Here's the normal ExpressionQuoter: https://github.com/IronLanguages/main/blob/7be8b73e246bfb029880a7ebd8885cc541b705ac/Runtime/Microsoft.Scripting.Core/Compiler/ExpressionQuoter.cs#L152 https://github.com/IronLanguages/main/blob/7be8b73e246bfb029... And here was the interpreter. I don't see the ExpressionQuoter, so either that's a newer fork of the code that was rolled into System.Core, or maybe a completely new implementation. https://github.com/IronLanguages/main/tree/master/Runtime/Microsoft.Dynamic/Interpreter https://github.com/IronLanguages/main/tree/master/Runtime/Mi... IIRC, ExpressionQuoter was mainly to support the Quote expression, and was always a bit buggy. The 3.5 version was seriously messed up, and our prerelease versions of .NET 4.0 also had various bugs, and very few tests. I tried to fix it by having it use the same reuse closure mechanism as the normal compiler. Funny that same feature caused issues later on. [1] one might wonder: why use StrongBox<T>, essentially boxing every parameter, rather than just generating a type with only the right fields? The reason was that generating a type in .NET 4.0 timeframe was absurdly slow. Like, a few hundred per second slow. I think this has been largely fixed now, but it was a huge performance problem for Iron* language runtimes back in the day
- CurtHagenlocher 12y ago
  So, do you think this is Dino's fault? :P
swalsh 12y ago
Wow, how did I not know about this ExpressionBlock class! There's always another gem waiting in the framework :D I wish i knew about it about a year ago for a project I did.
Permit 12y ago
I know a lot of people here balk at the idea of paying for tooling, but Red Gate's Reflector[1] is absolutely amazing for situations like this. Not only is it a decompiler, but it allows you to decompile at debug-time and step into third party libraries. Assuming they haven't been obfuscated, this is an extremely useful tool. I've used it to track down a number of issues within Visual Studio itself and within some of the non-open sourced components of Roslyn. [1] http://www.red-gate.com/products/dotnet-development/reflector/ http://www.red-gate.com/products/dotnet-development/reflecto...
- wffurr 12y ago
  You are justifying closed-source tools by using a closed-source tool to debug your closed-source tools?
  - allegory 12y ago
    That's exactly how the windows ecosystem works. Everything is pay per view. By the time we've got the PO's signed if it was open source then I'd have solved the problem at hand. To get around this, defensive purchasing is required: at least one VS ultimate license, at least one MSDN sub, ANTS profiler, IDA Pro, Azure VM allocation ready to roll for test machines and two guys who actually know something about all this. This is the safety belt cost: about $30000 + $120k a person
- voltagex_ 12y ago
  Don't pay for Reflector. Throw JetBrains some more money - DotPeek [1] integrates nicely with ReSharper, or you can use it on its own for free. I think the latest build can also provide PDBs to Visual Studio on the fly by acting as a source server (insane). After years of Reflector being a free tool, Redgate added a time-bomb to new downloads and released a statement [2] [1]: http://www.jetbrains.com/decompiler/ http://www.jetbrains.com/decompiler/ [2]: https://web.archive.org/web/20110205074826/http://www.red-gate.com/products/dotnet-development/reflector/announcement https://web.archive.org/web/20110205074826/http://www.red-ga...
jacquesm 12y ago
The fact that you can even do this without setting off a ton of alarmbells about failed checksums is what really scares me.
- Permit 12y ago
  How would you enforce that? First, hashing every byte of every loaded DLL destroys your runtime. Second, if you're able to modify one DLL, you could overwrite whatever DLL was tasked with computing the checksum.
- Locke1689 12y ago
  They recompiled with the new binaries.
- reubenbond 12y ago
  You can produce and consume signed libraries in .NET very easily. In practice it's becoming less common as .NET becomes more open. As an aside, OP says he works with Bart De Smet and they're messing with expression trees. I bet that this is work related to Cortana and the expression trees are used for compiling standing Rx (Reactive Extensions) queries which Cortana uses extensively on the phone and in the cloud.
- userbinator 12y ago
  The fact that people think you shouldn't be able to do this is what really scares me. :) I should be able to change the software on my own machine, stored on and running on hardware I own, in whatever way I desire and have it do what I want. (And in practice I have - opening a binary in a hex editor and changing a few bytes is not at all beyond me.)
  - jacquesm 12y ago
    Of course you should be. But it still should generate warnings and force you to override some fairly impressive 'do you know what you are doing?' warnings.
    - 12y ago
x0n 12y ago
I did the same for the Invoke-RestMethod cmdlet in PowerShell 3.0: http://nivot.org/blog/post/2013/05/07/PowerShell-Masochistic-Assembly-Patching-for-Guts-and-Glory http://nivot.org/blog/post/2013/05/07/PowerShell-Masochistic...
xorcist 12y ago
I take issue with the authors use of the word "patch". He's not actually patching the DLL, just decompiling and recompiling which are quite familiar for most programmers. There might be some black magic associated with this particular DLL that we non-.net programmers doesn't understand. I would personally be very careful with the recompiling dance, at least in other languages, as alignments and such might come out of place. A patch feels much less dangerous if this is something that is to be deployed. (I always also try to rig such builds so that the build bombs if the dependencies change. That way it doesn't survive version changes without forcing someone to take a long hard look at it.)
kelnos 12y ago
I feel like a "freetard" "greybeard" "basement-dweller" (insert-your-own-pejorative-here) saying this, but: yet another reason why I will never ever base my livelihood on a closed ecosystem. Open source is certainly not a panacea, but needing to do something like this is just ridiculous.
- TazeTSchnitzel 12y ago
  The point of free software is you can do what you want with it, including fixing it if needs be. I believe Stallman started the movement from frustration that he couldn't fix a problem in some software he used.
- allegory 12y ago
  As the former opposite of the freetard and greybeard, a mere corporate VB peon, you are 100% spot on. Most of the commercial software dev stuff is like regularly salting and sandpapering your genitals. And I'm not talking about fine grit paper either. Even relatively popular fields like .Net are painful on a daily basis. I really like to see inside the black boxes when they inevitably go wrong. Hell getting a backtrace out of a dump file on windows from commercial software is an art in itself for example. I had the fortune of inheriting a Sun SparcStation 20 in 1999 that was being thrown out. That and NetBSD literally drove a spike through my mind and entirely destroyed my conception of proprietary closed source software. I really wouldn't touch it now but the money sorting out all the shitty little problems is pretty good, even if it tries to stab you in the face 5 times a day. There's lots of work as well unlike my preferred field of Unix and C. So I agree with you but I'm a slut for cash and unreliable black boxes of software is good money even if it does feel like I'm the IT equivalent of a STD ridden stripper.

332 points by antics 12y ago

rubyrescue 12y ago

My knowledge of the GAC is out of date but isn't a bit of a security hole that you can replace that DLL?

deleted 12y ago
[deleted]
tomasr 12y ago
You need to be a local administrator to write to the GAC by default. See http://msdn.microsoft.com/en-us/library/yf1d93sz(v=vs.110).aspx http://msdn.microsoft.com/en-us/library/yf1d93sz(v=vs.110).a... So if you can modify the files in the GAC, you're already compromised at that point.
- bmm6o 12y ago
  Writing to the GAC is easy, but assemblies in the GAC are strongly-named. Actually replacing one without the private key would require forging a signature, which is what parent was probably referring to. You can give an assembly the same name, but if it's signed with a different key it gets a different strong name (and therefore goes in a slightly different directory).
jameshart 12y ago
They didn't replace it - it looks like they created a new assembly with its own strong name, registered it alongside the 'official' assembly, and referenced their assembly in place of the .NET native assembly at compile time. What's maybe surprising is that the C# compiler generated code 'adopts' the types in their hacked expressions library. But the C# compiler, wherever it depends on particular types that need to exist for it to do its thing, has always generally relied on finding those types by -name-, not by -name and assembly strong name-. For example, the C# 3 compiler understands the calling and declaration syntax of extension methods, but to compile an extension method, it requires an Attribute type - System.Runtime.CompilerServices.ExtensionAttribute - which was only introduced into .NET in v3.5. But if you make an attribute class with that name available to the C# compiler, you can compile code using C#3 extension method syntax against the .NET 2.0 framework libraries.
overgard 12y ago
I don't think it's a replacement, since signing the new DLL would require a new key. (GAC DLL identity basically comes down to name + signing key. So even if you call it System.Whatever, it has a unique identity)

breischl 12y ago

Wow, that is some fancy work. Though I shudder to think about deploying and maintaining that fix, and bringing new developers up to speed on why and how it was done. I was also about to tell him to skip all the ildasm stuff and just use the online reference source (referencesource.microsoft.com) ... but that assembly isn't in there. So I guess ildasm was the best option.

skrebbel 12y ago

The real news here is that this is considered special. In most other languages, they'd have forked the source, fixed it, recompiled it for their own uses and submitted a pull request or patch. I'm a big .NET fan, but the fact that we have to jump through such hoops to find and fix a bug, and then still have near certainty that we're going to have to reapply the patch for many updates to come, well, that's just a bit sad. It feels rather last-century, to be honest. Microsoft could save a lot of double work if they'd just open source .NET and attach a decent process to it. They can still be the Linus. Just consider my patches.

toomuchtodo 12y ago
I work at a startup that uses almost all open source tools for our stack. To not be able to fork, fix, pull request a bug in software seems...barbaric to me.
antics 12y ago
---Author here--- I should disclose that I actually work for Microsoft, and still chose the hard way. Fuck the police!
- Alupis 12y ago
  > and still chose the hard way Chose? Unless you work on the team that made that DLL, it's likely you'd be jumping through hoops even internally.
  - wwwwwwwwww 12y ago
    i'm glad someone who works at the company like you was able to enlighten us as to microsoft's internal practices oh wait
    - Scuttles 12y ago
      If you pay any attention to Microsoft, their business practices, news, reports, blogs, etc... then you would know the parent comment is pretty much spot on.

DanielBMarkham 12y ago

Sidebar: I haven't done deep C# in several years, since moving to F#. This code is getting to look butt-ugly. It is not a good thing if it continues like this. We already have C++. Don't need another one.

_random_ 12y ago
Don't worry, each new version of C# gets more features from F# and Scala, this year we will get more syntax sugar and an extensible compiler (with proper IDE support - unlike F#): https://roslyn.codeplex.com/wikipage?title=Language%20Feature%20Status&referringTitle=Home https://roslyn.codeplex.com/wikipage?title=Language%20Featur... . They will add pattern matching and maybe something more interesting in the version after that. There aren't too many practical reasons to pick F# over C# today and soon there will be even less. Lesson to learn: don't pick *ML language as a foundation. I am glad that Scala is evading F#'s academic fate.
- codygman 12y ago
  F#'s academic fate? I'm curious as to why you think this. I'm guessing you are of the opinion Haskell is forever trapped in academia?
noblethrasher 12y ago
I do both C# and F#. But, at this point I think in ML no matter what which language I'm using. That said, though I'm delighted by F#, there are things that I really miss about C#: * Haskell's type classes are awesome, and neither F# nor C# have them. But, you can realize them with a really simple pattern in C# whereas in F# you have to resort to reflection or generic hacks[1] (but inline functions usually make such hacks nicer in F#). * Sum types are awesome. F# realizes them in two ways: with discriminated unions and pattern matching (mainly), or with abstract classes and inheritance. Both are powerful approaches, but the latter is particularly useful when you need unbounded sum types. Unfortunately, inheritance is somewhat crippled in F# because you can't define protected members (though you can override them). I find myself needing this when I'm designing a solution rather than merely implementing one. For instance, I find that designing a language/compiler is much easier with C# than with idiomatic F#. * This is kind of a summary of the first two points: idiomatic C# is a better dynamic language, and I'm not referring to the `dynamic` keyword, though it certainly helps. I've created some really nice and secure APIs by making use of user-defined conversions. System.Linq.Xml is a good example of what I'm talking about. * Finally, idiomatic C# is a nicer classical OO language than idiomatic F#. This is kind of obvious since F# is a "functional language" with OO features whereas C# is an OO language with functional features. But, when I say classical OO, I mean the Smalltalk style OO that came before C++. Alan Kay defined that kind of OOP as (1) Encapsulation, (2) Message Passing, and (3) extreme late binding. Neither C# nor F# quite has (2) and (3), but C# gets me the closest without coloring too far outside of the lines. Of course, this begs the question that OO is better than functional. Having done both for a long time, I'm convinced that it is the case (though I do think learning functional programming is the fastest path to getting good at OO). Lamentably, I can't yet defend this point too vigorously because I've only just discovered the joy or Smalltalk. But, here is a preview of sorts: Imagine being able to update a database schema and having your statically-typed code continue to run without needing to recompile. [1] http://stackoverflow.com/questions/9868327/f-type-constraints-and-overloading-resolution http://stackoverflow.com/questions/9868327/f-type-constraint...

colanderman 12y ago

I did this once with GCC. "But GCC's open source!" you say. Well, GCC is next-to-impossible to compile for a target other than the host, especially if the target isn't x86 or ARM; and GCC maintainers insist on precise test cases to vet a bug, even if the issue is immediately obvious from reading the source code and the bug only occurs in certain very complex situations. (/me looks forward to the day Clang/LLVM becomes the default on Linux…)

ikonst 12y ago
Just recently I wrote https://github.com/ikonst/LongPathFix https://github.com/ikonst/LongPathFix to work around a clang/gcc issue, cause it seemed easier than going through the hoops of building clang for Win32, understanding its code and writing a patch good enough to be accepted.
zvrba 12y ago
> Well, GCC is next-to-impossible to compile for a target other than the host, especially if the target isn't x86 or ARM; You're exaggerating. I had no problems compiling gcc 3.something targeting MIPS-I on an x86 Linux host.
- colanderman 12y ago
  I can't tell if you're being sarcastic. Generally, compiling GCC to target another architecture involves not only recreating most of the root directory structure (notably include files) of the target on the host, but also compiling libc, bintools, and a few other more obscure libraries. And this all needs to be done in the correct order (which involves something like compiling half of libc before compiling GCC, and compiling the rest of libc afterward). Maybe in 3.x and MIPS it "just worked". My experience is 4.x and Tilera. Myself and another engineer dumped a week into that sinkhole before giving up. Clang/LLVM on the other hand… ./configure && make && make install. No other wacky dependencies or build steps. And it generates better code in many cases (particularly when dealing with structures).
  - microcolonel 12y ago

nope_42 12y ago

Should have just used dotPeek instead of ilspy and writing IL code by hand. Recompiling would have certainly been easier. http://www.jetbrains.com/decompiler/ http://www.jetbrains.com/decompiler/

antics 12y ago
---Author here--- AWESOME TIP, stoked to try it out, thanks!
- nervousvarun 12y ago
  This kinda goes against a popular theory that Microsoft mine JetBrains for Visual Studio functionality improvement ideas (or more likely the blog author just doesn't work on VS or w/ the VS dev team all that much) Awesome read btw!

cornholio 12y ago

Dude, that's not a DLL, it's a pile of half-digested scripting language. Real men patch in assembly.

stusmith1977 12y ago

In situations like this, you can report the bug to Microsoft Connect: 1. Submit bug report. 2. Wait six months. 3. MS tech will post a comment, "this will be fixed in the next release". 4. Wait two more years. 5. Bug report will be closed as "won't fix".

allegory 12y ago
This. One bug in IE9 clickOnce launching thanks to them changing how download prompting works. 1. Reported to connect whilst in preview release status. Closed. Reported again. Closed. FULL test cases provided. 2. We're a gold partner with a £500k spend a year on licenses. Partner support. 19 hours on the phone over 6 months, blame shifting between the IE and .net teams and a daily call to get the case closed without resolution. Got a registry patch from ass-end support after 4 months that we have to ship to 2000 users at 200 different companies rather than an upstream fix. This checks a check box in the security settings. They broke their own product and won't fix it. Basically you can't use JS to redirect to a clickonce URL. Now today, IIS just stopped serving shit with no errors, nothing. Can't get anything out of minidumps+windbg. Just stops. None of our code is running. Who am I going to call? Redhat that's who.
- Pxtl 12y ago
  Yup. Its funny as a dot net dev you learn to really love a few ms things and really hate the others. C# is a brilliant language. Entity framework is an awesome ORM. But anything to do with deployment? IIS? So very much rage. I'm in charge of the build process on my team and I'm quickly becoming "that guy in the corner who swears all the time".
  - locusm 12y ago
    Your still not cursing as much as the guy building msi installers though...
    - allegory 12y ago
      Again, this. Even with WiX it's painful. HKLM vs HKCU + corporate deployment hell + why the fuck do I need two MSIs to bootstrap the CLR and VSTO. Argh. I deal with VSTO, WiX, ClickOnce, IIS, COM, MSMQ and the usual bits. Pays well but it made my hair fall out and has taken a couple of years off my life at least. I long for the gong to ring so I can go home to my MacBook and OpenBSD (where I truly belong).

batmansbelt 12y ago

Fuck bro, that's hard-core.

b0b0b0b 12y ago

Why wasn't the DLL signed? Is this not a thing?

Permit 12y ago
DLLs are signed. But they aren't checked every time a DLL is loaded. It absolutely obliterates load time if you have to go through every byte of all your DLLs and hash them. (You have the option to turn this on within the registry, though) Signing is almost useless in .Net. And it's certainly not in place for security purposes.
kevingadd 12y ago
Signing wouldn't have addressed this because they're creating a new, modified executable. So they just make that one unsigned.

euroclydon 12y ago

I'm sad to report that I tried to replicate what the author did here and it didn't work... I went to the weekly tech lead meeting, and when it was my turn to talk, I said: "I think we should increase out internal NuGet package release interval to every 18 hours. F--k the Police!" People just looked at me weird.

jmesserly 12y ago

Funny, I had something to do with this code back in the day! I'm guessing it was a copy+paste bug and they copied from the LambdaCompiler, which uses StrongBox<T> for its closed-over parameters[1], since StrongBox<T>.Value is a field. The idea was to have the closures be really fast. The history of ET compiler: it started with LINQ in .NET 3.5. Originally it was pretty simple and just handled expressions. In .NET 4.0 we merged the entire codebase with the IronPython/IronRuby compiler trees, expanding the "expression trees" to handle statements. IIRC, it can generate almost any IL construct that you might need, and is usually a lot easier to work with. But we found .NET's runtime compiler (DynamicMethod) was a bit too slow for a lot of use cases. It also wasn't supported on some CLR configurations. To address this we wrote an interpreter and some heuristics to switch from interpreted to compiled. But the actual System.Linq.Expressions.Interpreter must have happened after 4.0, because I don't remember that at all. Instead we just shipped it as a shared library used by IronPython and IronRuby. Here's the normal ExpressionQuoter: https://github.com/IronLanguages/main/blob/7be8b73e246bfb029880a7ebd8885cc541b705ac/Runtime/Microsoft.Scripting.Core/Compiler/ExpressionQuoter.cs#L152 https://github.com/IronLanguages/main/blob/7be8b73e246bfb029... And here was the interpreter. I don't see the ExpressionQuoter, so either that's a newer fork of the code that was rolled into System.Core, or maybe a completely new implementation. https://github.com/IronLanguages/main/tree/master/Runtime/Microsoft.Dynamic/Interpreter https://github.com/IronLanguages/main/tree/master/Runtime/Mi... IIRC, ExpressionQuoter was mainly to support the Quote expression, and was always a bit buggy. The 3.5 version was seriously messed up, and our prerelease versions of .NET 4.0 also had various bugs, and very few tests. I tried to fix it by having it use the same reuse closure mechanism as the normal compiler. Funny that same feature caused issues later on. [1] one might wonder: why use StrongBox<T>, essentially boxing every parameter, rather than just generating a type with only the right fields? The reason was that generating a type in .NET 4.0 timeframe was absurdly slow. Like, a few hundred per second slow. I think this has been largely fixed now, but it was a huge performance problem for Iron* language runtimes back in the day

CurtHagenlocher 12y ago
So, do you think this is Dino's fault? :P

swalsh 12y ago

Wow, how did I not know about this ExpressionBlock class! There's always another gem waiting in the framework :D I wish i knew about it about a year ago for a project I did.

Permit 12y ago

I know a lot of people here balk at the idea of paying for tooling, but Red Gate's Reflector[1] is absolutely amazing for situations like this. Not only is it a decompiler, but it allows you to decompile at debug-time and step into third party libraries. Assuming they haven't been obfuscated, this is an extremely useful tool. I've used it to track down a number of issues within Visual Studio itself and within some of the non-open sourced components of Roslyn. [1] http://www.red-gate.com/products/dotnet-development/reflector/ http://www.red-gate.com/products/dotnet-development/reflecto...

wffurr 12y ago
You are justifying closed-source tools by using a closed-source tool to debug your closed-source tools?
- allegory 12y ago
  That's exactly how the windows ecosystem works. Everything is pay per view. By the time we've got the PO's signed if it was open source then I'd have solved the problem at hand. To get around this, defensive purchasing is required: at least one VS ultimate license, at least one MSDN sub, ANTS profiler, IDA Pro, Azure VM allocation ready to roll for test machines and two guys who actually know something about all this. This is the safety belt cost: about $30000 + $120k a person
voltagex_ 12y ago
Don't pay for Reflector. Throw JetBrains some more money - DotPeek [1] integrates nicely with ReSharper, or you can use it on its own for free. I think the latest build can also provide PDBs to Visual Studio on the fly by acting as a source server (insane). After years of Reflector being a free tool, Redgate added a time-bomb to new downloads and released a statement [2] [1]: http://www.jetbrains.com/decompiler/ http://www.jetbrains.com/decompiler/ [2]: https://web.archive.org/web/20110205074826/http://www.red-gate.com/products/dotnet-development/reflector/announcement https://web.archive.org/web/20110205074826/http://www.red-ga...

jacquesm 12y ago

The fact that you can even do this without setting off a ton of alarmbells about failed checksums is what really scares me.

Permit 12y ago
How would you enforce that? First, hashing every byte of every loaded DLL destroys your runtime. Second, if you're able to modify one DLL, you could overwrite whatever DLL was tasked with computing the checksum.
Locke1689 12y ago
They recompiled with the new binaries.
reubenbond 12y ago
You can produce and consume signed libraries in .NET very easily. In practice it's becoming less common as .NET becomes more open. As an aside, OP says he works with Bart De Smet and they're messing with expression trees. I bet that this is work related to Cortana and the expression trees are used for compiling standing Rx (Reactive Extensions) queries which Cortana uses extensively on the phone and in the cloud.
userbinator 12y ago
The fact that people think you shouldn't be able to do this is what really scares me. :) I should be able to change the software on my own machine, stored on and running on hardware I own, in whatever way I desire and have it do what I want. (And in practice I have - opening a binary in a hex editor and changing a few bytes is not at all beyond me.)
- jacquesm 12y ago
  Of course you should be. But it still should generate warnings and force you to override some fairly impressive 'do you know what you are doing?' warnings.
  - 12y ago

x0n 12y ago

I did the same for the Invoke-RestMethod cmdlet in PowerShell 3.0: http://nivot.org/blog/post/2013/05/07/PowerShell-Masochistic-Assembly-Patching-for-Guts-and-Glory http://nivot.org/blog/post/2013/05/07/PowerShell-Masochistic...

xorcist 12y ago

I take issue with the authors use of the word "patch". He's not actually patching the DLL, just decompiling and recompiling which are quite familiar for most programmers. There might be some black magic associated with this particular DLL that we non-.net programmers doesn't understand. I would personally be very careful with the recompiling dance, at least in other languages, as alignments and such might come out of place. A patch feels much less dangerous if this is something that is to be deployed. (I always also try to rig such builds so that the build bombs if the dependencies change. That way it doesn't survive version changes without forcing someone to take a long hard look at it.)

kelnos 12y ago

I feel like a "freetard" "greybeard" "basement-dweller" (insert-your-own-pejorative-here) saying this, but: yet another reason why I will never ever base my livelihood on a closed ecosystem. Open source is certainly not a panacea, but needing to do something like this is just ridiculous.

TazeTSchnitzel 12y ago
The point of free software is you can do what you want with it, including fixing it if needs be. I believe Stallman started the movement from frustration that he couldn't fix a problem in some software he used.
allegory 12y ago
As the former opposite of the freetard and greybeard, a mere corporate VB peon, you are 100% spot on. Most of the commercial software dev stuff is like regularly salting and sandpapering your genitals. And I'm not talking about fine grit paper either. Even relatively popular fields like .Net are painful on a daily basis. I really like to see inside the black boxes when they inevitably go wrong. Hell getting a backtrace out of a dump file on windows from commercial software is an art in itself for example. I had the fortune of inheriting a Sun SparcStation 20 in 1999 that was being thrown out. That and NetBSD literally drove a spike through my mind and entirely destroyed my conception of proprietary closed source software. I really wouldn't touch it now but the money sorting out all the shitty little problems is pretty good, even if it tries to stab you in the face 5 times a day. There's lots of work as well unlike my preferred field of Unix and C. So I agree with you but I'm a slut for cash and unreliable black boxes of software is good money even if it does feel like I'm the IT equivalent of a STD ridden stripper.