5 ms·

Building .NET projects is a world of pain and here's how we should solve it

41 points by maartenba 12y ago

yvesgoeleven 12y ago
You're right, we need to fix the sdk's as well to tackle dll hell.
oblio 12y ago
This is something Microsoft needs to push instead of the insane MSI stuff. Microsoft really needs an officially sanctioned dependency manager, maybe NuGet. Something with a standardized package format based on _archives_, not _executable installers_. The Microsoft ecosystem is light years behind OSS in this regard. Heck, CPAN was designed in 1993 and became available in 1997. Truly light years - CPAN predates the dot-com boom.
- maartenba 12y ago
  The good CPAN times :-) It was a bit of a cumbersome thing to work with as a newbie back then but at least it handled dependencies very well.
  - lucian1900 12y ago
    In many ways CPAN is still state of the art. It has first-class support for namespacing and mirroring, which appears to still elude most environments' packaging systems. And I'm saying this as someone that dislikes Perl a lot.
    - luser 12y ago
      I did some research on CPAN - comparing it to Ruby gems and Python's various packaging solutions... CPAN is the defacto Perl standard, mirroring and testing built in. With perlbrew, localLib and cpanm, the tooling for Perl is world class. I think Ruby and Python are still catching up here. No doubt they will, but CPAN is an amazing feat of engineering.
mattmanser 12y ago
I'm not really sure what point you're trying to make. This seems to be exactly where MS are moving to with nuget. Like all the new stuff, the WebAPIs, EF 6, OData, etc. is now being published as NuGet packages instead of installers or SDKs or whatever. NuGet seems to be a direct response to this very issue and you constantly mention it. So what are you asking? After all they can't change the past!
- maartenba 12y ago
  That is not entirely true. For example the Windows Azure SDK has to be installed. Database tools have to be installed if you want to be able to compile a database project. There are many of the new things that do not follow this approach (hopefully: yet).
  - maartenba 12y ago
    Added a P.S. to the post to reflect this: P.S.: A lot of the new packages like ASP.NET MVC and WebApi, the OData packages and such are being shipped as NuGet packages which is awesome. The ones that I am missing are those that require additional build targets that are typically shiipped in SDK's. Examples are the Windows Azure SDK, database tools and targets, ... I would like those to come aboard the NuGet train!
    - ygra 12y ago
      There's nothing technical that would prevent nuget from including MSBuild targets that are then picked up after installing the package, right? (Worst case would be that you'd have to use a wrapper project which gets the dependencies and then builds the actual project, in case project includes are handled before the nuget target.)
      - maartenba 12y ago
        True that! And I hope more SDK's are shipped in NuGet packages so we can define explicit dependencies instead of implicit ones.
ndepoel 12y ago
NuGet gets the job done admirably, but it was simply introduced too late; nearly 10 years after the introduction of .NET. It's good to see NuGet has gained traction very quickly, but there are still plenty of projects predating NuGet that have solved the dependency management problem in their own unique way. Migrating all of those to NuGet takes time and willing maintainers.
- anton_gogolev 12y ago
  NuGet itself leaves a lot to be desired. For one, it's used to install all kinds of crap (like jQuery) and sometimes has all too tight dependency on Visual Studio in its .ps1 scripts. Next, all the mess with having assemblies for different versions of .NET Framework inside single nupkg -- I want to see the person who decided this will be a good idea. Third, why include version numbers in directory names? To make it harder to update dependencies? Fourth, dependencies. "[1.3.2,1.5)" - how cool is this? Fifth, the whole "Package Restore" concept is flawed.
  - chton 12y ago
    Why would installing something like jQuery be a bad fit for Nuget? It's a dependency like any other, just because it's in JS doesn't mean you shouldn't handle it the same way you would other packages. Versions in directory names make sense if you've got multiple projects depending on different versions of the same package. Upgrading everything at once isn't always an option. Could you elaborate why package restore is flawed? And do you have an alternative that doesn't involve putting all your dependencies in source control?
    - anton_gogolev 12y ago
      Because there's a ton of package managers for JavaScript and NuGet is reinventing the wheel here, squared. Granted, very few (if any) of them work on Windows, but that's outside the scope here. And even if we accept that it's OK for NuGet to install JS dependencies, where do we draw the line? Install Bootstrap with NuGet? FontAwesome? Should NuGet execute SQL scripts while installing Elmah? Install MSI packages as part of "install-package"? All in all, DLLs are the most dominant type of dependencies for .NET and NuGet should laser-focus on those. As for multiple projects within the same solution depending on different versions of packages, that's asking for trouble. And again, why default to the illogical behavior of including version numbers in directory names when this is only justified in like 0.1% of cases? Package Restore is _very_ fragile, both in terms "what happens if NuGet server is down" and in terms of "Visual Studio has gone mad and does weird shit trying to restore packages". And I saw you using "source control" implying that the only thing to belong there are text-only source files. What if we call it "version control"? This will magically allow us to put all the binary dependencies and live a happy life. Cheers!
MichaelGG 12y ago
>Let’s look at the Node.js community and how they manage to do things Node.js? The one that pulls in like 10,000 files to run a simple grunt build?
- maartenba 12y ago
  I said "most" projects :-) But even with that I don't have to go through my build logs 50 times before finding out which dependencies have to be installed for the build to work. The build should "just work".
- chton 12y ago
  It's not perfect, but at least you don't have to manually install 10,000 files just to be able to build in the first place. Or worry that your build server is running on the wrong version of Windows. Or even do anything special to get it to build. It's not perfect, but it's better than what we have now by a huge margin.
- thespace123 12y ago
  The benefit of npm is that everything is modular and patchable. To me it outweighs the 10,000 files issue.
anton_gogolev 12y ago
If you have a couple minutes, take a look inside any of the Microsoft-shipped .targets file to see the mess in its full glory. That's programming in XML, kids. It boggles my mind why Microsoft went the XML route when designing MSBuild (apparently they were copying NAnt and various other "enterprise-grade" build systems from the Java world), but it would be _so much_ better if they created a special DSL for this particular purpose. When I see this, I want to kill a puppy or two: Condition="'%(Identity)' != '@(SelectedFiles)'"
- paulio 12y ago
  I completely agree. Terrible stuff. It gets even worse with TFS and its new workflow system. In fact workflow in general is terrible and once again it's ' programming in XML'. Do MS dogfood this stuff at all?
- mgkimsal 12y ago
  Considering there's a move in the Java world to Gradle, a DSL approach, yeah... MS could have jumped to the head of the class by avoiding this XML years ago and put more forward-thinking effort in this area. But... this isn't a new criticism of MS. In most areas, they simply adapt to what the prevailing trend is.
jeswin 12y ago
Building .Net projects cannot really be solved in a way that compares well with nix systems. - Windows had no programmability until recently with Powershell. On Linux, I can script literally everything from database installs to component updates. - Powershell. Unfortunately they got their design wrong. Unix is a text and file based OS; you can script any server configuration with tools like awk. Windows apps doesn't have such a concept. To do stuff, you call... duh.. API!* - The above resulted in millions of programmers not really getting it. Need a component, alright an MSI file. They don't understand how programmability of the environment can vastly simplify the act of programming itself. - NuGet solves only a part of the problem. For non-trivial projects, there are tons of things that you need to automate besides library references. In short, Windows is horrible if you have used anything else. And that is by design gone wrong, somewhere in the 90s. * When they made the decision to build Powershell, they also had a chance to adopt bash or cygwin. I consider this a historic loss; the lack of a familiar/tasteful shell effectively shut (many of) the best programmers out of the Windows eco-system forever.
- wesnerm2 12y ago
  Even before PowerShell, Windows had thousands of COM objects in the system that can invoked within a scripting language like .vbs or any language that has APIs for accessing com objects like ActiveState perl. The Windows Explorer shell, Windows Media Player, not to mention Microsoft Office applications, have object models that are exposed and available to programmed against. I am not so sure that UNIX has anything better, since it seems all the functionality that UNIX has is also available in Windows. Then, there is also WMI (Windows Management Instrumentation) objects for all the low-level system operations.
  - Negitivefrags 12y ago
    I'd just like to add to this that *nix developers don't appreciate COM for what it really added to windows. In Linux you only generally call C functions from higher level languages (and sometimes C++). Python programmers use python libraries and sometimes C libraries. Ruby programmers use ruby libraries and sometimes C libraries. Never will you see someone calling in to a Ruby library from a Python program. On windows via COM you get Perl calling in to C++ calling in to C# calling in to VB.NET.
LoneWolf 12y ago
Instead of node I would say look at Maven, I never had a problem with a java project that used maven, all I ever needed was maven, and a JDK. It may not be the best out there but to me seems pretty good considering it is as simple as download source code, and run "mvn package".
- mateuszf 12y ago
  IMO maven in nowhere near in comparison with how npm works with modules.
cincinnatus 12y ago
It's just a different bag of hurt but no more so than node or ruby. Setting up ruby boxes makes me homicidal, whereas .net feels like nothing. But then I have much more experience with the latter, and 'evolved' with it and it's precursors. That said, building .net is relatively trivial and it's tools are caught up. I think what he's looking for is http://fsharp.github.io/FAKE/ http://fsharp.github.io/FAKE/
- lipoicacid 12y ago
  After 7 years of setting up ruby servers, I can't say it's ever made me homicidal. RVM and rbenv on multiple different distributions have generally all been fine. What did your distro/environment look like?
romanovcode 12y ago
> Let’s look at the Node.js community and how they manage to do things. I think NuGet is much better then npm. When you download something from NuGet you get a dll file that is injected in your References. When you download something with npm --save-dev you get 1000+ files. Just sayin'
gecko 12y ago
Not that I'm really against this article, but psake (https://github.com/psake/psake https://github.com/psake/psake) is a more realistic end-goal to handle the kind of flexibility you really need in projects. (It's accomplishable through MSBuild pre/post actions, but those are murder to edit anywhere but Visual Studio.) The problem with it, which is potentially a big one, is that psake uses PowerShell, and is therefore Windows-only. I'm hopeful that PowerShell can be ported to Unix soon as part of the general open-sourcing of Microsoft's .NET stack. I wouldn't use PowerShell as a replacement for bash, but as a cross-platform equivalent to Groovy/CRaSH, it'd be very handy for exactly this kind of thing. (Think of how Gradle fits into the Java world.)
- profquail 12y ago
  If you want to run on both Windows and UNIX-style systems, try FAKE (https://github.com/fsharp/FAKE https://github.com/fsharp/FAKE). It's rapidly becoming a popular choice for building all sorts of .NET projects. Scott Hanselman recently covered it on his blog: http://www.hanselman.com/blog/ExploringFAKEAnFBuildSystemForAllOfNET.aspx http://www.hanselman.com/blog/ExploringFAKEAnFBuildSystemFor...
  - jinushaun 12y ago
    Or use Rake with Albacore, a Windows/MSBuild plugin for Rake, and be done with it.
jinushaun 12y ago
The biggest offender for me, besides nuget being pretty much useless for real work, is that you need to install Visual Studio to automate headless ASP.NET builds! That's insane. Even the simplest most barebones ASP project requires it. As a result, our Chef scripts include downloading and installing a bunch of MSIs from S3. I would love to be proven wrong.
rat87 12y ago
http://earlyandoften.wordpress.com/2011/12/17/dependency-management-in-net/ http://earlyandoften.wordpress.com/2011/12/17/dependency-man...

41 points by maartenba 12y ago

You're right, we need to fix the sdk's as well to tackle dll hell.

oblio 12y ago

This is something Microsoft needs to push instead of the insane MSI stuff. Microsoft really needs an officially sanctioned dependency manager, maybe NuGet. Something with a standardized package format based on _archives_, not _executable installers_. The Microsoft ecosystem is light years behind OSS in this regard. Heck, CPAN was designed in 1993 and became available in 1997. Truly light years - CPAN predates the dot-com boom.

maartenba 12y ago
The good CPAN times :-) It was a bit of a cumbersome thing to work with as a newbie back then but at least it handled dependencies very well.
- lucian1900 12y ago
  In many ways CPAN is still state of the art. It has first-class support for namespacing and mirroring, which appears to still elude most environments' packaging systems. And I'm saying this as someone that dislikes Perl a lot.
  - luser 12y ago
    I did some research on CPAN - comparing it to Ruby gems and Python's various packaging solutions... CPAN is the defacto Perl standard, mirroring and testing built in. With perlbrew, localLib and cpanm, the tooling for Perl is world class. I think Ruby and Python are still catching up here. No doubt they will, but CPAN is an amazing feat of engineering.

mattmanser 12y ago

I'm not really sure what point you're trying to make. This seems to be exactly where MS are moving to with nuget. Like all the new stuff, the WebAPIs, EF 6, OData, etc. is now being published as NuGet packages instead of installers or SDKs or whatever. NuGet seems to be a direct response to this very issue and you constantly mention it. So what are you asking? After all they can't change the past!

maartenba 12y ago
That is not entirely true. For example the Windows Azure SDK has to be installed. Database tools have to be installed if you want to be able to compile a database project. There are many of the new things that do not follow this approach (hopefully: yet).
- maartenba 12y ago
  Added a P.S. to the post to reflect this: P.S.: A lot of the new packages like ASP.NET MVC and WebApi, the OData packages and such are being shipped as NuGet packages which is awesome. The ones that I am missing are those that require additional build targets that are typically shiipped in SDK's. Examples are the Windows Azure SDK, database tools and targets, ... I would like those to come aboard the NuGet train!
  - ygra 12y ago
    There's nothing technical that would prevent nuget from including MSBuild targets that are then picked up after installing the package, right? (Worst case would be that you'd have to use a wrapper project which gets the dependencies and then builds the actual project, in case project includes are handled before the nuget target.)
    - maartenba 12y ago
      True that! And I hope more SDK's are shipped in NuGet packages so we can define explicit dependencies instead of implicit ones.

ndepoel 12y ago

NuGet gets the job done admirably, but it was simply introduced too late; nearly 10 years after the introduction of .NET. It's good to see NuGet has gained traction very quickly, but there are still plenty of projects predating NuGet that have solved the dependency management problem in their own unique way. Migrating all of those to NuGet takes time and willing maintainers.

anton_gogolev 12y ago
NuGet itself leaves a lot to be desired. For one, it's used to install all kinds of crap (like jQuery) and sometimes has all too tight dependency on Visual Studio in its .ps1 scripts. Next, all the mess with having assemblies for different versions of .NET Framework inside single nupkg -- I want to see the person who decided this will be a good idea. Third, why include version numbers in directory names? To make it harder to update dependencies? Fourth, dependencies. "[1.3.2,1.5)" - how cool is this? Fifth, the whole "Package Restore" concept is flawed.
- chton 12y ago
  Why would installing something like jQuery be a bad fit for Nuget? It's a dependency like any other, just because it's in JS doesn't mean you shouldn't handle it the same way you would other packages. Versions in directory names make sense if you've got multiple projects depending on different versions of the same package. Upgrading everything at once isn't always an option. Could you elaborate why package restore is flawed? And do you have an alternative that doesn't involve putting all your dependencies in source control?
  - anton_gogolev 12y ago
    Because there's a ton of package managers for JavaScript and NuGet is reinventing the wheel here, squared. Granted, very few (if any) of them work on Windows, but that's outside the scope here. And even if we accept that it's OK for NuGet to install JS dependencies, where do we draw the line? Install Bootstrap with NuGet? FontAwesome? Should NuGet execute SQL scripts while installing Elmah? Install MSI packages as part of "install-package"? All in all, DLLs are the most dominant type of dependencies for .NET and NuGet should laser-focus on those. As for multiple projects within the same solution depending on different versions of packages, that's asking for trouble. And again, why default to the illogical behavior of including version numbers in directory names when this is only justified in like 0.1% of cases? Package Restore is _very_ fragile, both in terms "what happens if NuGet server is down" and in terms of "Visual Studio has gone mad and does weird shit trying to restore packages". And I saw you using "source control" implying that the only thing to belong there are text-only source files. What if we call it "version control"? This will magically allow us to put all the binary dependencies and live a happy life. Cheers!

MichaelGG 12y ago

>Let’s look at the Node.js community and how they manage to do things Node.js? The one that pulls in like 10,000 files to run a simple grunt build?

maartenba 12y ago
I said "most" projects :-) But even with that I don't have to go through my build logs 50 times before finding out which dependencies have to be installed for the build to work. The build should "just work".
chton 12y ago
It's not perfect, but at least you don't have to manually install 10,000 files just to be able to build in the first place. Or worry that your build server is running on the wrong version of Windows. Or even do anything special to get it to build. It's not perfect, but it's better than what we have now by a huge margin.
thespace123 12y ago
The benefit of npm is that everything is modular and patchable. To me it outweighs the 10,000 files issue.

anton_gogolev 12y ago

If you have a couple minutes, take a look inside any of the Microsoft-shipped .targets file to see the mess in its full glory. That's programming in XML, kids. It boggles my mind why Microsoft went the XML route when designing MSBuild (apparently they were copying NAnt and various other "enterprise-grade" build systems from the Java world), but it would be _so much_ better if they created a special DSL for this particular purpose. When I see this, I want to kill a puppy or two: Condition="'%(Identity)' != '@(SelectedFiles)'"

paulio 12y ago
I completely agree. Terrible stuff. It gets even worse with TFS and its new workflow system. In fact workflow in general is terrible and once again it's ' programming in XML'. Do MS dogfood this stuff at all?
mgkimsal 12y ago
Considering there's a move in the Java world to Gradle, a DSL approach, yeah... MS could have jumped to the head of the class by avoiding this XML years ago and put more forward-thinking effort in this area. But... this isn't a new criticism of MS. In most areas, they simply adapt to what the prevailing trend is.

jeswin 12y ago

Building .Net projects cannot really be solved in a way that compares well with nix systems. - Windows had no programmability until recently with Powershell. On Linux, I can script literally everything from database installs to component updates. - Powershell. Unfortunately they got their design wrong. Unix is a text and file based OS; you can script any server configuration with tools like awk. Windows apps doesn't have such a concept. To do stuff, you call... duh.. API!* - The above resulted in millions of programmers not really getting it. Need a component, alright an MSI file. They don't understand how programmability of the environment can vastly simplify the act of programming itself. - NuGet solves only a part of the problem. For non-trivial projects, there are tons of things that you need to automate besides library references. In short, Windows is horrible if you have used anything else. And that is by design gone wrong, somewhere in the 90s. * When they made the decision to build Powershell, they also had a chance to adopt bash or cygwin. I consider this a historic loss; the lack of a familiar/tasteful shell effectively shut (many of) the best programmers out of the Windows eco-system forever.

wesnerm2 12y ago
Even before PowerShell, Windows had thousands of COM objects in the system that can invoked within a scripting language like .vbs or any language that has APIs for accessing com objects like ActiveState perl. The Windows Explorer shell, Windows Media Player, not to mention Microsoft Office applications, have object models that are exposed and available to programmed against. I am not so sure that UNIX has anything better, since it seems all the functionality that UNIX has is also available in Windows. Then, there is also WMI (Windows Management Instrumentation) objects for all the low-level system operations.
- Negitivefrags 12y ago
  I'd just like to add to this that *nix developers don't appreciate COM for what it really added to windows. In Linux you only generally call C functions from higher level languages (and sometimes C++). Python programmers use python libraries and sometimes C libraries. Ruby programmers use ruby libraries and sometimes C libraries. Never will you see someone calling in to a Ruby library from a Python program. On windows via COM you get Perl calling in to C++ calling in to C# calling in to VB.NET.

LoneWolf 12y ago

Instead of node I would say look at Maven, I never had a problem with a java project that used maven, all I ever needed was maven, and a JDK. It may not be the best out there but to me seems pretty good considering it is as simple as download source code, and run "mvn package".

mateuszf 12y ago
IMO maven in nowhere near in comparison with how npm works with modules.

cincinnatus 12y ago

It's just a different bag of hurt but no more so than node or ruby. Setting up ruby boxes makes me homicidal, whereas .net feels like nothing. But then I have much more experience with the latter, and 'evolved' with it and it's precursors. That said, building .net is relatively trivial and it's tools are caught up. I think what he's looking for is http://fsharp.github.io/FAKE/ http://fsharp.github.io/FAKE/

lipoicacid 12y ago
After 7 years of setting up ruby servers, I can't say it's ever made me homicidal. RVM and rbenv on multiple different distributions have generally all been fine. What did your distro/environment look like?

romanovcode 12y ago

> Let’s look at the Node.js community and how they manage to do things. I think NuGet is much better then npm. When you download something from NuGet you get a dll file that is injected in your References. When you download something with npm --save-dev you get 1000+ files. Just sayin'

gecko 12y ago

Not that I'm really against this article, but psake (https://github.com/psake/psake https://github.com/psake/psake) is a more realistic end-goal to handle the kind of flexibility you really need in projects. (It's accomplishable through MSBuild pre/post actions, but those are murder to edit anywhere but Visual Studio.) The problem with it, which is potentially a big one, is that psake uses PowerShell, and is therefore Windows-only. I'm hopeful that PowerShell can be ported to Unix soon as part of the general open-sourcing of Microsoft's .NET stack. I wouldn't use PowerShell as a replacement for bash, but as a cross-platform equivalent to Groovy/CRaSH, it'd be very handy for exactly this kind of thing. (Think of how Gradle fits into the Java world.)

profquail 12y ago
If you want to run on both Windows and UNIX-style systems, try FAKE (https://github.com/fsharp/FAKE https://github.com/fsharp/FAKE). It's rapidly becoming a popular choice for building all sorts of .NET projects. Scott Hanselman recently covered it on his blog: http://www.hanselman.com/blog/ExploringFAKEAnFBuildSystemForAllOfNET.aspx http://www.hanselman.com/blog/ExploringFAKEAnFBuildSystemFor...
- jinushaun 12y ago
  Or use Rake with Albacore, a Windows/MSBuild plugin for Rake, and be done with it.

jinushaun 12y ago

The biggest offender for me, besides nuget being pretty much useless for real work, is that you need to install Visual Studio to automate headless ASP.NET builds! That's insane. Even the simplest most barebones ASP project requires it. As a result, our Chef scripts include downloading and installing a bunch of MSIs from S3. I would love to be proven wrong.

rat87 12y ago

http://earlyandoften.wordpress.com/2011/12/17/dependency-management-in-net/ http://earlyandoften.wordpress.com/2011/12/17/dependency-man...