Solution wide MSBuild target

Edit: Recent changes in MSBuild has enabled a new way to set solution wide msbuild logic for each project:
http://blog.seravy.com/directory-build-targets-solution-wide-msbuild-target-part-2/

The holy grail for simplifying build systems for me would be to attach a MSBuild targets file to a solution which would be imported into every project. In a corporate setting it has always been hard to educate developers to modify the project file to include custom targets when creating brand new projects of any kind (test, program, installer, etc).

Around 5 years ago I got excited by the article by Sayed Ibrahim Hashimi as it explained a technique to see how MSBuild itself creates a project from a .sln file and how to hook into it by setting the environment variable Set MSBuildEmitSolution=1. I thought this was finally a way to hook in all those custom build steps that we had accumulated over the years.

Looking into this technique further it seems to be a great for solution wide processes (Say running tests, running stylecop on the entire solution) but was too high level to modify assembly resolution or anything project related. I decided to still give it a go to see if I could import targets into a project from this target file.

Why?

All the companies I have worked at using MSBuild have often had a need to inject a level of business process into our projects. Some of these seem simple, others not so much. For example at my current place of employment we have the need to tweak a number of parts of the system.

  1. Set the $(OutDir) based on a number of conditions.
  2. Set variables such as wixtargets so that we can point to a designated directory, allowing a specific build to use a specific version of Wix.
  3. Modifying the ReferencePaths variable for assembly resolution
  4. Custom dependency retrieval logic
  5. Custom unit test runner logic
  6. Analyzers such as FXCop and Stylecop
  7. Nuget package building and project restore
  8. Custom build server logic such as:
    1. Strong Signing the delayed signed projects
    2. Digital Signatures
    3. Code coverage tools
    4. Updating assembly version numbers from build server

Looking into the generated project

I was on the look out for an <import> target which i could utilize, most of which originate from Microsoft.Common.Targets. I knew the import with the greatest success would be this line:

<Import Project=$(CustomBeforeMicrosoftCommonTargets) Condition=‘$(CustomBeforeMicrosoftCommonTargets)’ != ” and Exists(‘$(CustomBeforeMicrosoftCommonTargets)’)/>
This means if i could find any means to set the property ‘CustomBeforeMicrosoftCommonTargets’ then i could get my hook into each project build.

I have uploaded an example of this scenario to GitHub to follow along. If you go into Example1 directory and run Build.bat it will create the two diagnostic files:

  • MySolution.sln.metaproj.tmp – This file contains the template that used to build the target file (Build targets are empty and imports have not been brought in.)
  • MySolution.sln.metaproj – The .tmp file has been resolved with all imports and properties fleshed out so that msbuild can execute it.

Looking at the .metaproj file I could see that MSBuild was being called with a set of properties so I thought i could override the target name with my own version of build with a new set of properties. This unfortunately didn’t work as it will remove all targets with reserved names on import.

Then I used a technique I like to call Property Injection. Looking at the metaproj I could see that if i couldn’t change the MSBuild call for the project, i could modify one of the properties to include my property. In the after.MySolution.sln.targets file i include this property.

<SolutionPath>$(SolutionPath);CustomAfterMicrosoftCommonTargets=$(MSBuildThisFileDirectory)MySolutionProjects.targets</SolutionPath>

What this does is when it resolves the property for the build target:

<MSBuild Projects=”@(ProjectReference)” BuildInParallel=”True” Properties=”BuildingSolutionFile=true; CurrentSolutionConfigurationContents=$(CurrentSolutionConfigurationContents); SolutionDir=$(SolutionDir); SolutionExt=$(SolutionExt); SolutionFileName=$(SolutionFileName); SolutionName=$(SolutionName); SolutionPath=$(SolutionPath)” SkipNonexistentProjects=”%(ProjectReference.SkipNonexistentProjects)”>

it will actually include $(SolutionPath) AND $(CustomAfterMicrosoftCommonTargets)!

Now every build is being called will invoke the custom targets. If you clone the git and run build.bat in example1 you will see:


CustomProjectLogic:
Scope: Project. I performing an action per project from a solution include.
Done Building Project “C:\Dev\Repos\Blog\2015-08 – Solution wide MSBuild target\Example1\Project1\Project1.csproj” (default targets).

CustomProjectLogic:
Scope: Project. I performing an action per project from a solution include.
Done Building Project “C:\Dev\Repos\Blog\2015-08 – Solution wide MSBuild target\Example1\Project2\Project2.csproj” (default targets).

Why doesn’t this work?

This seems to achieve exactly what I want, with one exception. This does not work from within visual studio, which is kind of a killer.

As another solution if you add a targets that Microsoft.Common.Targets imports (in one of the designated folders) you can get it to import from the solution directory. You can see an example of the file here.

This works from within visual studio and MSBuild command line executions, but I do not want to go to every developers machine and install it. Plus if they don’t install it they could be checking in code which hasn’t gone through quality gates, slowing down the system.

What do I want?

Ultimately I really want the default Microsoft.Common.Targets to be changed to add the following line:
<Import Project=”$(SolutionPath).targets” Condition=”Exists(‘$(SolutionPath).targets’)”/>
or
<Import Project=”$(SolutionDir)Custom.$(MSBuildThisFile)” Condition=”Exists(‘$(SolutionDir)Custom.$(MSBuildThisFile)’)”/>

This can be placed after line 31 where it imports $(MSBuildProjectFullPath).user. The first allows each solution to have a different custom import, the second would allow any solutions in the directory to use the same import (This is preferable to me, you can filter targets condition based on the solution if needed.) The variables ‘$(SolutionPath)’ and ‘$(SolutionDir)’ are set either via MSBuild when targeting a solution (You can see it in the .metaproj file) or set by the Visual Studio IDE (Stubs can be seen in Microsoft.Common.targets).

Realistically the Microsoft.Common.Targets file should need updating because $(MSBuildProjectFullPath).user should now be checking the new VS2015 user folder of $(SolutionDir)\.vs folder.

What about nuget?

Nuget could actually have a lot of practical use with one further change too. If the import was also to be added to Microsoft.Common.Targets:

<Import Project=”$(MSBuildProjectDirectory)\.import\*” Condition=”Exists(‘$(MSBuildProjectDirectory)\.import’)”/>
<Import Project=”$(SolutionDir).import\*” Condition=”Exists(‘$(SolutionDir).import\’)”/>

This would allow any targets file to be included at a project or solution level and it will be imported into the project.

This solves one of the goals as listed on the blog under Part of the Platform > Goals.

Leave Project Files Alone

Instead of adding assembly references with Hint Paths into projects through the DTE, we want to leave project files alone. This would avoid the XML merge conflicts that arise far too often. Only the package manifest (packages.config) would be updated when a package is installed.

While it wouldn’t tackle the assembly resolution (which they want to be resolved directly from the packages.config/project.json file) it would still allow custom targets to be imported. An install would place the file there, uninstall would remove the file. With the package name being the target file there wont be conflicts, happy days.

User Voice

I have made a user voice post about this as a feature request. Please go and vote for it to help this move along!

https://visualstudio.uservoice.com/forums/121579-visual-studio/suggestions/9674760-add-a-custom-solution-import-in-microsoft-common-t

Update: There has been some progress on a git hub issue from the now open sourced Msbuild project: https://github.com/Microsoft/msbuild/issues/222

Should the change go through it means that while a solution targets wont exist, a file could be placed in the same directory with the same effect, as long as all projects are in folders within the solution direction. Something easily controlled if it is in source control so a win win!

Nuget Project Extensions

Nuget has slowly grown from a simple package dependency management of references to build processes and content. As a avid supporter of all things Nuget I have found myself yearning for more flexibility with the packages. One area I would like to test the feasibility of is the ability for nuget packages to contain project types. Nuget already handles the use of custom msbuild projects, this feature lets a project reference a custom build step. I have been playing around with this in another of my projects Doc.Net and it does not quite do what I want it to.

Getting every developer in an organisation to add the same extensions, installers, sdk and frameworks has always been a hassle. There is also the debate of what do you install onto a build server which has resulted in us often stripping project such as Wix and Code analysis away from their tools and reference target files in build directories checked into source control. I have often thought there could be some sort of system to allow all instances of visual studio on a domain to automatically get extensions but the logistics are just too great.

Project extensions have always been difficult to setup but recently the entry barrier has been lowered with the new VS Project System Extensibility: http://blogs.msdn.com/b/visualstudio/archive/2015/06/02/introducing-the-project-system-extensibility-sdk-preview.aspx

So the plan is to modify the visual studio nuget extension so that it can load project types directly from nuget packages. Nuget packages can be added to the solution level and the extension could look at these and provide them in the new project dialog. There are some limitations which may make this impossible such as needing to know the project type guids at startup and package restore not occurring until build. Possible solutions to these is to project a ‘Nuget’ level project type which is blank and populated by the msbuild include only. This would miss out on many features of project configuration but it is better than nothing.

The benefits of this system would be huge:

  • Allow 3rd party systems like WIX and Monogame to use a nuget package instead of an install.
  • No one needs to install a collection of vs extensions, installers, etc.
  • Allow side by side copies of projects, such as one project building with wix 3.8 and another with the pre-released 4.0.

So I will test the waters and see what is possible. Stay tuned.

Installing .Net 4.5 and NOT Windows SDK 8

I thought I could get away with installing .Net 4.5 without VS2012 or any of the SDK’s so that I could get a manual copy of code analysis working on our build server.
I was very wrong.
The very act of installing .Net 4.5 threw a number of errors to my previously working builds.

C:WindowsMicrosoft.NETFramework64v4.0.30319Microsoft.Common.targets(2836,5): error MSB3086: Task could not find “AL.exe” using the SdkToolsPath “” or the registry key “HKEY_LOCAL_MACHINESOFTWAREMicrosoftMicrosoft SDKsWindowsv8.0AWinSDK-NetFx40Tools-x86”. Make sure the SdkToolsPath is set and the tool exists in the correct processor specific location under the SdkToolsPath and that the Microsoft Windows SDK is installed [C:x.csproj]

and

C:x2.csproj(178,3): error MSB4019: The imported project “C:Program Files (x86)MSBuildMicrosoftVisualStudiov11.0WebApplicationsMicrosoft.WebApplication.targets” was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.

What I did find is if i browsed to the registry key:

ComputerHKEY_LOCAL_MACHINESOFTWAREMicrosoftMSBuildToolsVersion4.0

And renamed the key ‘11.0’ to ‘11.0-DoNotUseYet’. it seemed to revert back to windows SDK 7.0.

The worst part is it is running Windows Server 2008 which means I can’t install the 8.0 SDK anyway.

Wix’s $(var.Project.TargetDir) varibles actually uses the TargetPath variable and not TargetDir

Who would have thought!

I have been doing some serious modifications to our build libraries and I modified all sorts of parameters. I used to set the msbuild property <OutDir>C:lol</OutDir> before the call to main project imports, however with the fantastic introduction of MSBuild 4 now there is the ability for targets to specify ‘BeforeTargets’ and ‘AfterTargets’. This means no longer is it required to mess around with properties in the common targets (BeforeBuildDependsOn) that rudely does NOT care about what is set before it came along.

So I try and achieve the goal of my target being able to included anywhere in the project. This works mostly fine but setting OutDir after common means there are a few properties which are wrongly set such as TargetDir and Target Path.

<TargetDir Condition="'$(OutDir)' != ''">$([MSBuild]::Escape($([System.IO.Path]::GetFullPath(`$([System.IO.Path]::Combine(`$(MSBuildProjectDirectory)`, `$(OutDir)`))`))))</TargetDir>
<TargetPath Condition=" '$(TargetPath)' == '' ">$(TargetDir)$(TargetFileName)</TargetPath>

This meant my wix project was now pointing to the wrong location for the TargetDir and TargetPath variables. So I add in the code to also modify TargetDir when OutDir is changed and wix does not care.

On further invesigation i hardcoded TargetPath to ‘C:ILikeCheeselolrofl.proj’ and even though TargetDir was something sensible wix would still assume targetpath was C:ILikeCheese.

Realistically, TargetDir property is used so little that It should be on the way out yet is still used so critically in Wix and  in DesignTimeResolveAssemblyReferences .

Conclusion is that you can not trust the build system to be simple!

HttpHandler loading Web.UI.Page classes without .aspx files

Our work product is a bit unique in regard to how our pages are coded. Due to the high amount of inheritance in our pages we implemented all the pages in class files which inherit Web.UI.Page and just had .aspx page stubs which pointed the code behind to this class. This worked well until we hit the annoying issue of having to synchronize the aspx files between resources, and making sure they pointed to the right level down the inheritance chain. A messy solution was presented which was hard to maintain, so I did a bit of research and by using a HttpHandler I found we could solve our problem by keeping all pages inside the dll itself.

I have included the code for the concept below and a small sample of how its used.

Continue reading HttpHandler loading Web.UI.Page classes without .aspx files

HTTP Headers in Asp.net to alter the transfer of files

I was tinkering around with sending files across a web site at work and found this article by particularly useful.

http://blog.tylerholmes.com/2008/05/http-headers-content-type-and-content.html

It covers the setting of the Mime Type, changing the file name that comes across and forcing it to appear as a pop up.

Continue reading HTTP Headers in Asp.net to alter the transfer of files

Poor Coder Tools

Across the years you tend to accumulate a series of tools through day to day computer use. This of course starts when you have little or no money so feature rich applications which are free/unrestricted are an important part of your collection. I have summed up my collection of tools below and links to where you can get them. One of my favourite sources is the beloved source forge.

Continue reading Poor Coder Tools