Bit Queue Launched!

Today I launched an extension for Twitch that allows streamers to perform queue management!
User
Bit Queue allows you to create queues for viewers to join, requiring custom fields and bits to join. Useful for creative talents, games or anything that is worth something to your fans when you’re in demand!

Many streamers lack a way to be compensated for their interaction with viewers. Bit Queue lets you close the gap between viewers and broadcaster by allowing them to join your queue and see their position. Joining can cost a flat out bit fee or if demand outweighs supply let them put their bits where their mouth is!
The bit queue lets you customize the queues in many ways including:
– Custom fields to satisfy the queue
– Restrict to followers or subscribers
– A cost to join the queue
– Allowing a bidding war over position
– A fast pass to the top of the queue

Be rewarded for your talents and fame.

Check it out at https://www.twitch.tv/ext/49qu7wv3myvul5fa7fxazo1bcsnc8j

Directory.Build.Targets (Solution wide MSBuild target – Part 2)

As an update to http://blog.seravy.com/solution-wide-msbuild-target/ I am happy to report that there is a far easier way to add in custom logic to all common project types in visual studio 2017/MSBuild 15.
As a result of the issue https://github.com/Microsoft/msbuild/issues/222 there in common.targets (and so most project types) will attempt to import the file Directory.Build.targets and/or Directory.Build.props in the same folder and traverse up until it hits the root drive. This means provided your solution is in a parent directory of all your project files it will load a Directory.Build.targets or/and Directory.Build.props in the same directory as it.

If you had multiple solutions and wanted to include specific logic for each of them you could use the $(SolutionPath) variable to put conditional logic.

See the example on github and run the solution from visual studio 2017 or run the batch file.
https://github.com/Serivy/Blog/tree/master/2016-12%20-%20Directory.Build.Targets


1>------ Rebuild All started: Project: ClassLibrary, Configuration: Debug Any CPU ------
2>------ Rebuild All started: Project: ConsoleApp, Configuration: Debug Any CPU ------
2> ConsoleApp -> C:\Users\Seravy\Desktop\Directory.Build.Example\ConsoleApp\bin\Debug\ConsoleApp.exe
2> Scope: Project. I performing an action per project from a solution include. C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0 C:\Users\Seravy\AppData\Local\Microsoft\MSBuild\15.0
1> ClassLibrary -> C:\Users\Seravy\Desktop\Directory.Build.Example\ClassLibrary\bin\Debug\ClassLibrary.dll
1> Scope: Project. I performing an action per project from a solution include. C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0 C:\Users\Seravy\AppData\Local\Microsoft\MSBuild\15.0
========== Rebuild All: 2 succeeded, 0 failed, 0 skipped ==========

MSBuild download file task

I had a need to download a file in MSBuild and didn’t want to assume any tools on the server. I coded together a simple task which lets you specify the URL to download, and optionally a file name or/and a output path. The goal was to keep it as small as possible, thus the bad formatting and non existent error checking.

<DownloadFile Url=”http://ftp.iinet.net.au/welcome.txt”  />
<DownloadFile Url=”http://ftp.iinet.net.au/welcome.txt” OutputFolder=”D:\” File=”awesome.txt” />

https://gist.github.com/Serivy/cbf2bc3eedeecedcb5cffc2fe3e4206d

<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
   <UsingTask TaskName="DownloadFile" TaskFactory="CodeTaskFactory" AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v12.0.dll">
    <ParameterGroup>
      <Url ParameterType="System.String" Required="true" />
      <File ParameterType="System.String" Required="false" />
      <OutputFolder ParameterType="System.String" Required="false" />
    </ParameterGroup>
    <Task>
      <Using Namespace="System.Web"/>
      <Code Type="Fragment" Language="cs"><![CDATA[
        using (var client = new System.Net.WebClient())
            { client.DownloadFile(Url, (OutputFolder != null ? OutputFolder + "/" : "") + (File ?? System.IO.Path.GetFileName(new Uri(Url).LocalPath))); }
        ]]></Code>
    </Task>
  </UsingTask>
</Project>

JQuery Queue unleashed

If you need a good way to get queues in javascript/typescript, i have been playing around with jquery.queue. Its suppose to be for transitions and animations but has lately been opened up for whatever you want (sort of). I was looking at getting a third party queue because of my monstrosity in the check in process like http://benalman.com/code/projects/jquery-message-queuing/examples/ajax/ but i didn’t want to use a third party library. I decided to ignore the warnings of ‘its for animation only’ and see how i could use it.

// Queue can be stored in an empty object like this or any other element.
var queuedObject = $({});

// You may have a collection of many items, or a recursive call to deal with a number of items
// that you wish to proces in an orderly way.
var unknownArray = [‘one’, ‘two’, ‘three’, ‘four’];
$.each(unknownArray, (i, item) => {
// We can either queue them in loop/recursion or send the queued functions all through.
queuedObject.queue(“nameOfOperation”, (next: Function) => {
// Add the logic you want the queued item to do, this could include a deffered object.
alert(‘Message: ‘ + item);

// Calling the next function removes the item from the queue and kicks off the next one.
next();
});
});
// This starts the queue going. You can use queuedObject.clearQueue() or queuedObject.stop() to clear/pause during operation.
queuedObject.dequeue(“nameOfOperation”);

Restoring nuget packages on build servers or without Visual Studio

Nuget’s package restore is automated by the visual studio extension but what about when msbuild.exe runs against the .sln? When you execute the build without the packages you will end up with errors:

warning MSB3245: Could not resolve this reference. Could not locate the assembly

Fortunately we can create a file called ‘after.<SolutionName>.sln.targets’ replacing <SolutionName> with your soltion and putting it in in the folder with the solution and Nuget.exe.

Put the follow contents:

<?xml version=”1.0″ encoding=”utf-8″?>
<Project ToolsVersion=”14.0″ xmlns=”http://schemas.microsoft.com/developer/msbuild/2003″>
<Target Name=”RestoreNuget” BeforeTargets=”Build”>
<Exec Command=”&quot;Nuget.exe&quot; restore &quot;$(SolutionPath)&quot;” WorkingDirectory=”$(MSBuildProjectDirectory)”/>
</Target>
</Project>

Now when you run msbuild.exe ExampleApplication.sln

RestoreNuget:
“Nuget.exe” restore “C:\Dev\Repos\Blog\2015-11 – Solution Package Restore\ExampleApplication.sln”
Installing ‘Microsoft.Web.Xdt 2.1.1’.
Successfully installed ‘Microsoft.Web.Xdt 2.1.1’.
Installing ‘NuGet.Core 2.9.0’.
Successfully installed ‘NuGet.Core 2.9.0’.

You can also do things like Upgrade with the -safe flag for continuous integration situations. You can see the example of this on GitHub:

Solution wide MSBuild target

Edit: Recent changes in MSBuild has enabled a new way to set solution wide msbuild logic for each project:
http://blog.seravy.com/directory-build-targets-solution-wide-msbuild-target-part-2/

The holy grail for simplifying build systems for me would be to attach a MSBuild targets file to a solution which would be imported into every project. In a corporate setting it has always been hard to educate developers to modify the project file to include custom targets when creating brand new projects of any kind (test, program, installer, etc).

Around 5 years ago I got excited by the article by Sayed Ibrahim Hashimi as it explained a technique to see how MSBuild itself creates a project from a .sln file and how to hook into it by setting the environment variable Set MSBuildEmitSolution=1. I thought this was finally a way to hook in all those custom build steps that we had accumulated over the years.

Looking into this technique further it seems to be a great for solution wide processes (Say running tests, running stylecop on the entire solution) but was too high level to modify assembly resolution or anything project related. I decided to still give it a go to see if I could import targets into a project from this target file.

Why?

All the companies I have worked at using MSBuild have often had a need to inject a level of business process into our projects. Some of these seem simple, others not so much. For example at my current place of employment we have the need to tweak a number of parts of the system.

  1. Set the $(OutDir) based on a number of conditions.
  2. Set variables such as wixtargets so that we can point to a designated directory, allowing a specific build to use a specific version of Wix.
  3. Modifying the ReferencePaths variable for assembly resolution
  4. Custom dependency retrieval logic
  5. Custom unit test runner logic
  6. Analyzers such as FXCop and Stylecop
  7. Nuget package building and project restore
  8. Custom build server logic such as:
    1. Strong Signing the delayed signed projects
    2. Digital Signatures
    3. Code coverage tools
    4. Updating assembly version numbers from build server

Looking into the generated project

I was on the look out for an <import> target which i could utilize, most of which originate from Microsoft.Common.Targets. I knew the import with the greatest success would be this line:

<Import Project=“$(CustomBeforeMicrosoftCommonTargets)“ Condition=“‘$(CustomBeforeMicrosoftCommonTargets)’ != ” and Exists(‘$(CustomBeforeMicrosoftCommonTargets)’)“/>
This means if i could find any means to set the property ‘CustomBeforeMicrosoftCommonTargets’ then i could get my hook into each project build.

I have uploaded an example of this scenario to GitHub to follow along. If you go into Example1 directory and run Build.bat it will create the two diagnostic files:

  • MySolution.sln.metaproj.tmp – This file contains the template that used to build the target file (Build targets are empty and imports have not been brought in.)
  • MySolution.sln.metaproj – The .tmp file has been resolved with all imports and properties fleshed out so that msbuild can execute it.

Looking at the .metaproj file I could see that MSBuild was being called with a set of properties so I thought i could override the target name with my own version of build with a new set of properties. This unfortunately didn’t work as it will remove all targets with reserved names on import.

Then I used a technique I like to call Property Injection. Looking at the metaproj I could see that if i couldn’t change the MSBuild call for the project, i could modify one of the properties to include my property. In the after.MySolution.sln.targets file i include this property.

<SolutionPath>$(SolutionPath);CustomAfterMicrosoftCommonTargets=$(MSBuildThisFileDirectory)MySolutionProjects.targets</SolutionPath>

What this does is when it resolves the property for the build target:

<MSBuild Projects=”@(ProjectReference)” BuildInParallel=”True” Properties=”BuildingSolutionFile=true; CurrentSolutionConfigurationContents=$(CurrentSolutionConfigurationContents); SolutionDir=$(SolutionDir); SolutionExt=$(SolutionExt); SolutionFileName=$(SolutionFileName); SolutionName=$(SolutionName); SolutionPath=$(SolutionPath)” SkipNonexistentProjects=”%(ProjectReference.SkipNonexistentProjects)”>

it will actually include $(SolutionPath) AND $(CustomAfterMicrosoftCommonTargets)!

Now every build is being called will invoke the custom targets. If you clone the git and run build.bat in example1 you will see:

…
CustomProjectLogic:
Scope: Project. I performing an action per project from a solution include.
Done Building Project “C:\Dev\Repos\Blog\2015-08 – Solution wide MSBuild target\Example1\Project1\Project1.csproj” (default targets).
…
CustomProjectLogic:
Scope: Project. I performing an action per project from a solution include.
Done Building Project “C:\Dev\Repos\Blog\2015-08 – Solution wide MSBuild target\Example1\Project2\Project2.csproj” (default targets).
…

Why doesn’t this work?

This seems to achieve exactly what I want, with one exception. This does not work from within visual studio, which is kind of a killer.

As another solution if you add a targets that Microsoft.Common.Targets imports (in one of the designated folders) you can get it to import from the solution directory. You can see an example of the file here.

This works from within visual studio and MSBuild command line executions, but I do not want to go to every developers machine and install it. Plus if they don’t install it they could be checking in code which hasn’t gone through quality gates, slowing down the system.

What do I want?

Ultimately I really want the default Microsoft.Common.Targets to be changed to add the following line:
<Import Project=”$(SolutionPath).targets” Condition=”Exists(‘$(SolutionPath).targets’)”/>
or
<Import Project=”$(SolutionDir)Custom.$(MSBuildThisFile)” Condition=”Exists(‘$(SolutionDir)Custom.$(MSBuildThisFile)’)”/>

This can be placed after line 31 where it imports $(MSBuildProjectFullPath).user. The first allows each solution to have a different custom import, the second would allow any solutions in the directory to use the same import (This is preferable to me, you can filter targets condition based on the solution if needed.) The variables ‘$(SolutionPath)’ and ‘$(SolutionDir)’ are set either via MSBuild when targeting a solution (You can see it in the .metaproj file) or set by the Visual Studio IDE (Stubs can be seen in Microsoft.Common.targets).

Realistically the Microsoft.Common.Targets file should need updating because $(MSBuildProjectFullPath).user should now be checking the new VS2015 user folder of $(SolutionDir)\.vs folder.

What about nuget?

Nuget could actually have a lot of practical use with one further change too. If the import was also to be added to Microsoft.Common.Targets:

<Import Project=”$(MSBuildProjectDirectory)\.import\*” Condition=”Exists(‘$(MSBuildProjectDirectory)\.import’)”/>
<Import Project=”$(SolutionDir).import\*” Condition=”Exists(‘$(SolutionDir).import\’)”/>

This would allow any targets file to be included at a project or solution level and it will be imported into the project.

This solves one of the goals as listed on the blog under Part of the Platform > Goals.

Leave Project Files Alone

Instead of adding assembly references with Hint Paths into projects through the DTE, we want to leave project files alone. This would avoid the XML merge conflicts that arise far too often. Only the package manifest (packages.config) would be updated when a package is installed.

While it wouldn’t tackle the assembly resolution (which they want to be resolved directly from the packages.config/project.json file) it would still allow custom targets to be imported. An install would place the file there, uninstall would remove the file. With the package name being the target file there wont be conflicts, happy days.

User Voice

I have made a user voice post about this as a feature request. Please go and vote for it to help this move along!

https://visualstudio.uservoice.com/forums/121579-visual-studio/suggestions/9674760-add-a-custom-solution-import-in-microsoft-common-t

Update: There has been some progress on a git hub issue from the now open sourced Msbuild project: https://github.com/Microsoft/msbuild/issues/222

Should the change go through it means that while a solution targets wont exist, a file could be placed in the same directory with the same effect, as long as all projects are in folders within the solution direction. Something easily controlled if it is in source control so a win win!

Nuget Project Extensions

Nuget has slowly grown from a simple package dependency management of references to build processes and content. As a avid supporter of all things Nuget I have found myself yearning for more flexibility with the packages. One area I would like to test the feasibility of is the ability for nuget packages to contain project types. Nuget already handles the use of custom msbuild projects, this feature lets a project reference a custom build step. I have been playing around with this in another of my projects Doc.Net and it does not quite do what I want it to.

Getting every developer in an organisation to add the same extensions, installers, sdk and frameworks has always been a hassle. There is also the debate of what do you install onto a build server which has resulted in us often stripping project such as Wix and Code analysis away from their tools and reference target files in build directories checked into source control. I have often thought there could be some sort of system to allow all instances of visual studio on a domain to automatically get extensions but the logistics are just too great.

Project extensions have always been difficult to setup but recently the entry barrier has been lowered with the new VS Project System Extensibility: http://blogs.msdn.com/b/visualstudio/archive/2015/06/02/introducing-the-project-system-extensibility-sdk-preview.aspx

So the plan is to modify the visual studio nuget extension so that it can load project types directly from nuget packages. Nuget packages can be added to the solution level and the extension could look at these and provide them in the new project dialog. There are some limitations which may make this impossible such as needing to know the project type guids at startup and package restore not occurring until build. Possible solutions to these is to project a ‘Nuget’ level project type which is blank and populated by the msbuild include only. This would miss out on many features of project configuration but it is better than nothing.

The benefits of this system would be huge:

  • Allow 3rd party systems like WIX and Monogame to use a nuget package instead of an install.
  • No one needs to install a collection of vs extensions, installers, etc.
  • Allow side by side copies of projects, such as one project building with wix 3.8 and another with the pre-released 4.0.

So I will test the waters and see what is possible. Stay tuned.

Installing .Net 4.5 and NOT Windows SDK 8

I thought I could get away with installing .Net 4.5 without VS2012 or any of the SDK’s so that I could get a manual copy of code analysis working on our build server.
I was very wrong.
The very act of installing .Net 4.5 threw a number of errors to my previously working builds.

C:WindowsMicrosoft.NETFramework64v4.0.30319Microsoft.Common.targets(2836,5): error MSB3086: Task could not find “AL.exe” using the SdkToolsPath “” or the registry key “HKEY_LOCAL_MACHINESOFTWAREMicrosoftMicrosoft SDKsWindowsv8.0AWinSDK-NetFx40Tools-x86”. Make sure the SdkToolsPath is set and the tool exists in the correct processor specific location under the SdkToolsPath and that the Microsoft Windows SDK is installed [C:x.csproj]

and

C:x2.csproj(178,3): error MSB4019: The imported project “C:Program Files (x86)MSBuildMicrosoftVisualStudiov11.0WebApplicationsMicrosoft.WebApplication.targets” was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.

What I did find is if i browsed to the registry key:

ComputerHKEY_LOCAL_MACHINESOFTWAREMicrosoftMSBuildToolsVersion4.0

And renamed the key ‘11.0’ to ‘11.0-DoNotUseYet’. it seemed to revert back to windows SDK 7.0.

The worst part is it is running Windows Server 2008 which means I can’t install the 8.0 SDK anyway.