Send Your Automated Build via Hightail (Formerly YouSendIt)

Here at InterKnowlogy we are always looking for ways to optimize our business. We’ve been using automated builds for sometime now. They are seriously one of the best things since sliced bread! Who doesn’t love to see a giant green checkmark stating their check-in succeeded. Or yelling names down the hallway when someone else causes a huge red ‘X’ to show up due to a failed check-in. As long as those names are aimed at you that is… We’ve been struggling with one problem lately with our release builds. If we have someone working offsite and they need to get the build after it completes they have to VPN into our network, go to the build directory, and copy the deliverable to their local machine. We do a lot of really cool graphically intense applications, which can mean large deliverables. This then turns into a really long difficult process to get a single deliverable. After a lot of discussions we come up with a really cool idea to use Hightail.

Hightail (Formerly YouSendIt)

We use Hightail for sending and receiving large files as I’m sure many of you do. We love the feature allowing us to specify an expiration date on a file so it will no longer take up space on our account. I did a lot of research on DropBox, Box, SkyDrive, and a few other options and each had pros and cons. In the end, Hightail was superior. It supports expiration dates, emails, and an easy API.

BuildSendService

We created a REST service that lives on our build server named the BuildSendService. This service is responsible for accepting a file path, expiration date, and details for sending the email with a link to the file. Information received is immediately added to a queue and which is then processed asynchronously. This allows the caller of the service to continue on to do more work instead of waiting for the file to be hightailed. The service asynchronously processes each item in the queue using the Hightail API, which is brain dead simple! It’s really great! Hightail takes care of the email for us once the file is uploaded. Recipients can then quickly download the file from their inbox! AWESOME!

SendBuild Custom Build Action

In order to leverage the BuildSendService most efficiently we built a Custom Build Action named SendBuild. In our Build Process Template, after the build has completed all other tasks and the final deliverable is ready for use in the final build directory, the SendBuild action contacts the BuildSendService to hightail the build to all desired recipients. It is not a mandatory action and is ignored if no recipients are specified. We wanted this action to be as fast as possible. That is why the BuildSendService accepts the parameters and queues the information immediately. This frees up our build server to process the next request immediately instead of waiting for a large deliverable to be uploaded to Hightail.

Conclusion

While this process is still new to us here at InterKnowlogy it is showing promise. Overtime, I’m sure we’ll tweak the current implementation to make it better and help use run more effectively and efficiently. Also, Hightail is just awesome! They have been super helpful answering all of our questions and pointing me in the right direction for development. They have a .NET sample application, which as far as I can tell implements each of their APIs. That was the best source of information. Their documentation is mostly good, but could use some extra explanations. If you want to do something similar for your build process let me know. I’m happy to help where I can.

Unit Test filtering for TFS builds using Test Explorer in VS 2012

One of the major new features in Visual Studio 2012 is the Text Explorer tool window, which consolidates 2010’s Test View and Test Results windows and adds support for using different testing frameworks together in one place. There are definitely some positive aspects to Test Explorer in comparison to its predecessors, but as a completely new piece of functionality it unfortunately left out some key features that were previously available.

One of the places it fell short was in filtering of tests to enable running a specific subset of the tests in your solution, especially when working with a set of tests set up for TFS’s Team Build. When working with small sets of tests it could be a minor annoyance, but working with hundreds or thousands of tests made it basically unusable. Thanks to Visual Studio’s new release model of frequent updates, these shortcomings are already starting to be addressed with November’s Update 1.

The preferred method of specifying tests to run with builds in TFS is by using attributes on test methods, specifically the Priority and TestCategory attributes. VS2010’s Test View displays a configurable grid listing all available tests in the open solution.
image

Continue reading

Migrating TFS Content to the Cloud

We are looking into using the Microsoft’s hosted Team Foundation Service, and the first question that always comes up is “I have an existing TFS project, how do I get my source (and work items) up there as a starting point?”  So I decided to look into it, and found the TFS Integration Platform written by the ALM Rangers at Microsoft.  MSDN Magazine has a good article to use for background and to get you started, but I had to do some custom tweaks to get mine to work.

First off – let’s set the goal:  migrate source code (including history) and Work Items from an on-premise TFS to the Microsoft hosted TFS.  I’m starting with a brand new Team Project in the hosted TFS, so if you merge code into an existing project or branch, your mileage may vary.

Second – we use a custom process template (modified from SCRUM) here at IK, and we are going to use the Agile process template online, so that adds a little complexity to the migration, as I have to map fields from the local process template to the hosted one.

Preparation

Create a work item query in the source TFS that includes the work items you want to migrate.  In my case, I have SPRINT types in the local TFS, but those don’t exist in the destination Agile process template, so my query includes only Product Backlog Items (that we’ll map to User Stories), Tasks, and Bugs.

Integration Tool

Fire up the tool, and click Create New. Select the “Team Foundation Server\VersionControlAndWorkItemTracking.xml” template.  This gives you all the configuration settings you need for migrating both source control and work items.

Configure the source (left side) and destination (right side) connection information for each of Version Control and Work Item Tracking.  I used VS 11 (TFS 2012) for both my local and hosted TFS connections.

Choosing Source and Destination

Source Control

By default, the source control session will migrate from the root of the source tree in the local repository to the root of the source tree in the remote one.  This caused a bunch of conflict errors for me, since even the newly created Team Project in the hosted TFS has some build template files in the BuildProcessTemplates folder.  Since the source and the destination both have the same files, there are conflicts during the merge.  You can work around them by specifying which version to use, but I avoid the hassle by filtering the source that’s migrated to a sibling tree that does not include the BuildProcessTemplates folder.  The tool seems to provide a way to cloak a directory such as the BuildProcessTemplates folder, but I could not get that to work.

So for both source and destination, I choose the source tree one level below the root.  Click the “…” button next to each side of the PATHS section to point to the correct source directory.  My config now looks like this:

Filter Source Paths

Work Items

Now we’re going to limit the migration to only the work items that are returned by the query we created above.  Click the “…” button next to the edit box in the “Queries” section and navigate to the query, mine is called “Migrate these Work Items” (again, it only includes Product Backlog Items, Tasks, and Bugs; not Sprints). 

Choose Filter Query

Notice when you come back from choosing the query, the actual CONTENT of the query is inserted into the edit box, instead of just a reference to the query.  I thought that was kind of interesting.

Custom Mapping

Finally, as I described above, I have to do some custom mappings to get the work items from our custom SCRUM template to the hosted TFS Agile template (we want to use some fields that are in Agile and not in Scrum in the hosted TFS, and process templates can’t be customized yet). 

Click the “Custom Settings” button, which gives you a dialog with the settings XML.  I chose to edit the XML in VS and then just paste it in here.  Here’s the top section of what mine looks like:

Custom Settings XML

Here is a zip that contains the settings I used.  You can use it as a starting point and then adjust as needed.  There is a section at the bottom that attempts to map users from the source to destination TFS systems, but I couldn’t get that to work either. 

That’s it – SAVE and START.  Watch the progress, and check for any conflicts. If you get some, you’ll have to resolve each issue independently.  It looks like the tool does a bunch of analysis and validation first, and will only proceed to the actual migration when there are no conflicts.   

Results

When it’s done, I go look at the source in the destination hosted TFS, and see the version history!  (Notice how the check-ins come through as the user that was running the tool, with some extra text to show when/how it was done.)

Results Source

Hope that helps.

Executing a custom TFS 2010 BuildActivity on the Controller

Most of the time when you create a custom Build Activity for TFS 2010 you want it to run on the Build Agent. Occasionally, you will come across a situation where you need your activity to execute as the very first thing before the build has even started or as the very last thing after the build is finalized. It’s very easy to configure your custom Build Activity to run on the controller. All Build Activities classes have the BuildActivity attribute. This attribute takes one argument, HostEnvironmentOption, which has four options: Agent, Controller, All, or None.

[BuildActivity(HostEnvironmentOption.All)]
public sealed class SendEmailActivity : CodeActivity
{

By setting the HostEnvironmentOption to All or Controller we can run our custom Build Activity on the build controller. The Agent option will restrict the activity to the build agent. Most of the time I use the All option unless I have a good reason not to.

If you don’t use the right HostEnvironmentOption you will get an error like the following:

TF215097: An error occurred while initializing a build for build definition \Internal\Test Build Definition: The build process failed validation. Details: Validation Error: The private implementation of activity ‘1: DynamicActivity’ has the following validation error: TF28000: Activity ‘SendEmailActivity’ must be used in the context of an AgentScope.

Visualizing Changes Across TFS Branches

Moving individual changes between different branches isn’t difficult but can quickly become a headache to keep track of when dealing with many branches and changesets. Fortunately Visual Studio 2010 has some great tools to track the path of changes across a set of TFS branches.

To begin we can look at the branch structure to get an idea of how everything is related. This view can be accessed from Source Control Explorer by clicking on a branch (MAIN here) and selecting Branching and Merging->View Hierarchy. From this we can see that there is a MAIN branch with Dev and Release branches off of it, each with their own sub-branches.

image

Now if I want to see the state of a changeset across branches I can right click it in the History and select Track Changeset. Continue reading

SD .NET Developer Group Presentation Resources

Here are the slides from my “Automating Builds with MSBuild and Team Foundation Server” presentation in 2 formats:

Small PDF (900K)
PowerPoint Show (3.3MB)

And the sample project from my MSBuild demos (look at the project files as XML)

Some other useful links:

Deploying .NET Applications at Amazon
MSBuild on MSDN – http://msdn2.microsoft.com/en-us/library/wea2sca5.aspx
Community Tasks – http://msbuildtasks.tigris.org/
MSBuild Sidekick – http://www.attrice.info/msbuild/

Team Build Lifecycle

EndToEndIteration target order with dependencies:

<>

BeforeEndToEndIteration

BuildNumberOverrideTarget

InitializeEndToEndIteration – Updates drop location

<>

BeforeClean

CoreClean – Deletes all old files

AfterClean

<>

Clean

<>

InitializeBuild – Creates SolutionRoot folder

<>

InitializeWorkspace – Delete and recreate workspace

BeforeGet

CoreGet – Gets all Team Project source

AfterGet

BeforeLabel

CoreLabel – Labels all files in workspace

AfterLabel

<>

PreBuild

<>

BeforeCompile

CoreCompile – Prep and build solutions

AfterCompile

<>

Compile

<>

GetChangeSetsAndUpdateWorkItems – Update reporting

<>

PostBuild

<>

BeforeTest

CoreTest – Run tests and save results

AfterTest

<>

Test

PackageBinaries

<>

TeamBuild

<>

BeforeDropBuild

<>

CoreDropBuild – Copy output and build log to drop location

CopyLogFiles – Copy other logs to drop location

AfterDropBuild

<>

DropBuild

AfterEndToEndIteration

<>

EndToEndIteration

New targets can be added through dependencies at any place marked with a <> by overriding xxxDependsOn properties, where xxx is the target marked in red immediately following the marker.

Targets marked in blue are empty targets designed to be overridden.