TeamCity and Octopus Deploy Tips and Tricks

Setting Version

It is nice to have TeamCity set build number. I tend to use major.minor.build.revision for AssemblyVersion and major.minor.revision.build for AssemblyInformationalVersion (product version). So in AssemblyInfo.cs we have for example:

[assembly: AssemblyVersion("2.2.*")] // AssemblyFileVersionAttribute is not supplied, so the AssemblyVersionAttribute is used for the Win32 file version that is displayed on the Version tab of the Windows file properties dialog.
[assembly: AssemblyInformationalVersion("2.2.1")] // A.k.a. product version

In order to have the build number inserted, create two file content replacer build features with the following configurations:

Path pattern: “**/AssemblyInfoGlobal*.cs”
File encoding: <Auto-detect>
Search for: (^\s*\[\s*assembly\s*:\s*((System\s*\.)?\s*Reflection\s*\.)?\s*AssemblyVersion(Attribute)?\s*\(\s*@?\”)(([0-9\*]+\.)+)[0-9\*]+(\”\s*\)\s*\])
Match case: true
Replace with: $1$5\%build.number%.*$7

Path pattern: “**/AssemblyInfoGlobal*.cs”
File encoding: <Auto-detect>
Search for: (^\s*\[\s*assembly\s*:\s*((System\s*\.)?\s*Reflection\s*\.)?\s*AssemblyInformationalVersion(Attribute)?\s*\(\s*@?\”)(([0-9\*]+\.?)+)(\”\s*\)\s*\])
Match case: true
Replace with: $1$5.\%build.number%$7

Build and Publish Multiple Branches

I usually want some flexibility in what branch to build, and I have found that the following settings work quite well.

To make it possible to build and publish multiple branches, you can have wildcards in the VCS Root Branch specification, e.g.:

+:refs/heads/release/*
+:refs/heads/develop
+:refs/heads/feature/*

The default branch in this case would probably be set to

refs/heads/develop

Now, you probably only want automatic build and publish for certain branches, not all. To have automatic build for just develop, go to the build configuration Triggers setting and set e.g. the following branch filter:

+:<default>

Now, you can click the ellipses next to Run, go to the Changes tab and select the desired  branch to build.

Build Pull Requests

You probably also want to have automatic build (but not publish) of pull requests. This is accomplished by having a separate build configuration for that, with the following VCS Root Branch specification:

+:refs/pull/(*/merge)
-:develop
-:master

This means pull requests are built but not the develop and master branches. (For example, pull request #1 have branch name refs/pull/1/merge.

Deploy a Specific Version

The default behavior of the OctopusDeploy: Create release step is to create a release of the latest version. If you want to build and deploy another version, probably from a release branch, you can do like this:

  1. Create a new VCS Root with default branch set to e.g. refs/heads/release/2.2 and use this in your build configuration.
  2. In General Settings, set Build Number Format to e.g. 2.2.1.%build.counter%.
  3. In your Deploy/create release step, set Release number to %build.number%, and Additional command line arguments to –packageversion=%build.number%. This will make octo.exe use this version as default for every package. You can override that with the package parameter, e.g. –packageversion=%build.number% –package=EntityFramework:1.6.2.

Note: It would be better to read the version from AssemblyInfo.cs rather than hard-configuring it, but I haven’t tried that out yet. It would require some scripting.

NuGet Publish

In a TeamCity NuGet Publish step, instead of specifying packages one by one, you can use:

**\obj\octopacked\*.nupkg

Integration Tests

I prefer to run integration tests as part of deployment rather than build, for two reasons:

  • It takes quite some time to run them, and I don’t like really long builds.
  • To a large extent, integration tests test configuration, so it make sense to run them on the target environment rather than on the build server.

Here are the steps I have used to facilitate integration testing as part of deployment. In the integration test project:

  • Add the OctoPack NuGet package.
  • Add app.config transforms. They must be called <project>.IntegrationTests.dll.<environment>.config, build action should be None and copy to output directory should be Copy if newer.
  • Add a PostDeploy.ps1 script. This could look like the example below. Make sure it has the same properties as the ones in previous step.

In TeamCity:

  • Add the integration project to NuGet publish step, or use wildcard as described above.

In Octopus Deploy:

  • Add a new Deploy a NuGet package step and choose the integration test package.

Example PostDeploy.ps1

# Clean-up
Remove-Item Project.IntegrationTests.dll.*.config

# Run integration tests
choco upgrade visualstudio2015testagents -y

$exePath = "C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\MSTest.exe"
$testBinariesFolder = "."
$testBinariesFilter = "*.IntegrationTests.dll"
$scheme = $OctopusParameters['scheme']
$hostname = $OctopusParameters['hostname']
$webAppPath = $scheme + "://" + $hostname 
$webAppPhysicalPath = $OctopusParameters['Octopus.Action[Deploy Web].Output.Package.InstallationDirectoryPath']

# Search for integration test DLLs
$testDlls = ""
(Get-ChildItem -Path $testBinariesFolder -Filter $testBinariesFilter).FullName | ForEach-Object { $testDlls += "/testcontainer:""$_"" " }

# Exclude some categories.
$environment = $OctopusParameters['Octopus.Environment.Name']
if ($environment -match "xxx") { $categories = '/category:"CategoryX"' }
if ($environment -match "yyy") { $categories = '/category:"CategoryY"' }
if ($environment -match "zzz") { $categories = '/category:"CategoryZ"' }
if ($categories -eq $null -or $categories -eq "") {
    Write-Error "Unknown environment ""$environment"". Integration tests will not be run." -ErrorAction Continue
    return
}

# Start the test
Write-Output "& ""$exePath"" $testDlls $categories"
Invoke-Expression "& ""$exePath"" $testDlls $categories"

# Check results
if ($LASTEXITCODE -eq 0) {
    Write-Output "All integration tests passed."
} else {
    Write-Error "One or more integration tests failed." -ErrorAction Continue
}

# Upload result file
$testResultFiles = Get-ChildItem -Path .\TestResults -Filter *.trx -ErrorAction SilentlyContinue
if ($testResultFiles -ne $null)
{
	$resultMessage = "Test results available at "
	$testResultFiles | ForEach-Object {
		Move-Item $_.FullName $webAppPhysicalPath
		$resultMessage += "$webAppPath/$($_.Name) "
	}
	Write-Output $resultMessage

	# Allow download of the result
	if ((Get-WebConfiguration -Filter "//staticContent/mimeMap[@fileExtension='.trx']" -PSPath IIS:\) -eq $null)
	{
		Add-WebConfiguration -Filter "//staticContent" -PSPath IIS:\ -Value @{fileExtension=".trx";mimeType="application/x-test"}
	}
} else {
	Write-Error "Found no test results." -ErrorAction Continue
}

# Always return 0 because we don't want to fail the deployment until integration tests are stable.
exit 0

There are several things to note here.

  • To do…
Advertisements

Fixing Missing NuGet Packages

I have encountered this problem more than one time. Somehow, maybe when merging changes between branches, there are references in .NET projects to assemblies in NuGet packages that are not present in packages.config, which in turn can result in issues when updating packages to later versions. I put together a PowerShell script to detect this issues. It searches all .csproj files in subfolders to the current one.

# Get reference hintpath to packages:
$pattern = "<HintPath>.*Packages\\(?<id>(?:[a-zA-Z]+\.?)+)\.(?<version>(?:\d+\.)+\d+)\\.*</HintPath>"

Get-ChildItem -Path . -Filter "*.csproj" -File -Recurse | foreach {
    $projFilename = $_.FullName
    Write-Output "`r`n*** $projFilename ***"
    $packageFilename = "$($_.DirectoryName)\packages.config"
    if ([System.IO.File]::Exists($packageFilename))
    {
        $projtext = Get-Content -Path $projFilename
        $projtext | foreach {
            if ($_ -imatch $pattern)
            {
                $id = $Matches.Item("id")
                $version = $Matches.Item("version")
                Write-Output "$($Matches[0])"
                $foundInPackages = Select-String -Pattern "id=""$id"" version=""$version""" -Path $packageFilename -Quiet
                if ($foundInPackages) {
                    Write-Output (Select-String -Pattern "id=""$id"" version=""$version""" -Path $packageFilename).Line
                } else {
                    Write-Warning "Not found in package file!"
                }
            }
        }
    } else {
        Write-Warning "There is no package file at $packageFilename!"
    }
}

Unit Testing ASP.NET Web API

This is one of those things that I encounter now and then and always forgot how to do. I have to search the web and most of the results are for ASP.NET MVC, not for Web API. This time I found the solution on http://www.peterprovost.org/blog/2012/06/16/unit-testing-asp-dot-net-web-api/ and decided to document it.

var config = new HttpConfiguration();
var request = new HttpRequestMessage(HttpMethod.Get, "http://host/api/my");
var route = config.Routes.MapHttpRoute("DefaultApi", "api/{controller}/{id}");
var routeData = new HttpRouteData(route, new HttpRouteValueDictionary { { "controller", "my" } });
var controller = new MyController();
controller.ControllerContext = new HttpControllerContext(config, routeData, request);
controller.Request = request;
controller.Request.Properties[HttpPropertyKeys.HttpConfigurationKey] = config;

I’m not sure you need the route stuff, but the config is needed.

Another variant of this is constructing an HttpRequestMessage that be passed to the UrlHelpeconstructor:

var config = new HttpConfiguration();
var request = new HttpRequestMessage(HttpMethod.Get, "http://host/2.2/api/se/15/controller");
var route = config.Routes.MapHttpRoute(
            name: RoutesConstants.IncludeResourceSpecificOptionsRouteName,
            routeTemplate: "api/{country}/{deviceVersion}/{controller}/{id}",
            defaults:
                new
                {
                    id = RouteParameter.Optional,
                    country = RouteParameter.Optional,
                    deviceVersion = RouteParameter.Optional
                });
request.Properties[HttpPropertyKeys.HttpConfigurationKey] = config;
var urlHelper = new UrlHelper(request);

Another related problem is if the controller accesses HttpContext.Current.Request. That you can just assign:

HttpContext.Current = new HttpContext(new HttpRequest(null, "http://someurl", null), new HttpResponse(new StringWriter()));

Uninstalling NuGet Packages the Hard Way

We were recently in a situation where TeamCity had “cleaned” some NuGet packages that some of our projects depended on. Now they didn’t build in TeamCity. I thought this would be easy to fix: Just upgrade them to the latest version, or uninstall the old and install the new one. But bot of these methods failed, because the package manager failed to build the dependency graph, because it didn’t find the packages. So this was a catch 22!

It turned out that it was possible to manually remove references to these packages by editing the packages.config files, and then run Install-Package. I spent time to automate this, and came up with the following PowerShell script:

$pattern = 'id=”Neonstingray.Nettv4.DataTransfer” | id="Neonstingray.Nettv4.Domain" | id="Neonstingray.Nettv4.Configuration"’
Get-ChildItem -Filter packages.config -Recurse | Select-String -Pattern $pattern | Group Path | ForEach-Object { (Get-Content -Path $_.Name) | where { $_ -NotMatch $pattern } | Set-Content -Encoding UTF8 -Path $_.Name }
$source = "https://<myhost>/httpAuth/app/nuget/v1/FeedService.svc"
install-package Neonstingray.Nettv4.DataTransfer -projectname Valtech.Ready4Air.Ingest.UnitTests -source $source
install-package Neonstingray.Nettv4.DataTransfer -projectname Valtech.Ready4Air.Ingest.Ooyala -source $source
install-package Neonstingray.Nettv4.DataTransfer -projectname Valtech.Ready4Air.Ingest.Ooyala.IntegrationsTests -source $source
install-package Neonstingray.Nettv4.DataTransfer -projectname Valtech.Ready4Air.Ingest.Ooyala.UnitTests -source $source
install-package Neonstingray.Nettv4.DataTransfer -projectname Valtech.Ready4Air.Ingest.PartnerApi -source $source
install-package Neonstingray.Nettv4.DataTransfer -projectname Valtech.Ready4Air.Ingest.PartnerApi.IntegrationTests -source $source
install-package Neonstingray.Nettv4.DataTransfer -projectname Valtech.Ready4Air.Ingest.PartnerApi.UnitTests -source $source

Troubleshooting Memory Leaks in .NET

I recently had problems with a memory leak in an ASP.NET application. I learned some useful tools thanks to Fredrik Haglund.

The first one is “handle”, which is part of the Sysinternals suite. With

handle -s -p <pid>

you can display the number of handles for a given process. This could be useful for applications that e.g. open files but do not close them.

The second one is “procdump”, which is also part of the Sysinternals suite. This will dump the entire process by first creating a copy:

procdump64 -ma -r <pid>

This dump can then be analyzed using the Debug Diagnostics Tool from Microsoft.