Fixing Missing NuGet Packages

I have encountered this problem more than one time. Somehow, maybe when merging changes between branches, there are references in .NET projects to assemblies in NuGet packages that are not present in packages.config, which in turn can result in issues when updating packages to later versions. I put together a PowerShell script to detect this issues. It searches all .csproj files in subfolders to the current one.

# Get reference hintpath to packages:
$pattern = "<HintPath>.*Packages\\(?<id>(?:[a-zA-Z]+\.?)+)\.(?<version>(?:\d+\.)+\d+)\\.*</HintPath>"

Get-ChildItem -Path . -Filter "*.csproj" -File -Recurse | foreach {
    $projFilename = $_.FullName
    Write-Output "`r`n*** $projFilename ***"
    $packageFilename = "$($_.DirectoryName)\packages.config"
    if ([System.IO.File]::Exists($packageFilename))
    {
        $projtext = Get-Content -Path $projFilename
        $projtext | foreach {
            if ($_ -imatch $pattern)
            {
                $id = $Matches.Item("id")
                $version = $Matches.Item("version")
                Write-Output "$($Matches[0])"
                $foundInPackages = Select-String -Pattern "id=""$id"" version=""$version""" -Path $packageFilename -Quiet
                if ($foundInPackages) {
                    Write-Output (Select-String -Pattern "id=""$id"" version=""$version""" -Path $packageFilename).Line
                } else {
                    Write-Warning "Not found in package file!"
                }
            }
        }
    } else {
        Write-Warning "There is no package file at $packageFilename!"
    }
}

Uninstalling NuGet Packages the Hard Way

We were recently in a situation where TeamCity had “cleaned” some NuGet packages that some of our projects depended on. Now they didn’t build in TeamCity. I thought this would be easy to fix: Just upgrade them to the latest version, or uninstall the old and install the new one. But bot of these methods failed, because the package manager failed to build the dependency graph, because it didn’t find the packages. So this was a catch 22!

It turned out that it was possible to manually remove references to these packages by editing the packages.config files, and then run Install-Package. I spent time to automate this, and came up with the following PowerShell script:

$pattern = 'id=”Neonstingray.Nettv4.DataTransfer” | id="Neonstingray.Nettv4.Domain" | id="Neonstingray.Nettv4.Configuration"’
Get-ChildItem -Filter packages.config -Recurse | Select-String -Pattern $pattern | Group Path | ForEach-Object { (Get-Content -Path $_.Name) | where { $_ -NotMatch $pattern } | Set-Content -Encoding UTF8 -Path $_.Name }
$source = "https://<myhost>/httpAuth/app/nuget/v1/FeedService.svc"
install-package Neonstingray.Nettv4.DataTransfer -projectname Valtech.Ready4Air.Ingest.UnitTests -source $source
install-package Neonstingray.Nettv4.DataTransfer -projectname Valtech.Ready4Air.Ingest.Ooyala -source $source
install-package Neonstingray.Nettv4.DataTransfer -projectname Valtech.Ready4Air.Ingest.Ooyala.IntegrationsTests -source $source
install-package Neonstingray.Nettv4.DataTransfer -projectname Valtech.Ready4Air.Ingest.Ooyala.UnitTests -source $source
install-package Neonstingray.Nettv4.DataTransfer -projectname Valtech.Ready4Air.Ingest.PartnerApi -source $source
install-package Neonstingray.Nettv4.DataTransfer -projectname Valtech.Ready4Air.Ingest.PartnerApi.IntegrationTests -source $source
install-package Neonstingray.Nettv4.DataTransfer -projectname Valtech.Ready4Air.Ingest.PartnerApi.UnitTests -source $source

Patching Assembly Version in TeamCity

We use TeamCity for building our .NET solutions and Octopus Deploy for deployment. We use semantic versioning using the <major>.<minor>.<patch>.<build> pattern, and I wanted to automatically set AssemblyInformationalVersion (a.k.a. product version) in all built assemblies. This was set to 2.0.2 in AssemblyInfo.cs, so I had to add the build number.

This was fairly easy using a build feature in TeamCity. Select the desired build configuration (I called it Build and Publish), and instead of going to build steps, click on build features in the left menu, and select the File Content Replacer type and begin by loading the AssemblyInformationalVersion in AssemblyInfo (C#) template. I then modified the search pattern to:

(^\s*\[\s*assembly\s*:\s*((System\s*\.)?\s*Reflection\s*\.)?\s*AssemblyInformationalVersion(Attribute)?\s*\(\s*@?\")(([0-9\*]+\.?)+)(\"\s*\)\s*\])

This will capture the following groups:

1: [assembly: AssemblyInformationalVersion(“
5: 2.0.2
6: 2
7: “)]

so the replacement is:

$1$5.\%build.number%$7

As AssemblyVersion, I wanted to stick with Microsoft’s standard <major>.<minor>.<build>.<revision>, and in the projects, this was:

[assembly: AssemblyVersion("2.0.*")]

I wanted to change that to

[assembly: AssemblyVersion("2.0.nnn.*")]

The search pattern in case is

(^\s*\[\s*assembly\s*:\s*((System\s*\.)?\s*Reflection\s*\.)?\s*AssemblyVersion(Attribute)?\s*\(\s*@?\")(([0-9\*]+\.)+)[0-9\*]+(\"\s*\)\s*\])

and the replacement

$1$5\%build.number%.*$7

Logging to Database with Log4net

I find it very useful to have all web applications and other executables within a solution log to a common table in the database, for monitoring and debugging purposes. This is fairly easy to accomplish with Log4net.

First, create the table:

CREATE TABLE [dbo].[Log4net] ( 
  [ID] [int] IDENTITY (1, 1) NOT NULL ,
  [Date] [datetime] NOT NULL ,
  [Hostname] [varchar] (255) NOT NULL ,
  [Thread] [varchar] (255) NOT NULL ,
  [Level] [varchar] (20) NOT NULL ,
  [Logger] [varchar] (255) NOT NULL ,
  [Message] [varchar] (8000) NOT NULL ,
  [Exception] [varchar] (8000) NULL
)

Here is an example Log4net appender configuration:

      <appender name="AdoNetAppender_SqlServer" type="log4net.Appender.AdoNetAppender">
        <connectionType value="System.Data.SqlClient.SqlConnection, System.Data, Version=1.0.3300.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
        <connectionString value="data source=foo;initial catalog=bar;integrated security=true" />
        <bufferSize value="1" />
        <UseTransactions value="false"/>
        <filter type="log4net.Filter.LevelRangeFilter">
          <levelMin value="INFO" />
        </filter>
        <commandText value="INSERT INTO Log4net ([Date],[Hostname],[Thread],[Level],[Logger],[Message],[Exception]) VALUES (@log_date, @hostname, @thread, @log_level, @logger, @message, @exception)" />
        <parameter>
          <parameterName value="@log_date" />
          <dbType value="DateTime" />
          <layout type="log4net.Layout.PatternLayout" value="%date{yyyy'-'MM'-'dd HH':'mm':'ss'.'fff}" />
          <!--<layout type="log4net.Layout.PatternLayout" value="%date" />-->
        </parameter>
        <parameter>
          <parameterName value="@hostname" />
          <dbType value="String" />
          <size value="255" />
          <layout type="log4net.Layout.PatternLayout" value="%property{log4net:HostName}" />
        </parameter>
        <parameter>
          <parameterName value="@thread" />
          <dbType value="String" />
          <size value="255" />
          <layout type="log4net.Layout.PatternLayout" value="%thread" />
        </parameter>
        <parameter>
          <parameterName value="@log_level" />
          <dbType value="String" />
          <size value="50" />
          <layout type="log4net.Layout.PatternLayout" value="%level" />
        </parameter>
        <parameter>
          <parameterName value="@logger" />
          <dbType value="String" />
          <size value="255" />
          <layout type="log4net.Layout.PatternLayout" value="%logger" />
        </parameter>
        <parameter>
          <parameterName value="@message" />
          <dbType value="String" />
          <size value="-1" />
          <layout type="log4net.Layout.PatternLayout" value="%message" />
        </parameter>
        <parameter>
          <parameterName value="@exception" />
          <dbType value="String" />
          <size value="-1" />
          <layout type="log4net.Layout.PatternLayout" value="%exception" />
        </parameter>
      </appender>

And here is how I write warnings and errors to both file and database:

      <root>
        <level value="WARN" />
        <appender-ref ref="ErrorFileAppender" />
        <appender-ref ref="AdoNetAppender_SqlServer" />
      </root>

For completeness, here is a pattern layout I usually use for file appenders:

<conversionPattern value="%date %-5level [%thread] %logger: %message%newline" /> 

A common problem with logging to the database is access rights. To debug such issues, it is convenient to turn on internal log4net logging. That is done with an application setting:

<configuration>
    <appSettings>
        <add key="log4net.Internal.Debug" value="true"/>
    </appSettings>
</configuration>

Internal debugging messages are written to the console and to System.Diagnostics.Trace. You can use the debugger to see these messages, or use DebugView from SysInternals.