Tips and Tricks for NancyFx and .NET Core

In my current assignment, we’re using Nancy on ASP.NET Core and .NET Core 2.0. Nancy for .NET Core is currently in alpha, and since documentation is lacking, and since this is not exactly the “super-duper-happy-path”, I thought I would write down some discoveries.

Application Settings

With WebAPI, you define an options class, in my case ConnectionStrings, and add this to the Startup class:

public void ConfigureServices(IServiceCollection services)
{
    // ...
    services.Configure(Configuration.GetSection("ConnectionStrings"));
    // ...
}

But that is an extension method in OptionsConfigurationServiceCollectionExtensions which only works with WebAPI’s DI container. And the Configuration property in the Startup class is set by the runtime, but Nancy is using another bootsrapper. So the question was how to (1) pass the configuration to the Nancy bootstrapper and (2) how to adapt this for TinyIoC which is used in Nancy by default.

The solution for (1) is to have a Custom bootstrapper and pass the configuration to it’s constructor. To solve (2) I wrote my own extension method for TinyIoC.

public class Startup
{
    // ...
    public void Configure(IApplicationBuilder app, IHostingEnvironment env)
    {
        if (env.IsDevelopment())
            app.UseDeveloperExceptionPage();
        app.UseOwin(x => x.UseNancy(options => options.Bootstrapper = new CustomBootstrapper(_configuration)));
    }
}

public class CustomBootstrapper : DefaultNancyBootstrapper
{
    private readonly IConfiguration _configuration;

    public CustomBootstrapper(IConfiguration configuration)
    {
        _configuration = configuration;
    }

	protected override void ConfigureApplicationContainer(TinyIoCContainer container)
    {
        base.ConfigureApplicationContainer(container);
        container.Configure(_configuration.GetSection("NameOfAppsettingsSection"));
    }
}

internal static class ContainerExtensions
{
    public static void Configure(this TinyIoCContainer container, IConfiguration config) where TOptions : class, new()
    {
        var options = new TOptions();
        config.Bind(options);
        container.Register(options);
    }
}

Then, just add your options class as a parameter to classes where it is needed:

public class MyClass
{
    public MyClass(MyOptions options)
    {
        // ...
    }
    // ...
}
Advertisements

Fixing Missing NuGet Packages

I have encountered this problem more than one time. Somehow, maybe when merging changes between branches, there are references in .NET projects to assemblies in NuGet packages that are not present in packages.config, which in turn can result in issues when updating packages to later versions. I put together a PowerShell script to detect this issues. It searches all .csproj files in subfolders to the current one.

# Get reference hintpath to packages:
$pattern = "<HintPath>.*Packages\\(?<id>(?:[a-zA-Z]+\.?)+)\.(?<version>(?:\d+\.)+\d+)\\.*</HintPath>"

Get-ChildItem -Path . -Filter "*.csproj" -File -Recurse | foreach {
    $projFilename = $_.FullName
    Write-Output "`r`n*** $projFilename ***"
    $packageFilename = "$($_.DirectoryName)\packages.config"
    if ([System.IO.File]::Exists($packageFilename))
    {
        $projtext = Get-Content -Path $projFilename
        $projtext | foreach {
            if ($_ -imatch $pattern)
            {
                $id = $Matches.Item("id")
                $version = $Matches.Item("version")
                Write-Output "$($Matches[0])"
                $foundInPackages = Select-String -Pattern "id=""$id"" version=""$version""" -Path $packageFilename -Quiet
                if ($foundInPackages) {
                    Write-Output (Select-String -Pattern "id=""$id"" version=""$version""" -Path $packageFilename).Line
                } else {
                    Write-Warning "Not found in package file!"
                }
            }
        }
    } else {
        Write-Warning "There is no package file at $packageFilename!"
    }
}

Uninstalling NuGet Packages the Hard Way

We were recently in a situation where TeamCity had “cleaned” some NuGet packages that some of our projects depended on. Now they didn’t build in TeamCity. I thought this would be easy to fix: Just upgrade them to the latest version, or uninstall the old and install the new one. But bot of these methods failed, because the package manager failed to build the dependency graph, because it didn’t find the packages. So this was a catch 22!

It turned out that it was possible to manually remove references to these packages by editing the packages.config files, and then run Install-Package. I spent time to automate this, and came up with the following PowerShell script:

$pattern = 'id=”Neonstingray.Nettv4.DataTransfer” | id="Neonstingray.Nettv4.Domain" | id="Neonstingray.Nettv4.Configuration"’
Get-ChildItem -Filter packages.config -Recurse | Select-String -Pattern $pattern | Group Path | ForEach-Object { (Get-Content -Path $_.Name) | where { $_ -NotMatch $pattern } | Set-Content -Encoding UTF8 -Path $_.Name }
$source = "https://<myhost>/httpAuth/app/nuget/v1/FeedService.svc"
install-package Neonstingray.Nettv4.DataTransfer -projectname Valtech.Ready4Air.Ingest.UnitTests -source $source
install-package Neonstingray.Nettv4.DataTransfer -projectname Valtech.Ready4Air.Ingest.Ooyala -source $source
install-package Neonstingray.Nettv4.DataTransfer -projectname Valtech.Ready4Air.Ingest.Ooyala.IntegrationsTests -source $source
install-package Neonstingray.Nettv4.DataTransfer -projectname Valtech.Ready4Air.Ingest.Ooyala.UnitTests -source $source
install-package Neonstingray.Nettv4.DataTransfer -projectname Valtech.Ready4Air.Ingest.PartnerApi -source $source
install-package Neonstingray.Nettv4.DataTransfer -projectname Valtech.Ready4Air.Ingest.PartnerApi.IntegrationTests -source $source
install-package Neonstingray.Nettv4.DataTransfer -projectname Valtech.Ready4Air.Ingest.PartnerApi.UnitTests -source $source

Troubleshooting Memory Leaks in .NET

I recently had problems with a memory leak in an ASP.NET application. I learned some useful tools thanks to Fredrik Haglund.

The first one is “handle”, which is part of the Sysinternals suite. With

handle -s -p <pid>

you can display the number of handles for a given process. This could be useful for applications that e.g. open files but do not close them.

The second one is “procdump”, which is also part of the Sysinternals suite. This will dump the entire process by first creating a copy:

procdump64 -ma -r <pid>

This dump can then be analyzed using the Debug Diagnostics Tool from Microsoft.

Processing Requests and Responses with HttpClient

In a recent project, I wanted to log outgoing requests and incoming responses using my own (JSON) format. This can be accomplished using a custom handler derived from DelegatingHandler. Here is an implementation that can be used in e.g. a console program:

        internal class LoggingHandler : DelegatingHandler
        {
            private static int _requestNumber = 0;
            private static readonly string LogFolder;

            static LoggingHandler()
            {
                LogFolder = Path.Combine(Path.GetTempPath(), "Log");
                if (Directory.Exists(LogFolder))
                {
                    foreach (var file in Directory.GetFiles(LogFolder))
                    {
                        File.Delete(file);
                    }
                }
                else
                    Directory.CreateDirectory(LogFolder);
            }

            protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
            {
                try
                {
                    var response = await base.SendAsync(request, cancellationToken);

                    if (request.Method == HttpMethod.Post)
                    {
                        var logEntry = new LogEntry
                        {
                            Request = new Request
                            {
                                Method = request.Method,
                                Uri = request.RequestUri,
                                Headers = request.Headers,
                                Content = await GetContent(request.Content)
                            },
                            Response = new Response
                            {
                                Headers = response.Headers,
                                StatusCode = response.StatusCode,
                                Content = await GetContent(response.Content)
                            }
                        };

                        var logEntryString = JsonConvert.SerializeObject(logEntry, Formatting.Indented);
                        var parts = request.RequestUri.AbsolutePath.Split('/');
                        string controller = parts.Length >= 6 ? parts[5] : "";
                        string path = Path.Combine(LogFolder, string.Format("{0:0000}0_{1}_{2}_{3}.json", ++_requestNumber, logEntry.Request.Method, controller, logEntry.Response.StatusCode));
                        File.WriteAllText(path, logEntryString, Encoding.UTF8);
                        System.Diagnostics.Debug.WriteLine("Wrote {0:N0} characters to {1}", logEntryString.Length, path);
                    }

                    return response;
                }
                catch (Exception ex)
                {
                    logger.Fatal(ex);
                    throw;
                }
            }

            private static async Task<object> GetContent(HttpContent content)
            {
                try
                {
                    if (content == null)
                        return null;
                    string s = await content.ReadAsStringAsync();
                    s = s.Trim();
                    if (s.StartsWith("["))
                        return JArray.Parse(s);
                    else if (s.StartsWith("{"))
                        return JObject.Parse(s);
                    else
                        return s;
                }
                catch (Exception ex)
                {
                    return ex.Message;
                }
            }
        }

Usage is simple:

var httpClient = new HttpClient(new LoggingHandler());

However, the above implementation didn’t work in a web application. The call to base.SendAsync never returns for some reason. I suppose it has to do with a different threading model. Luckily, I found the solution, to use Task.ContinueWith, in the following blog post: http://byterot.blogspot.se/2012/05/aspnet-web-api-series-messagehandler.html. Here is the alternative implementation.

        protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
        {
            try
            {
                _logger.Debug($"{GetType().Name} sending request  '{request.RequestUri}'...");
                return base.SendAsync(request, cancellationToken).ContinueWith(task =>
                {
                    _logger.Debug($"Task status is {task.Status}");
                    var response = task.Result;
                    _logger.Debug($"{GetType().Name} got response from'{request.RequestUri}'.");

                    if (request.Method == HttpMethod.Post)
                    {
                        var logEntry = new LogEntry
                        {
                            Request = new Request
                            {
                                Method = request.Method,
                                Uri = request.RequestUri,
                                Headers = request.Headers,
                                Content = GetContent(request.Content)
                            },
                            Response = new Response
                            {
                                Headers = response.Headers,
                                StatusCode = response.StatusCode,
                                Content = GetContent(response.Content)
                            }
                        };

                        var logEntryString = JsonConvert.SerializeObject(logEntry, Formatting.Indented);
                        var controller = GetController(request);
                        string path = Path.Combine(LogFolder, $"{++_requestNumber:0000}0_{logEntry.Request.Method}_{controller}_{logEntry.Response.StatusCode}.json");
                        _logger.Debug($"Writing {logEntryString.Length:N0} characters to {path}...");
                        File.WriteAllText(path, logEntryString, Encoding.UTF8);
                        _logger.Debug($"Wrote {logEntryString.Length:N0} characters to {path}.");
                    }

                    return response;
                }, cancellationToken);
            }
            catch (Exception ex)
            {
                _logger.Fatal(ex.Message, ex);
                throw;
            }
        }

        private static object GetContent(HttpContent content)
        {
            try
            {
                if (content == null)
                    return null;
                string s = content.ReadAsStringAsync().Result;
                s = s.Trim();
                if (s.StartsWith("["))
                    return JArray.Parse(s);
                else if (s.StartsWith("{"))
                    return JObject.Parse(s);
                else
                    return s;
            }
            catch (Exception ex)
            {
                return ex.Message;
            }
        }

Entity Framework Wonderland: Migrations and Seeding

This post is a practical dive into Entity Framework 6 migrations and seeding in an ASP.NET MVC 5 web application.

Initial Setup

Suppose we have the following model:

    public class Entity1
    {
        public int Id { get; set; }
        public DateTime Created { get; set; }
        public string Value { get; set; }
    }

    public class MyContext : DbContext
    {
        public MyContext() : base("MyDb")
        {
        }

        public virtual DbSet<Entity1> Entity1 { get; set; }
    }

Add a new MVC 5 controller with views, using Entity Framework  and call it Entity1Controller.

Now, if you run the web application and go to http://localhost:xxxxx/Entity1, the database should be automagically created.

Making a Model Change

Suppose I add another string property (Value2) to Entity1 and hit F5 to run the web application again, I get an exception:

An exception of type ‘System.InvalidOperationException’ occurred in EntityFramework.dll but was not handled in user code

Additional information: The model backing the ‘MyContext’ context has changed since the database was created. Consider using Code First Migrations to update the database (http://go.microsoft.com/fwlink/?LinkId=238269).

OK, let’s fix that. Open Package Manager Console and type

Enable-Migrations

This will add two files to the Migrations folder:

  • <date and time>_IniticalCreate.cs
  • Configuration.cs

Configuration looks like this:

    internal sealed class Configuration : DbMigrationsConfiguration<Models.MyContext>
    {
        public Configuration()
        {
            AutomaticMigrationsEnabled = false;
            ContextKey = "EFMigration1.Models.MyContext";
        }

        protected override void Seed(Models.MyContext context)
        {
            //  This method will be called after migrating to the latest version.

            //  You can use the DbSet<T>.AddOrUpdate() helper extension method 
            //  to avoid creating duplicate seed data. E.g.
            //
            //    context.People.AddOrUpdate(
            //      p => p.FullName,
            //      new Person { FullName = "Andrew Peters" },
            //      new Person { FullName = "Brice Lambson" },
            //      new Person { FullName = "Rowan Miller" }
            //    );
            //
        }
    }

Let’s add a seed:

            context.Entity1.Add(new Models.Entity1
            {
                Created = DateTime.Now,
                Value = "foo",
                Value2 = "bar"
            });

Now hit F5 again. The same exception again! Well, looking at the initial migration, it is understandable, because it doesn’t contain the newly added property. Let’s create a new migration:

Add-Migration Value2

Hit F5 again. Boom! The same exception again. A simple solution now would be to use the Update-Database command. If we do that, the Value2 column is added and the seed data is inserted. Now F5 works.

This is fine for a development environment, but for testing and production it is IMHO not suitable. We have two options:

  1. Configure the application to migrate automatically as outlined in Code First Migrations on MSDN.
  2. Use Migrate.exe

Configuration

Migrate to Latest Version

Enabling automatic migration involves one line of code:

Database.SetInitializer(new MigrateDatabaseToLatestVersion<MyContext, Configuration>());

In this case, I choose to Create a new class EfConfig in the App_Start folder, and call it from Global.asax.cs. This will run migrations if necessary, and seed every time.

Note: Don’t confuse this with the AutomaticMigrationsEnabled property, which will effectively disable versioning of your database.

Create Database If Not Exists

As an aside note, the default initializer is CreateDatabaseIfNotExists, which means that the database is created if it does not exists, but not upgraded when the model changes. You can also set this explicitly, but there is no configuration parameter. So the question is: How do you seed in this case? The solution is to create a custom initializer that derives from CreateDatabaseIfNotExists and move the seeding to that:

    internal class MyInitializer : CreateDatabaseIfNotExists<Models.MyContext>
    {
        protected override void Seed(Models.MyContext context)
        {
            context.Entity1.Add(new Models.Entity1
            {
                Created = DateTime.Now,
                Value = "Initializer seed",
                Value2 = "baz"
            });
        }
    }

Then, use this initializer instead:

Database.SetInitializer(new MyInitializer());

Since migration is only done once, seed is only called once.

Note however that if you do Update-Database, it is the Seed method in Configuration that is used!

Use Migrate.exe

If you want more control of the migration process, you could manually, or as part of the deployment process, call Migrate.exe, which is located in the tools subfolder of the Entity Framework package:

pushd
copy ..\packages\EntityFramework.6.1.3\tools\migrate.exe bin
cd bin
migrate.exe EFMigration1.dll /startupConfigurationFile=..\Web.config /verbose
del migrate.exe
popd

This will run migrations if necessary, and seed.