Troubleshooting Memory Leaks in .NET

I recently had problems with a memory leak in an ASP.NET application. I learned some useful tools thanks to Fredrik Haglund.

The first one is “handle”, which is part of the Sysinternals suite. With

handle -s -p <pid>

you can display the number of handles for a given process. This could be useful for applications that e.g. open files but do not close them.

The second one is “procdump”, which is also part of the Sysinternals suite. This will dump the entire process by first creating a copy:

procdump64 -ma -r <pid>

This dump can then be analyzed using the Debug Diagnostics Tool from Microsoft.

Integration Testing a Web App with Forms Authentication

This turned out to be harder than I thought. I had to not only figure out how to post form fields using .Net code but also handle anti-forgery cookies and hidden fields.

When you request a protected URL, the browser is redirected to the login page. The header of this response contains a Set-Cookie header and the body a hidden input. When the login form is posted, these must be sent back and match.

Here is some example code that does all this:

        private string LogIn(string startUrl)
        {
            try
            {
                //var proxy = new WebProxy("127.0.0.1", 8888); // Fiddler
                IWebProxy proxy = null;

                // Make the request. This will result in the login page being returned.
                var request = WebRequest.CreateHttp(startUrl);
                request.Proxy = proxy;
                var response = (HttpWebResponse)request.GetResponse();
                Assert.That(response.StatusCode, Is.EqualTo(HttpStatusCode.OK));
                
                // Get request verification token cookie:
                string setCookie = response.Headers["Set-Cookie"];
                Assert.That(setCookie, Is.Not.Null);
                var requestVerificationTokenCookie = setCookie.Split(';')[0];

                // Get form action to be posted to later:
                string responseBody = new StreamReader(response.GetResponseStream()).ReadToEnd();
                var regex = new Regex("form action=\"([^\"]*)\"");
                Assert.That(regex.IsMatch(responseBody));
                string formAction = regex.Match(responseBody).Groups[1].Value;
                Assert.That(formAction, Is.Not.Null);

                // Get request verification token form field:
                regex = new Regex("<input name=\"__RequestVerificationToken\" .* value=\"([^\"]*)\"");
                Assert.That(regex.IsMatch(responseBody));
                string requestVerificaitonToken = regex.Match(responseBody).Groups[1].Value;
                
                // Post the login form:
                var baseUri = new Uri(Properties.Settings.Default.AppUrl);
                var localUri = new Uri(formAction, UriKind.Relative);
                var loginUri = new Uri(baseUri, localUri);
                request = WebRequest.CreateHttp(loginUri);
                request.Proxy = proxy;
                request.Method = HttpMethod.Post.Method;
                request.Headers.Add(HttpRequestHeader.Cookie, requestVerificationTokenCookie);
                string postData = $"__RequestVerificationToken={requestVerificaitonToken}&UserId={Properties.Settings.Default.UserId}&Password={Properties.Settings.Default.Password}";
                byte[] byteArray = Encoding.UTF8.GetBytes(postData);
                request.ContentType = "application/x-www-form-urlencoded";
                request.ContentLength = byteArray.Length;
                Stream dataStream = request.GetRequestStream();
                dataStream.Write(byteArray, 0, byteArray.Length);
                dataStream.Close();
                request.AllowAutoRedirect = false; // Prevents redirecting back to returnUrl.
                response = (HttpWebResponse) request.GetResponse();
                Assert.That(response.StatusCode, Is.EqualTo(HttpStatusCode.Found)); // Redirect to original url
                setCookie = response.Headers["Set-Cookie"];
                Assert.That(setCookie, Is.Not.Null);
                var sessionCookie = setCookie.Split(';')[0];
                return sessionCookie;
            }
            catch (WebException ex)
            {
                Assert.Fail(ex.ToString());
                return null;
            }
        }

Now, this can be used in a test like this:

        [Test]
        public void Test1()
        {
            string url = "http://localhost/...";
            string sessionCookie = LogIn(url);
            var response = Request(sessionCookie, url);
            Assert.That(response, Contains.Substring("(Some string that is always in correct response)"));
        }

        private static string Request(string sessionCookie, string url)
        {
            var webClient = new WebClient();
            webClient.Headers.Add(HttpRequestHeader.Cookie, sessionCookie);
            string response = webClient.DownloadString(url);
            return response;
        }

Migrating from ASP.NET Identity to SQL Membership

If you crate a new ASP.NET MVC 5 project you have four choices for authentication and one of them is “individual user accounts”. This will create a lot of template code using ASP.NET Identity. This is not the same as classic SQL membership system. New tables will be created in your database.

I was in the situation that I had to work with the classic membership system for compatibility reasons. So what changes must be made to the application?

1. Remove the following from web.config:

<remove name="FormsAuthentication" />

2. Still in web.config, add a connection string and membership and rolemanager sections and set authentication mode to forms:

<connectionStrings>
  <add name="AspNetConnection" connectionString="Data Source=MyServer;Initial Catalog=aspnetdb;…" providerName="System.Data.SqlClient" />
</connectionStrings>
<system.web>
  <authentication mode="Forms">
    <forms loginUrl="~/Account/Login" name="MyApp" />
  </authentication>
  <membership defaultProvider="SqlMembershipProvider">
    <providers>
      <clear />
      <add 
        name="SqlMembershipProvider" 
        type="System.Web.Security.SqlMembershipProvider" 
        connectionStringName="AspNetConnection" 
        applicationName="MyApp" />
    </providers>
  </membership>
  <roleManager defaultProvider="SqlRoleProvider"
      enabled="true"
      cacheRolesInCookie="true"
      cookieName=".ASPROLES"
      cookieTimeout="30"
      cookiePath="/"
      cookieRequireSSL="true"
      cookieSlidingExpiration="true"
      cookieProtection="All" >
    <providers>
      <clear />
      <add
        name="SqlRoleProvider"
        type="System.Web.Security.SqlRoleProvider"
        connectionStringName="AspNetConnection"
        applicationName="Cassius" />
    </providers>
  </roleManager>
</system.web>

3. In AccountViewModel, you can delete everything but LoginViewModel. LoginViewModel can contain only the following:

public class LoginViewModel
{
    [Required]
    [Display(Name = "User id")]
    public string UserId { get; set; }

    [Required]
    [DataType(DataType.Password)]
    [Display(Name = "Password")]
    public string Password { get; set; }
}

4. In AccountController, you can delete everything except Login and LogOff. They should look something like this:

//
// GET: /Account/Login
[AllowAnonymous]
public ActionResult Login(string returnUrl)
{
    ViewBag.ReturnUrl = returnUrl;
    return View();
}

//
// POST: /Account/Login
[HttpPost]
[AllowAnonymous]
[ValidateAntiForgeryToken]
public async Task<ActionResult> Login(LoginViewModel model, string returnUrl)
{
     if (!ModelState.IsValid)
     {
          return View(model);
     }

     if (System.Web.Security.Membership.ValidateUser(model.UserId, model.Password))
     {
         System.Web.Security.FormsAuthentication.SetAuthCookie(model.UserId, false);
         return RedirectToLocal(returnUrl);
     }
     else
     {
         ModelState.AddModelError("", "Invalid user name or password.");
         return View(model);
     }
 }

//
// POST: /Account/LogOff
[HttpPost]
[ValidateAntiForgeryToken]
public ActionResult LogOff()
{
     System.Web.Security.FormsAuthentication.SignOut();
     return RedirectToAction("Index", "Home");
}

5. Login.cshtml can be simplified as well:

@using ResellerData.WebAdmin.Models
@model LoginViewModel
@{
    ViewBag.Title = "Log in";
}
<h2>@ViewBag.Title.</h2>
<div class="row">
<div class="col-md-8">
        <section id="loginForm">
            @using (Html.BeginForm("Login", "Account", new { ReturnUrl = ViewBag.ReturnUrl }, FormMethod.Post, new { @class = "form-horizontal", role = "form" }))
            {
                @Html.AntiForgeryToken()
<h4>Use a local account to log in.</h4>

<hr />

@Html.ValidationSummary(true, "", new { @class = "text-danger" })
<div class="form-group">
                    @Html.LabelFor(m => m.UserId, new { @class = "col-md-2 control-label" })
<div class="col-md-10">
                        @Html.TextBoxFor(m => m.UserId, new { @class = "form-control" })
                        @Html.ValidationMessageFor(m => m.UserId, "", new { @class = "text-danger" })</div>
</div>
<div class="form-group">
                    @Html.LabelFor(m => m.Password, new {@class = "col-md-2 control-label"})
<div class="col-md-10">
                        @Html.PasswordFor(m => m.Password, new {@class = "form-control"})
                        @Html.ValidationMessageFor(m => m.Password, "", new {@class = "text-danger"})</div>
</div>
<div class="form-group">
<div class="col-md-offset-2 col-md-10">
                        <input type="submit" value="Log in" class="btn btn-default" /></div>
</div>
}
        </section></div>
</div>
@section Scripts {
    @Scripts.Render("~/bundles/jqueryval")
}

6. Then, delete a lot of unecessary files:

  • IdentityConfig.cs
  • Startup.Auth.cs
  • ManageController.cs
  • IdentityModels.cs
  • ManageViewModel.cs
  • Everything under Views\Account except Login.cshtml
  • Everything under Views\Manage

Logging All Exceptions in ASP.NET and WCF

I find it useful to log all uncaught exception centrally in ASP.NET web applications and WCF services. In ASP.NET web applications, it is relatively easy – just put logging code in Global.asax.cs like this:

        protected void Application_Error(object sender, EventArgs e)
        {
            var ex = Server.GetLastError();
            if (ex is HttpUnhandledException && ex.InnerException != null)
                ex = ex.InnerException;

            var httpException = ex as HttpException;
            if (httpException != null && httpException.GetHttpCode() == 404)
            {
                // Log this
                Uri urlReferrer = null;
                try
                {
                    urlReferrer = Request.UrlReferrer;
                }
                catch (Exception)
                {
                    // ignored
                }
                Log.WarnFormat("{0} (Client address={1}, UrlReferrer={2})", ex.Message, Request.UserHostAddress, urlReferrer);
            }
            else
                Log.Error(ex.Message, ex);
        }

One thing to note is that uncaught exceptions are wrapped in an HttpUnhandledException – that is why I extract the inner exception. Another thing to note is that you may not want to log 404 not found or as here, log it with a different level.

With WCF, it is (as always) a little more complicated. Putting code in Application_Error in Global won’t work. You have to write one or two classes that implement System.ServiceModel.Dispatcher.IErrorHandler and System.ServiceModel.Description.IServiceBehavior, and do some configuration work. There are different ways to do this. See e.g. http://codeidol.com/csharp/wcf/Faults/Error-Handling-Extensions/. But here is my simplistic solution:

using System;
using System.Collections.ObjectModel;
using System.ServiceModel;
using System.ServiceModel.Channels;
using System.ServiceModel.Configuration;
using System.ServiceModel.Description;
using System.ServiceModel.Dispatcher;
using log4net;

namespace MyNamespace
{
    public class ErrorHandler : IErrorHandler, IServiceBehavior
    {
        private static readonly ILog Log = LogManager.GetLogger(typeof(ErrorHandler));

        #region IErrorHandler Members

        public void ProvideFault(Exception error, MessageVersion version, ref Message fault)
        {
            var faultException = new FaultException("Error caught in API error handler. Please see API server log for details.");
            var messageFault = faultException.CreateMessageFault();
            fault = Message.CreateMessage(version, messageFault, faultException.Action);
        }

        public bool HandleError(Exception error)
        {
            Log.Error(error.Message, error);
            return false;
        }

        #endregion

        #region IServiceBehavior Members

        // This behavior modifies no binding parameters.
        public void AddBindingParameters(
            ServiceDescription description,
            ServiceHostBase serviceHostBase,
            Collection<ServiceEndpoint> endpoints,
            BindingParameterCollection parameters
            )
        {
        }

        // This behavior is an IErrorHandler implementation and must be applied to each ChannelDispatcher.
        public void ApplyDispatchBehavior(ServiceDescription description, ServiceHostBase serviceHostBase)
        {
            foreach (ChannelDispatcher chanDisp in serviceHostBase.ChannelDispatchers)
            {
                chanDisp.ErrorHandlers.Add(this);
            }
        }

        public void Validate(ServiceDescription description, ServiceHostBase serviceHostBase)
        {
        }

        #endregion
    }

    public class ErrorHandlerBehavior : BehaviorExtensionElement
    {
        protected override object CreateBehavior()
        {
            return new ErrorHandler();
        }

        public override Type BehaviorType
        {
            get { return typeof(ErrorHandler); }
        }
    }
}

In ProvideFault, which executes on the request thread, I return a generic error to the client (for security reasons). It is logged in HandleError, which executes after the response has been sent.

Here is part of web.config to use this error handler:

  </system.serviceModel>
    <extensions>
      <behaviorExtensions>
        <add name="ErrorHandler" type="MyNamespace.ErrorHandlerBehavior, MyNamespace" />
      </behaviorExtensions>
    </extensions>
    <behaviors>
      <serviceBehaviors>
        <behavior name="MyBehavior">
          <!-- ... -->
          <ErrorHandler/>
        </behavior>
      </serviceBehaviors>
    </behaviors>
  </system.serviceModel>

Error Handling in ASP.NET MVC

Basically, there are two ways of customizing error handling in ASP.NET MVC.

One is to turn on customErrors in web.config and add HandleErrorAttribute in FilterConfig.cs or to a specific controller or method. This will display the Views/Shared/Error view.

A better approach is to add a Application_Error method in Global.asax where you can log the error. Here is an example with specific code for DbEntityValidationException (with credits to Slauma on Stackoverflow):

        protected void Application_Error(object sender, EventArgs e)
        {
            var ex = Server.GetLastError();
            _log.Error("Error caught in Global.Application_Error", ex);
            var dbeve = ex as System.Data.Entity.Validation.DbEntityValidationException;
            if (dbeve != null)
            {
                StringBuilder sb = new StringBuilder();
                foreach (var eve in dbeve.EntityValidationErrors)
                {
                    sb.AppendLine(string.Format("- Entity of type \"{0}\" in state \"{1}\" has the following validation errors:",
                        eve.Entry.Entity.GetType().FullName, eve.Entry.State));
                    foreach (var ve in eve.ValidationErrors)
                    {
                        sb.AppendLine(string.Format("-- Property: \"{0}\", Value: \"{1}\", Error: \"{2}\"",
                            ve.PropertyName,
                            eve.Entry.CurrentValues.GetValue<object>(ve.PropertyName),
                            ve.ErrorMessage));
                    }
                }
                _log.Error(sb.ToString());
            }
        }

If you just want to see the details of a DbEntityValidationException during debugging, you can add a watch with the name “((System.Data.Entity.Validation.DbEntityValidationException)$exception).EntityValidationErrors”.