18 January 2008

Verifying mock expectations in TearDown

I'm currently in a unit testing-oriented period. I've had various discussions with different project teams and colleagues about the subject. One of the things I noticed in a piece of unit test code was that some people were verifying mock expectations in the fixture TearDown. I also found some examples of this practice on the net. The test code was looking like this:


   1:      [TestFixture]
   2:      public class SomeTestFixture
   3:      {
   4:          private MockRepository _mocks;
   5:   
   6:          [SetUp]
   7:          public void Setup()
   8:          {
   9:              _mocks = new MockRepository();
  10:          }
  11:   
  12:          [TearDown]
  13:          public void TearDown()
  14:          {
  15:              _mocks.VerifyAll();
  16:          }
  17:   
  18:          [Test]
  19:          public void SomeTest()
  20:          {
  21:              ISomeInterface interfaceMock = _mocks.CreateMock<ISomeInterface>();
  22:              interfaceMock.SomeMethod();
  23:              _mocks.ReplayAll();
  24:   
  25:              TestedObject sut = new TestedObject();
  26:              sut.SomeInterface = interfaceMock;
  27:              sut.InvokeSomeMethod();
  28:          }
  29:      }

Personally, I don't find it is a good practice, and I would not recommend it. There are I think various reasons for this.

When we look at the basics of automated unit testing, each test case execution happens in four distinct phases, as described in xUnit Patterns: 1- Setup, 2- Exercise, 3- Verify, 4- Tear down. Clearly, the unit testing framework foresees a nice place to put the code pertaining to each of steps: the Setup, Teardown and Test methods. As you might have guessed, the test method should contain steps 2 and 3. When you don't follow that practice, you're in fact breaking the nice organization that your framework brings to your test code, as well as the semantics of the Teardown method (which is only supposed to contain step 4 code).

Also, when we look at the test code (lines 21-27), it is not obvious what we're in fact trying to verify. Did the developer forget to call VerifyAll? Did he forget to make some assertions? It is not intuitive to look at Teardown to actually find the end of the test case logic.

But that's not all. Suppose that you have many test cases in a fixture, and one of the test breaks because of unmet expectations. The stack trace would look like this:

Rhino.Mocks.Exceptions.ExpectationViolationException: ISomeInterface.SomeMethod(); Expected #1, Actual #0.

at Rhino.Mocks.MockRepository.VerifyAll()
at Be.Aprico.Test.SomeTestFixture.TearDown() in SomeTestFixture.cs:line 24


The problem here is that is not obvious from the stack trace which test case failed. Of course, the test runner will tell you which test actually failed, but nevertheless, I prefer to have a stack trace that has its top where the problem is actually located, and not at some unrelated place.

Finally, what if your fixture has real teardown code that must be executed? Should you in this case verify the expectations before or after the teardown code?


        // Should I do this ?
[TearDown]
public void TearDown()
{
_mocks.VerifyAll();

Cleanup();
}

// Or this ?
[TearDown]
public void TearDown()
{
Cleanup();

_mocks.VerifyAll();
}

In the first case, you run the risk that when expectations are not met in one of the test cases, the Cleanup call is in fact never executed due to the thrown exception. This can have bad consequences (mainly interacting, erratic tests, causing unrelated tests to fail because of the previous failed test leftovers). You can correct this by encapsulating the VerifyAll call with a try…finally block, but this in my opinion makes the teardown logic more complicated than it should.

In the second case, what we're really doing is swapping steps 3 and 4 of our 4-phases test. This goes against standard well-known practices, and doesn't help understanding the test code.

To conclude, I would recommend to always put the verification code close to the exercise code. That way, you ensure the test case is readable and complete. My personal preferred syntax is the following (using Rhino Mocks):


        [Test]
public void SomeTest()
{
ISomeInterface interfaceMock = _mocks.CreateMock<ISomeInterface>();
using (_mocks.Record())
{
interfaceMock.SomeMethod();
}
using (_mocks.Playback())
{
TestedObject sut = new TestedObject();
sut.SomeInterface = interfaceMock;
sut.InvokeSomeMethod();
}
}

I find the using a very elegant, clear way to delimit expectations definition and actual exercise. This makes the test code very readable. But this is only a matter of personal preferences, there are other equally valid syntaxes!

13 January 2008

Disposable mocks

I was writing some unit tests for a class that had a dependency on an interface that inherited from IDisposable. I'm using Rhino Mocks (one of the best mock objects library I know) to mock the dependencies of the class being tested.

The class itself implemented IDisposable also, to dispose correctly of its resources. The code for the class looked approximately like this:


using System;


namespace DisposableMock
{

public interface ISomeInterface : IDisposable
{
void DoStuff();
}

public class SomeDisposableClass : IDisposable
{
private ISomeInterface _someInterface;

public ISomeInterface SomeInterface
{
set { _someInterface = value; }
}

public void DoSomeOtherStuff()
{
_someInterface.DoStuff();
}

#region IDisposable Members

public void Dispose()
{
_someInterface.Dispose();
}

#endregion
}
}

The test fixture looked like this:


using NUnit.Framework;
using Rhino.Mocks;

namespace DisposableMock.Test
{
/// <summary>
/// A test fixture for <see cref="SomeDisposableClass"/>
/// </summary>
[TestFixture]
public class SomeDisposableClassFixture
{
private MockRepository _mocks;
private SomeDisposableClass _testee;
private ISomeInterface _someInterface;

[SetUp]
public void Setup()
{
_mocks = new MockRepository();
_testee = new SomeDisposableClass();
_someInterface = _mocks.DynamicMock<ISomeInterface>();
}

[TearDown]
public void TearDown()
{
if (_testee != null)
_testee.Dispose();
}

[Test]
public void Test()
{
// Expectations setup
_someInterface.DoStuff();
_mocks.ReplayAll();

// Replay
_testee.SomeInterface = _someInterface;
_testee.DoSomeOtherStuff();

// Expectations verification
_mocks.VerifyAll();
}
}
}

What's interesting here is the TearDown method. As a good .NET citizen, the test fixture is cleaning up after itself, and it calls the Dispose method of the object under test. Unfortunately, this makes the test fail miserably with an exception in Teardown.

The stack trace looks like this:

System.InvalidOperationException: This action is invalid when the mock object is in verified state.


 

at Rhino.Mocks.Impl.VerifiedMockState.MethodCall(IInvocation invocation, MethodInfo method, Object[] args)

at Rhino.Mocks.MockRepository.MethodCall(IInvocation invocation, Object proxy, MethodInfo method, Object[] args)

at Rhino.Mocks.Impl.RhinoInterceptor.Intercept(IInvocation invocation)

at Castle.DynamicProxy.AbstractInvocation.Proceed()

at ISomeInterfaceProxy71b077222e754186ac7f599256a49a48.Dispose()

at DisposableMock.SomeDisposableClass.Dispose() in SomeDisposableClass.cs:line 29

at DisposableMock.Test.SomeDisposableClassFixture.TearDown() in SomeDisposableClassFixture.cs:line 28


 

Indeed, the fact that I'm calling Dispose on the tested object in Teardown (where expectations were already verified on the mock as this occurs in the test method body), which calls Dispose on the mocked interface is a problem. You can't use a mock in Rhino Mocks after its expectations were verified, even if you're calling a method that does not make part of your expectations. So, I was thinking about how to solve this…

I must call Dispose on the tested object, and I found that Teardown was the right place to do it. I thought I could inject a null reference on _testee.SomeInterface, but this is really not a good thing as it could have side effects on _testee. But maybe Teardown is after all not the right place to Dispose of _testee. So I came with this solution:


 


        [Test]
public void Test()
{
// Expectations setup
_someInterface.DoStuff();
_mocks.ReplayAll();

// Replay
using (_testee)
{
_testee.SomeInterface = _someInterface;
_testee.DoSomeOtherStuff();
}
_mocks.VerifyAll();
}

But frankly, I don't like that. Dispose is really part of cleanup and so should logically go to Teardown, and not pollute every test method. By looking on the MockRepository class, I noticed a method that sounds as it could maybe help me: BackToRecordAll. Let's give it a try…


        [TearDown]
public void TearDown()
{
_mocks.BackToRecordAll();
if (_testee != null)
_testee.Dispose();
}

[Test]
public void Test()
{
// Expectations setup
_someInterface.DoStuff();
_mocks.ReplayAll();

// Replay
_testee.SomeInterface = _someInterface;
_testee.DoSomeOtherStuff();
_mocks.VerifyAll();
}


Aaah much better! This time I can again call Dispose on the mocks without any exception being thrown. Of course, going back to record mode in Teardown is not that intuitive, but still I find it better than having to write cleanup code in my test cases.

05 January 2008

Visual Studio 2008 multi-targeting: using C# 3.0 for your .NET framework 2.0 apps

A friend recently told me he was curious about how it would be possible to use the C# 3.0 compiler to compile applications targeting the .NET 2.0 framework, and this while leveraging the new C# 3.0 features. Indeed, there are no reasons why the 2.0 framework and runtime would have any issues to run C# 3.0 applications, as long as they reference only assemblies from the 2.0 framework. This is because all new C# 3.0 language features were implemented at the compiler level, thus not requiring any change to the runtime (there is no newer runtime than the 2.0).


So I did a quick test with VS 2008, and I was pleased to notice that this indeed works like a charm. At first, I thought that multi-targeting also meant using the compiler corresponding to the target framework, but apparently, VS 2008 always compiles your source files using the C# 3.0 compiler, even when you're targeting another framework than 3.5.


A quick example: the following code


using System;
using System.Collections.Generic;

namespace ConsoleApplication
{
class Program
{
static void Main(string[] args)
{
var foo = 2;
Console.WriteLine(foo);
List<int> fooList = new List<int>() { foo, foo++, foo++ };
foreach (var x in fooList)
Console.WriteLine(x);
Console.ReadLine();
}
}
}


gets compiled into IL, and Reflector shows the equivalent C# 2.0 syntax of the Main method:



private static void Main(string[] args)
{
int foo = 2;
Console.WriteLine(foo);
List<int> <>g__initLocal0 = new List<int>();
<>g__initLocal0.Add(foo);
<>g__initLocal0.Add(foo++);
<>g__initLocal0.Add(foo++);
List<int> fooList = <>g__initLocal0;
foreach (int x in fooList)
{
Console.WriteLine(x);
}
Console.ReadLine();
}


Of course, the above example only uses some of the new C# 3.0 features (type inference and collection initializers), and using LINQ will require adding references to.NET framework 3.5 assemblies (System.Core.dll and System.Data.Linq.dll). Even if you decide to recreate LINQ yourself from scratch, which is perfectly possible, you'll need at least a reference to System.Core.dll because of extension methods that require the System.Runtime.CompilerServices.ExtensionAttribute.


Here's a summary table of new C# 3.0 features that can actually be used in .NET 2.0 applications.


C# 3.0 Feature

Supported on .NET Framework 2.0?

Local Variable Type Inference

Yes

Object Initializers

Yes

Collection Initializers

Yes

Anonymous Types

Yes

Auto-Implemented Properties

Yes

Extension Methods

No (needs ExtensionAttribute in System.Core.dll)

Query Expressions

No (needs Extension Methods)

Expression Trees

No (needs IQueryable in System.Core.dll)

Implicitly-Typed Arrays

Yes

Lambda Expressions

Yes

Partial Methods

Yes



As a conclusion, even if you're still stuck with .NET 2.0 as your runtime environment, you can already leverage some C# 3.0 features in your code. And VS 2008 supports it!

Visual Studio 2008 Project Creation Failed

If you're playing with Guidance Automation eXtensions and you run across this annoying issue where you can no longer create any project in VS 2008, check out this post. That solved the problem on my machine.

03 January 2008

Is ObjectSpaces not quite dead yet?

This morning I was browsing through the .NET Framework 2.0 directory (C:\Windows\Microsoft.NET\Framework\v2.0.50727) on my XP SP2 laptop. And I noticed a very surprising file there…

Coming back home this evening, I checked my second laptop that runs Vista, and yes, it is there too…

Here's what I saw:



It seems that some ObjectSpaces bits finally made it to RTM ;-)

Oh, and by the way, I don't know what caused the apparition of this file because it is not on my XP SP2 desktop…Some hotfix?

01 January 2008

My first post…

So, that's it, as this new year is starting, I've decided to blog a little bit. The idea was already in the air for a few months now, but I always thought I wouldn't have enough time to do it.


Some changes in my personal life make me now think that I probably will have more free time in 2008 than before, so maybe I can bring my very modest contribution to the community.


I expect to be mostly blogging about software engineering topics, but who knows, there could be other subjects also from time to time.


So let's keep things short this time, I have to prepare my next post which will hopefully be more interesting than this one!