Mocking frameworks in .Net
A few month ago I played with some mocking frameworks in .Net. There are already some comparisons available (here, here or here). In this blog post I want to show which frameworks are available and which one fits best for agile development.
You could download the source code from github.com.
Software under Test (SUT)
To demonstrate and evaluate all the different mocking framework I had to create a scenario where it is appropriate to use a mocking framework. So I chose an user entity which I want to validate and in a successful case also to persist in a database.
namespace Mocking
{
public class User
{
public IUserGateway Gateway
{
get; set;
}
public User()
{
this.Gateway = new UserGateway();
}
public User(IUserGateway gateway)
{
this.Gateway = gateway;
}
public bool Persist(IUserValidator validator)
{
bool bValid = validator.Validate(this);
if(bValid) bValid = this.Gateway.Persist(this);
return bValid;
}
}
}
This is the User entity class which implements the active record pattern. As you see, there are three injection patterns realized, the constructor injection and setter injection for the gateway and the parameter injection for the validator.
namespace Mocking
{
public class UserGateway : IUserGateway
{
public virtual bool Persist(User user)
{
return true;
}
}
}
The UserGateway class is responsible to persist the entity. In my scenario it is just a dummy, because I don’t want to setup a database for this demo.
namespace Mocking
{
public class UserValidator : IUserValidator
{
public bool Validate(User user)
{
if(user == null) return false;
return true;
}
}
}
The validator is also more or less just a dummy for this demo.
Both interfaces, IUserGateway and IUserValidator, has the same methods, properties as the classes which implement them.
NMock2
One of the first mocking frameworks in .Net which I was looking at was NMock2. Specially the cheat sheet make it really easy to learn this framework.
namespace MockingTest
{
[TestClass]
public class NMock2Test
{
[TestMethod]
public void TestPersist()
{
Mockery mocks = new Mockery();
//Create mocks
IUserGateway mockGateway = mocks.NewMock<IUserGateway>();
IUserValidator mockValidator = mocks.NewMock<IUserValidator>();
//Create user
User user = new User();
//Expectations
using(mocks.Ordered)
{
Expect.Once.On(mockValidator).Method("Validate").With(user).
Will(Return.Value(true));
Expect.Once.On(mockGateway).Method("Persist").With(user).
Will(Return.Value(true));
}
//Assign Gateway
user.Gateway = mockGateway;
//Test method
Assert.AreEqual(true, user.Persist(mockValidator));
mocks.VerifyAllExpectationsHaveBeenMet();
}
}
}
You could use the AAA-Syntax (Arrange-Act-Assert). Also more extended features like the ordered calls with the using syntax is very nice.
But, there is are also negative points: NMock2 uses “magic strings”. In your Expectations you have to declare the called method with a strings, which makes your tests very brittle when your refactor your code. For example if you have to rename a method, even the compiler could not help you. An other negative point is, that you can only mock interfaces. When you have to mock a class, you have to change your design by adding an interface just for mocking.
NMock3
Recently I discover, that there is already a follower of NMock2 on codeplex: NMock3.
namespace NMock3Test
{
[TestClass]
public class NMock3Test
{
[TestMethod]
public void TestPersist()
{
MockFactory factory = new MockFactory();
//Create mocks
Mock<IUserGateway> mockGateway = factory.CreateMock<IUserGateway>();
Mock<IUserValidator> mockValidator = factory.CreateMock<IUserValidator>();
//Create user
User user = new User();
//Expectations
using(factory.Ordered)
{
mockValidator.Expects.One.MethodWith(m => m.Validate(user)).WillReturn(true);
mockGateway.Expects.One.MethodWith(m => m.Persist(user)).WillReturn(true);
}
//Assign gateway
user.Gateway = mockGateway.MockObject;
//Test method
Assert.AreEqual(true, user.Persist(mockValidator.MockObject));
factory.VerifyAllExpectationsHaveBeenMet();
}
}
}
The migration is quite easy: Replace the reference from your test project from the NMock2.dll with a new reference to the NMock3.dll. After that you could use the new approach with the MockFactory and the lambda expressions in your Expectations.
NMock3 introduce type safety and readiness for refactorings combined with the easiness of NMock2. They also keep the tradition of a good cheat sheet.
With NMock3 I couldn’t mock a class (even when I followed the exceptions…"). The creation of stubs should be possible. The project is still a young one and so the documentation isn’t that good as it should be. But NMock3 remain an interesting mocking framework.
Simple.Mocking
When I looked for mocking frameworks I also found this new and not very well known framework Simple.Mocking on codeplex. Its also avoid “magic strings” and you could also use the AAA-Syntax. Simple.Mocking has only one committer, Mikael Waltersson.
namespace MockingTest
{
[TestClass]
public class SimpleNetTest
{
[TestMethod]
public void TestPersist()
{
//Create mocks
var expectationScope = new ExpectationScope();
IUserGateway mockGateway = Mock.Interface<IUserGateway>(expectationScope);
IUserValidator mockValidator = Mock.Interface<IUserValidator>(expectationScope);
//Create user
User user = new User();
//Expectations
Expect.Once.MethodCall(() => mockValidator.Validate(user)).Returns(true);
Expect.Once.MethodCall(() => mockGateway.Persist(user)).Returns(true);
//Assign gateway
user.Gateway = mockGateway;
//Test method
Assert.AreEqual(true, user.Persist(mockValidator));
AssertExpectations.IsMetFor(expectationScope);
}
}
}
The ExpectationsScope is used to avoid to call AssertExpectations.IsMetFor for each mock. This construct is special and I’m (currently) not a big fan of it.
With Simple.Mocking you could just mocking interfaces and delegates, you couldn’t mock classes.
Rhino.Mocks
Rhino.Mocks is one of the famous .Net mocking frameworks. As Simple.Mocking there is also only one committer to this framework, Oren Eini (alias Ayende Rahien).
namespace MockingTest
{
[TestClass]
public class RhinoMockTest
{
[TestMethod]
public void TestPersistWithRecordingReplay()
{
MockRepository mocks = new MockRepository();
IUserGateway gateway = mocks.StrictMock<IUserGateway>();
IUserValidator validator = mocks.StrictMock<IUserValidator>();
//Create user
User user = new User();
//Expectations
using(mocks.Ordered())
{
Expect.Call(validator.Validate(user)).Return(true);
Expect.Call(gateway.Persist(user)).Return(true);
}
//Stop recording, start replay
mocks.ReplayAll();
//Assign gateway
user.Gateway = gateway;
//Test method
Assert.AreEqual(true, user.Persist(validator));
mocks.VerifyAll();
}
[TestMethod]
public void TestPersistWithArrangeActAssert()
{
var gateway = MockRepository.GenerateMock<IUserGateway>();
var validator = MockRepository.GenerateMock<IUserValidator>();
//Create user
User user = new User();
//Expectations
validator.Expect(x => x.Validate(user)).Return(true);
gateway.Expect(x => x.Persist(user)).Return(true);
//Assign gateway
user.Gateway = gateway;
//Test method
Assert.AreEqual(true, user.Persist(validator));
//Asserts
validator.VerifyAllExpectations();
gateway.VerifyAllExpectations();
}
}
}
Rhino.Mocks supports two ways to use the framework: the record-replay and the AAA-syntax. My favorite is clearly the AAA-syntax, because there is much less code needed (reduced accidental complexity). Rhino.Mocks uses also Lambda-Expressions to avoid “magic strings”, so refactoring shouldn’t be a problem.
Moq
Moq is another mocking framework which takes advantage of the .Net 3.0 language features like lambda expressions. So Moq doesn’t use “magic strings” and it enables also the AAA-syntax. There are two ways how to create mocks. You could create mocks directly with the Mock class or you could use the MockFactory class. For both scenarios I created a test.
namespace MockingTest
{
[TestClass]
public class MoqTest
{
[TestMethod]
public void TestPersist()
{
var gateway = new Mock<IUserGateway>();
var validator = new Mock<IUserValidator>();
User user = new User();
//Expectations
validator.Setup(x => x.Validate(user)).Returns(true).AtMostOnce();
gateway.Setup(x => x.Persist(user)).Returns(true).AtMostOnce();
//Assign gateway
user.Gateway = gateway.Object;
//Test method
Assert.AreEqual(true, user.Persist(validator.Object));
validator.VerifyAll();
gateway.VerifyAll();
}
[TestMethod]
public void TestPersistWithFactory()
{
MockFactory factory = new MockFactory(MockBehavior.Strict);
//Class works, but methods have to be virtual -> not nice
var mockGateway = factory.Create<UserGateway>();
var mockValidator = factory.Create<IUserValidator>();
User user = new User();
//Expectations
mockValidator.Setup(x => x.Validate(user)).Returns(true).AtMostOnce();
mockGateway.Setup(x => x.Persist(user)).Returns(true).AtMostOnce();
//Assign gateway
user.Gateway = mockGateway.Object;
//Test method
Assert.AreEqual(true, user.Persist(mockValidator.Object));
factory.VerifyAll();
}
}
}
I prefer the MockFactory approach, because you don’t need to verify each mock (its reduce the test code which is a good thing). Be sure, that in this scenario you don’t forget the parameter MockBehavior.Strict which makes your mock objects real mocks. Without this parameter they are more like Stubs, which don’t fail and just return a default value.
But there is a little problem: If you want to mock a class, then all the methods have to be virtual, but Moq has no problem to mock an interface.
Typemock Isolator
Unfortunately Typemock Isolator isn’t free, but you could get a trial to try this great mocking framework. As you could see in the TestPersistWithReflectiveMocking Typemock Isolator could mock classes, also non-virtual methods. Great. It also doesn’t use “magic strings”. But wait, you could not mock interfaces? With the new AAA-syntax it is possible and it looks very nice.
namespace MockingTest
{
[TestClass]
public class TypeMockTest
{
[TestMethod]
public void TestPersistWithNaturalMocking()
{
MockManager.Init();
User user = new User();
// Natural mocking
using(RecordExpectations recorder = new RecordExpectations())
{
IUserGateway mockGateway = new UserGateway();
IUserValidator mockValidator = new UserValidator();
//Expectations
recorder.ExpectAndReturn(mockValidator.Validate(user), true);
recorder.ExpectAndReturn(mockGateway.Persist(user), true);
}
//Create instances
IUserGateway gateway = new UserGateway();
IUserValidator validator = new UserValidator();
//Assign gateway
user.Gateway = gateway;
//Method
Assert.AreEqual(true, user.Persist(validator));
MockManager.Verify();
}
[TestMethod]
public void TestPersistWithReflectiveMocking()
{
MockManager.Init();
//Create mocks
Mock mockGateway = MockManager.Mock<UserGateway>();
Mock mockValidator = MockManager.Mock<UserValidator>();
//Create user
UserGateway gateway = new UserGateway();
IUserValidator validator = new UserValidator();
User user = new User(gateway);
//Expectations
mockValidator.ExpectAndReturn("Validate", true).Args(user);
mockGateway.ExpectAndReturn("Persist", true).Args(user);
//Method
Assert.AreEqual(true, user.Persist(validator));
MockManager.Verify();
}
[TestMethod]
[Isolated]
public void TestPersistWithAAA()
{
//Arrange
//Create mocks
UserGateway gateway = Isolate.Fake.Instance<UserGateway>();
IUserValidator validator = Isolate.Fake.Instance<IUserValidator>();
//Create user
User user = new User(gateway);
//Expectations
Isolate.WhenCalled(() => validator.Validate(user)).WillReturn(true);
Isolate.WhenCalled(() => gateway.Persist(user)).WillReturn(true);
//Act
bool bPersist = user.Persist(validator);
//Assert
Assert.AreEqual(true, bPersist);
}
}
}
Typemock Isolator also supports the record-replay approach (demonstrated in the TestPersistWithNaturalMocking test method). But as you already know, I prefer the AAA-syntax.
The question about Typemock Isolator is, if it is worth to pay for a mocking framework if there are so much other free opportunities? Typemock Isolator is very powerful and for some companies it is very important to get support and also guarantee that the product will be continued for example ten years. In such a scenario Typemock Isolator would be definitely an option. In reaction of such scenarios, also other providers of mocking frameworks offer commercial support for their frameworks, for example Rhino.Mocks.
The undocumented – NUnit Mocking
It is not very known that there exists also in NUnit a mocking framework. It is possible to mock interfaces and classes if the inherit from MarshalByRefObject.
namespace MockingTest
{
[TestClass]
public class NUnitMockTest
{
[TestMethod]
public void TestPersist()
{
//Gateway
DynamicMock mockGateway = new DynamicMock(typeof(IUserGateway));
IUserGateway gateway = (IUserGateway) mockGateway.MockInstance;
//Validator
DynamicMock mockValidator = new DynamicMock(typeof(IUserValidator));
IUserValidator validator = (IUserValidator)mockValidator.MockInstance;
//User
User user = new User(gateway);
//Expectations
mockValidator.ExpectAndReturn("Validate", true, user);
mockGateway.ExpectAndReturn("Persist", true, user);
Assert.AreEqual(true, user.Persist(validator));
mockValidator.Verify();
mockGateway.Verify();
}
}
}
The mocking framework is easy to use and if you already use NUnit, then you don’t need an additional framework.
As I know, it isn’t official and there is no documentation. You have also to use “magic strings” for methods and the limitation for classes also reduce the benefit.
Conclusion
For a modern mocking framework it is essential that its enable refactoring by not using “magic strings”. It is also essential for me that it allows to use the AAA-syntax. A third essential thing is that it allows you to mock classes. You don’t want to create interface just for mocking, for example for your domain model. Avoid accidental complexity is very important, so you shouldn’t add unnecessary code just for mocking. That doesn’t mean that “design for testability” is wrong, “design for testability” means, that you design your code in a way, that you have the possibility to test your code, for example you avoid huge classes (god object anti-pattern) with a lot of private methods or you avoid creation of instances in methods (unable to use dependency injection).
So, which framework fulfill all the requirements? NMock3, which inherit the easiness of NMock2 combined with the elimination of the “magic strings”? Or Typemock Isolator which also eliminate the “magic strings” and could even mock classes?
For me, there isn’t currently a clear winner visible, maybe Typemock makes the race. But there are frameworks which I would no longer use, as NMock2 or NUnit mocking. They don’t fulfill my requirements clearly. That doesn’t mean they aren’t suitable for your scenarios.
Also to be clear: There are other topics that should be considered: mocking events, Stubs, etc. The list of the mocking frameworks isn’t complete, for example Moles is a framework which could be interesting too.
12 thoughts on “Mocking frameworks in .Net”
Thanks.
Nice reference, I already used it today in my project to lookup AAA mode in RhinoMocks.
I’m interested to know how you found out about the typemock API –
what docs or web pages did you look at?
I’m asking because
– isolator can indeed fake interfaces
– Isolator has a much better new API for the past year
@Roy Osherove
Hi Roy
Thanks for your question and hints. One resource was http://www.typemock.com/Docs/UserGuide/OverView.html, but there is obviously a newer one at http://www.typemock.com/Docs/UserGuide/. I added a sample with the new AAA-syntax. Sorry, that I missed that.
Hi
I’m a team member of the NMock2 project. And I want to add that we stopped further development of NMock2 because there are (in my eyes) better frameworks now (Moq, Rhino Mocks, NMock3). We only maintain NMock2 for all the projects out there that still use it (and my own of course).
We switched to Moq in our projects. Moq allows very clean AAA syntax and is very well suited for TDD.
But your exampe with Moq is not the way it should be programmed. A better solution would be this:
[TestMethod]
public void TestPersist()
{
// Arrange
var gateway = new Mock();
var validator = new Mock();
User user = new User();
validator.Setup(x => x.Validate(user)).Returns(true);
gateway.Setup(x => x.Persist(user)).Returns(true);
user.Gateway = gateway.Object;
// Act
bool result = user.Persist(validator.Object);
// Assert
Assert.IsTrue(result);
}
Because in this example the Mocks are actually Stubs, we do not need to verify anything on them. The test will only succeed if the get called; otherwise Moq would return false as a default value and the test would fail.
Unfortunately, your example does not show the major advantage of Moq over NMock2/3:
You can make assertions about what had to happen on the mock objects in the act part of the unit test and not in the arrange part.
NMock2:
Expect.Once.On(mock).Method(“Foo”);
testee.CallFooOnMock();
mockery.VerifyAllExpectationsHaveBeenMet()
Moq:
teestee.CallFooOnMock();
mock.Verify(m => m.Foo(), Times.Once)
I think that this syntax is much clearer than the factory approach because it is strictly AAA.
Cheers,
Urs
@Urs Enzler
Hi Urs
Thank you for your detailed comment. I modified the Moq examples on github and in the blog post. Your example shows, as you said, the usage of stubs and not mocks. I want to show the shortest way to express expectations. So you don’t have to use verify to express how many times the method should be called, you could use the (new) setup method.
With the verify method I couldn’t define, what the return value should be.
I agree with you, that the factory approach is a little bit noisier, but at the end you could profit to call only once the verify-method.
Just a comment about simple.mocking, the ExpectationScope is primarily for setting up ordered/more complex expectations involving one or more mock objects. The ability to assert that all expectation has been met for several mock objects at a time is just a side effect. In most cases the ordering is not important and the ExpectationScope (or Mock Factory) can be omitted which means less boilerplate code.
It is an active choice not to implement mocking of abstract and concrete classes, to promote a code against interfaces, not implementations development style. Mocking parts of an implementation, which is what you do when you mock specific methods of an abstract or concrete class adds complexity and is not as clear as mocking an interface.
Simple.Mocking is something I developed when me and the team I worked with at the moment didnt find any mocking framework we were really satisfied with. I will continue to correct bugs and maybe add some functionality but I will strive to keep the API small and simple but still useable.