assumption of advancement

Last week I migrated several projects from NUnit to MSTest. The developers use the Developer version of Microsoft Visual Studio Team System, so they have integrated unit-test support for MSTest. In this post I show you all the problems and work I had to migrate the tests from NUnit to MSTest.

Less assert-methods in MSTest
In NUnit you have comparison assert methods, exception asserts, more type asserts, more utility asserts and so on (you see what I mean). There already exist little projects to add more assert methods for MSTest.
The problem with less assert method is the following:

Assert.Greater(nCount, 5);

This snippet is much more readable and maintainable than the following:

Assert.IsTrue(nCount > 5);

But that’s just my opinion. There are for sure people who like the second snippet more than the first one.

Attribute mappings
When you planning to migrate your NUnit Tests to MSTest you will need a mapping between the attributes of NUnit and MSTest. You’ll find on the internet mostly the following configuration:
[TestFixture] -> [TestClass]
[Test] -> [TestMethod]
[SetUp] -> [TestInitialize]
[TearDown] -> [TestCleanup]
[TestFixtureSetUp] -> [ClassInitialize]
[TestFixtureTearDown] -> [ClassCleanup]
That’s fine, but wait a minute! For two of those mappings you couldn’t just use find and replace. TestFixtureSetUp and TestFixtureTearDown aren’t static methods in NUnit. Methods with the attribute ClassInitialize or ClassCleanup has to be static. The method with the attribute ClassInitialize has even a parameter Testcontext. For those two methods you have to do a little bit more work.

TestFixtureSetup vs. ClassInitialize
As I mention above the difference between the attributes TestFixtureSetup and ClassInitialize (or TestFixtureTearDown and ClassCleanup), there’s still the question which one is conceptionally correct. Or why Microsoft did it different in comparison to NUnit.
I found the interesting answer in the book xUnit Test Patterns of Gerard Meszaros (on p. 384). Microsoft implements the JUnit approach, what means there will be for each test method a new instance of the test class. James Newkirk (one of the authors of NUnit) regret the decision in NUnit to us the same instance for all the test methods.

Ignore-attribute, no comment
The developers used a lot the attribute ignore in NUnit to deactivate unit tests for explorating testing. In NUnit you could add a comment to justify why the test should be ignored.
In MSTest the ignore attribute does also exists, but you couldn’t add a comment.

Resharper & MSTest
Since Resharper 4.5 you could also use this amazing tool with MSTest. Before the version 4.5 it was also possible to use it with MSTest, but you have to install a plug-in to do that.
From my perspective the Resharper unit test runner is much more better than the Visual Studio test runner.

ExpectedException and Visual Studio
In Visual Studio Professional and above you have an integrated unit-test runner (similar to eclipse, netbeans, etc.). There were a lot of unit-tests which expected an exception with the type Exception. That’s works fine in Resharper, on the build server (TFS) but not in the test runner of Visual Studio. Here you’ll receive the following warning:
UTA017: MyUnitTest has invalid ExpectedException attribute. The type must inherit from Exception.
This is a bug of the test runner, as others already described.

Execute several assemblies
One of the projects which I migrated from NUnit to MSTest runs unit tests of different assembles. So it had a nunit-file like that:

<NUnitProject>
    <Settings activeconfig="Web">
    <Config name="Web" appbase="Web" configfile="web.config" binpathtype="Auto">
        <assembly path="bin\Product.NUnit.ComponentA.dll"/>
        <assembly path="bin\Product.NUnit.ComponentB.dll"/>
        <assembly path="bin\Product.NUnit.ComponentC.dll"/>
    </Config>
</NUnitProject>

I didn’t found a way to do that in MSTest.

Conclusion
With the migration the developers gain code coverage and the integration of unit-testing in Visual Studio (for those who hadn’t Resharper). On the other hand they lost readability, because they have to express an assertion with code with a primitive assert method.
If I could give you an advice, it’s the following: Migrate, if you have at least one of the Visual Studio Team System Editions (Developer, Test or Suite). With Visual Studio Professional (integrates a unit-test runner) and without Resharper, it is also an option to migrate to MSTest.

I saw recently the talk “Comparing Java Web Framworks” on Parleys.com and when the speaker talks about JSF, he used the word “VB”. I was a little bit surprised, that also in the java world the VB-stereotype is known.
But what about is this VB-stereotype? Are there programmers who fit any more this stereotype?
Commonly the stereotype is used for people who don’t know a lot about software developing or engineering and produce really bad code.
Whe I use this stereotype (and I try to avoid it, because it sounds more or less a bit arrogant), I mean the following:
A programmer fits the VB-stereotype when he knows not much about programming and software engineering but he is smart enough to be productive with a tool which allows him to be productive (for example a RAD-Tool).
Since VB.Net is available, the stereotype is not that much used any more, but it still exists. But when I googled for answers I found similar discussions on c2.com and stackoverflow.com. As you see, my definition of the stereotype is not based on any language, it described habits of software developers (similar to code smells).
Because the stereotype is used even today and it is used also as prejudice for not using VB.Net, I found recently a campaign which fight against this.
As I already said, I try to avoid using this stereotype. When I meet a software developer who fits this stereotype, then I try to show him how he could improve his code and his engineering knowledge.

When you create your unit tests for a method in the SUT (software under test) you will ask yourself how to structure the code in the test method.
I saw two kind of syntaxes which help to structure the code in a unit test method (well, actually there are at least three, but the third syntax is just chaos, so this is definitely not the way how to do it).

SEVT-Syntax

[TestFixture]
public class ThingTest
{
    [Test]
    public void TestSomething()
    {
        // Setup

        // Exercise

        // Verify

        // Teardown
    }
}

I found this syntax in the xUnit Test Patterns book by Gerard Meszaros. Obviously this syntax is inspired by classic unit frameworks (like junit or nunit). There is one thing, which could be critical: this syntax could be misused when in the teardown block exist code, which has to run always (in other words, which has to be correctly in teardown-method). But when an assert fails, the teardown block would not be invoked. To prevent this case, you have to do extra effort (try/finally construct).
Gerard Meszaros named this usage of the syntax Four-Phase Test. He also showed, that the last phase (the teardown block) could be done by a teardown-method.

AAA-Syntax

[TestFixture]
public class ThingTest
{
    [Test]
    public void TestSomething()
    {
        // Arrange

        // Act

        // Assert
    }
} 

This syntax I found on the blog of William Wake and ayende. I prefer this syntax, because it sounds good and therefore I don’t forget it. I like also the words which are more precise and there are no competition with the setup- or teardown-methods. It is also clear where to put the asserts.

Conclusion

The most important thing is that you structure your code in your unit-test. It is also very important that you test only one concern per test method. A syntax helps you to create readable, maintainable and specific (one test-concern per test method) tests.