In my last post I accused NUnit of not supporting row tests. Well I was wrong, it does, it just doesn’t call them RowTest as MbUnit does they are referred to as parameterised tests which, is a better name for them as that’s what they really are… tests with parameters. To be totally fair (to me) I was looking for “row tests” and it didn’t occur to me that they would be called something different in NUnit, especially since I found their wiki article talking about Rowtests – that implied to me that row test was the correct term to be researching.

I’m not sure why but something in my mind persuaded me to take another look at some of Roy Osherove’s presentations, I’d watched them before so it was probably a surfacing memory. Anyway in Unit Testing Best Practices Roy demonstrates the use of  TestCase, which is the NUnit attribute for RowTest. I haven’t fully explored it yet (there are 24 overloads), but what I have seen is the same as in MbUnit and I’ve started using it in my tests in place of test names such as Add_SingleNumber_ReturnsValue and Add_SingleNumber_ReturnsValue2 etc…

So NUnit I am very sorry for doubting you!

E-mail | Post RSS Comments

If you’ve been following my blog for a little while you’ll see that my tdd-kata has been progressing, I’m still not able to finish the whole thing but I’m definitely getting better at it. I have noticed that my tests seem to run slower than the demos even though I’m writing pretty simple tests that only run in memory and don’t touch anything external. In The Art of Agile Development James Shore states that you should be able to run about 100 unit tests in a second… as things were I was able to get 9 unit tests to run in around 2 seconds. Something isn’t right.

I’m using the current version of MbUnit 3 (3.2.5.17) really because when I started looking into unit testing I figured Mb (Much Better) Unit Testing… it’s gotta be better than NUnit. However it looks like the problem is Gallio trying to do too much.

At present Gallio can run tests from MbUnit versions 2 and 3, MSTest, NBehave, NUnit, xUnit.Net, csUnit, and RSpec.  Gallio provides tool support and integration with AutoCAD, CCNet, MSBuild, NAnt, NCover, Pex, Powershell, Resharper, TestDriven.Net, TypeMock, and Visual Studio Team System.

Which is great but because it’s doing so much I think it has more work to do (somehow it has to parse and understand the results from all those other frameworks and merge them into just 1 report) and that contributes to a slower experience. I had a quick look to see if I could set it so it only looks for MbUnit tests and ignores all the others, but alas… I couldn’t find such an option. I may be wrong here – I haven’t spent very long looking, and actually I’m hoping that someone reading this post can point me to a configuration option that does just that.

It seems rather odd to me that MbUnit / Gallio would be set, by default / ‘out of the box’ to be the slowest it can be. I would have thought (and again I’m no expert here) that most people would use just a single unit testing framework per project / solution and anyone that’s using more than one framework is likely to be relatively advanced and so able to make configuration changes to support their other frameworks. I would go so far as to say that it is a positive barrier to anyone looking to adopt MbUnit.

Fortunately MbUnit is really a super-set of NUnit so, by way of  a test, I ran my test suite for the tdd kata using MbUnit and then changed my references to NUnit and ran the tests again. MbUnit took 2.54 seconds, NUnit took 1.28 seconds.  Now I realise this is for a really small test sample but that’s significant to me, the longer I have to wait about for tests to run the less happy I am. Well, I say I ran the same tests… in actual fact there is a very useful feature of MbUnit that’s missing in NUnit – RowTests and I’ve come to love (and depend, for clarity’s sake) on RowTests. So I had to create separate tests for each row that I had in MbUnit so actually I ran a few more tests in NUnit than I ran in MbUnit but NUnit was still faster.

Ok so NUnit is faster but MbUnit has row tests… does NUnit support row tests. Well it seems that RowTests were in NUnit 2.4.7 but, as the comments in that post attest, they were removed in 2.5 but not really documented. Googling I quickly got to the the wiki answer for RowTest on NUnit which pointed me to the RowTestExtension for NUnit which I have just tried it and... I can't get it to work.
The version that's up on Google Code was built against 2.4.8 (and last changes were in 2008, so it doesn’t look like it is being developed or supported). When I dropped the dlls into the extensions folder for NUnit and tried to access the feature... nothing, it also wasn’t registering in Visual Studio. So then I downloaded the source files and tried to recompile them against 2.5.7, which is the version I've got. There were some build errors, methods not found and I'm putting that down to changes in NUnit between these versions. Well I'm not familiar enough with NUnit to know what the namespace changes would be or even where to start looking (plus I've not got the time), so it looks like RowTests are not possible with NUnit - which is too bad as they really kick ass in MbUnit (just a shame MbUnit/Gallio is so much slower)... maybe I'll look to xUnit.net in future?

E-mail | Post RSS Comments