More NUnit Tricks
I was doing some coverage tests of a segment of our code use
AutomatedQA's
AQTime. It's a nice tool and I've
plugged it before.
One of the key features of .NET that we use is a mix of managed and unmanaged code. For the most part, the APIs to get us that ".NETty" feel to our classes are done in C#. For the actual pixel pushing, we lean towards unmanaged C++ where the cycles really count.
As you can imagine, trying to create meaningful benchmarks with our code base is challenging. AQTime does work with both managed and unmanaged code, but there's a caveat. Since we like Unit Testing and continuous integration, most of our testing is done through
NUnit or for a single engineer, using TestDriven.NET's VS plug-in. The problem is that NUnit confounds AQTime with respect to unmanaged code, so AQTime doesn't pick up on the unmanaged code getting run, so coverage isn't measured.
The workaround is to build a console application that runs the unit tests itself and code coverage works fine for that. That left me with the problem of running the actual tests themselves. I did some cursory checking and didn't find anything that leapt out me and said "make this class, give it an assembly/class name and it'll run the tests in the NUnit way". Because I'm a geek, I knocked this together. It's not perfect, but if you've been curious about reflection, attributes and so on, this is a nice little snippet of practical use.
private static bool IsTest(MethodInfo info)
{
// this lets us know if a method is a valid [Test] method
object[] attrs = info.GetCustomAttributes(typeof(TestAttribute), true);
if (attrs.Length == 0) // no attribute
return false;
// make sure it's non-static and public
return !info.IsStatic && info.IsPublic);
}
private static MethodInfo GetMethodWithAttribute(MethodInfo[] methods, Type attr)
{
// find a method with a given attribute type
foreach (MethodInfo method in methods)
{
if (!method.IsPublic || method.IsStatic)
continue;
object[] attrs = method.GetCustomAttributes(attr, true);
if (attrs != null && attrs.Length > 0)
return method;
}
return null;
}
private static MethodInfo GetSetup(MethodInfo[] methods)
{
// Gets the setup method - returns null if there is none
return GetMethodWithAttribute(methods, typeof(SetUpAttribute));
}
private static MethodInfo GetTeardown(MethodInfo[] methods)
{
// Gets the teardown method - returns null if there is none
return GetMethodWithAttribute(methods, typeof(TearDownAttribute));
}
private static void RunAllTests(object o)
{
// run all the tests methods in the given object
MethodInfo[] methods = o.GetType().GetMethods();
MethodInfo setup = GetSetup(methods);
MethodInfo teardown = GetTeardown(methods);
foreach (MethodInfo method in methods)
{
if (IsTest(method))
{
try
{
Console.WriteLine(method.Name);
if (setup != null)
setup.Invoke(o, null);
method.Invoke(o, null);
}
catch (Exception err)
{
// since I'm doing coverage I don't care about failures
// so I just toss them out to the console
// in addition - there should be some code to look for
// [ExpectedException], but again, I don't need it here
Console.WriteLine("threw " + err.GetType().Name + ": " + err);
}
finally
{
if (teardown != null)
teardown.Invoke(o, null);
}
}
}
}
When using this you'd call RunAllTests on a new instance of a test fixture object. In my case, I want to run specific test objects for coverage so I do something like RunAllTests(new AbnormalityTests()) and off it goes. If you wanted to test across an entire assembly, you need to go further. Given an Assembly object, you need to get all the types via GetTypes(), then loop through and find each type that is decorated with TestFixtureAttribute, then pull out its default constructor, invoke it, and pass the resulting object into RunAllTests().
Now, here's something to think about as you're looking at C# 3.0. Included in C# is the ability to use database style accessors on things that are collections and have appropriate interfaces. Consider this: an assembly is a database of classes (Types) each of which is really a database of methods. I find this exciting from a code-writing-code point of view because it's possible to start doing automatic approximate matching for methods and creating anonymous adapter delegates to "fix" problems for you. Think about it this way - if you screw up argument order in a method call, its becomes really simple to find one or more methods that match the args and reorder them while flagging a warning. Build this into a language as part of the runtime as well then you can do fault-tolerant runtime linking. Schweet.