Tag: unit testing

In the past, I’ve talked about ways to easily write unit tests for Cocoa applications, including [tests for user interfaces using target-action][1] and [tests for interfaces using Cocoa bindings][2]. There are some strategies you can apply to make writing tests for Cocoa code even easier, though. They’re just straightforward object-oriented programming, but sometimes we can forget that all the techniques you might use in your main code base can also be applied to our test code.

So here’s one trick that you can use in writing tests for Cocoa user interfaces, especially in ways that will make test-driven development easier.

### Use a Common Base Class with Check Methods

The first, probably most important thing to do is *use your own common base class* for your tests. Don’t derive your tests directly from `SenTestCase`; instead, derive them from your own `MyTestCase` class that in turn derives from `SenTestCase`. This gives you a place to put all of the customization that’s appropriate for your project.

Sometimes you might need a series of assertions to verify some particular state. However, that series of assertions will be the same every time you need to verify that state. Or the assertions themselves aren’t very *intention revealing* so you always wind up putting a comment above them describing what they’re really doing.

### Checking Target-Action

A simple example of this is checking a target-action connection in a user interface. Say you have a view controller that presents a collection of objects managed by an array controller. Its view has an Add button that should send `-addObject:` to the array controller. You might write a test for it like this:

– (void)testAddButtonSendsAddObjectToArrayController {
STAssertEquals([viewController.addButton target], viewController.arrayController,
@”The Add button should target the array controller.”);
STAssertEquals([viewController.addButton action], @selector(addObject:),
@”The Add button should send the -addObject: action.”);
}

That’s not too difficult to understand, but it could be made simpler — it could be done in a single assertion. You’d just write a method to check both the target and action at once and then use that method from your test, like this:

// in your test base class…

/*! Tells whether the control sends the action to the target. */
– (BOOL)checkControl:(NSControl *)control
sendsAction:(SEL)action
toTarget:(id)target
{
return ([control action] == action)
&& ([control target] == target);
}

// in the tests specifying the view controller’s behavior…

– (void)testAddButtonSendsAddObjectToArrayController {
STAssertTrue([self checkControl:viewController.addButton
sendsAction:@selector(addObject:)
toTarget:viewController.arrayController],
@”The Add button’s action should send -addObject: to the array controller.”);
}

That makes the intention behind the entire test a lot clearer, and it makes writing the test easier & safer since you can’t (say) forget to check either the target or the action.

It *does* lose a tiny bit of information: If the test fails, you’ll have to look at your xib file instead of the failure message to determine whether it’s because the target or the action isn’t set as you’ve specified. However, the trade-off in making the test easier to read and write is worth it here.

### Checking Outlets

This is even worthwhile for single assertions, such as those you’d use to test that your outlets are connected in the first place. For example, you might initially write a test that your view controller is your table view’s delegate like this:

– (void)testViewControllerIsTableViewDelegate {
STAssertEquals([viewController.tableView delegate], viewController,
@”The table view’s delegate should be the view controller.”);
}

Rewriting it to be more intention-revealing with a simple check method would make it look like this:

// in your test base class…

/*! Tells whether the outlet is connected to the given destination. */
– (BOOL)checkOutlet:(id)outlet connectsTo:(id)destination {
return outlet == destination;
}

// in the tests specifying the view controller’s behavior…

– (void)testViewControllerIsTableViewDelegate {
STAssertTrue([self checkOutlet:[viewController.tableView delegate]
connectsTo:viewController],
@”The table view’s delegate should be the view controller.”);
}

You’re not saving any code by writing your test this way — you’re actually writing more — but its *complexity* has gone down because it requires less effort to see what it’s actually trying to do.

### Checking Bindings

This is even worthwhile in situations where you may still need a few extra assertions. For example, Cocoa bindings are specified using a lot more information than just outlets and target-acton connections; you won’t always want to check (and specify the value of) all of it, but you can easily make the common parts clearer.

Going back to our Add button example, as is typical its enabled state should be bound to the array controller’s `canAdd` property. Writing a test to specify this involves using `-infoForBinding:` and interpreting the results, which takes a couple lines of code and a couple of assertions:

– (void)testAddButtonEnabledStateIsBoundToArrayControllerCanAdd {
NSDictionary *bindingInfo = [viewController.addButton infoForBinding:NSEnabledBinding];
STAssertNotNil(bindingInfo,
@”The Add button’s enabled state should be bound.”);

id observedObject = [bindingInfo objectForKey:NSObservedObjectKey];
STAssertEquals(observedObject, viewController.arrayController,
@”The Add button’s enabled state should be bound to the array controller.”);

NSString *observedKeyPath = [bindingInfo objectForKey:NSObservedKeyPathKey];
STAssertEqualObjects(observedKeyPath, @”canAdd”,
@”The Add button’s enabled state should be bound through the ‘canAdd’ key path.”);
}

This isn’t too complicated, but it does start to get tedious, especially given that you have to remember to distinguish between `STAssertEquals` (pointer equality) and `STAssertEqualObjects` (object equivalence). Let’s put the tedium in one place:

/*! Tells whether the object’s binding is connected through the given key path. */
– (BOOL)checkObject:(id)source
hasBinding:(NSString *)binding
toObject:(id)destination
through:(NSString *)keyPath
{
NSDictionary *bindingInfo = ;
id observedObject = [bindingInfo objectForKey:NSObservedObjectKey];
NSString *observedKeyPath = [bindingInfo objectForKey:NSObservedKeyPathKey];

return (bindingInfo != nil)
&& (observedObject == destination)
&& [keyPath isEqualToString:observedKeyPath];
}

// in the tests specifying the view controller’s behavior…

– (void)testAddButtonEnabledStateIsBoundToArrayControllerCanAdd {
STAssertTrue([self checkObject:viewController.addButton
hasBinding:NSEnabledBinding
toObject:viewController.arrayController
through:@”canAdd”],
@”The Add button’s enabled state should be bound to the array controller’s ‘canAdd’ property.”);
}

Much clearer!

[1]: https://eschatologist.net/blog/?p=10 “Unit testing Cocoa user interfaces: Target-Action”
[2]: https://eschatologist.net/blog/?p=12 “Unit testing Cocoa user interfaces: Cocoa Bindings”

About a year ago, I wrote about unit testing target-action connections for Cocoa user interfaces. That covers the traditional mechanism by which user interfaces have typically been constructed in Cocoa since the NeXTstep days. However, with the release of Mac OS X 10.3 Panther we’ve had a newer interface technology available — Cocoa bindings — which has presented some interesting application design and testing challenges.

Among other hurdles, to properly use Cocoa bindings in your own applications, you need to ensure that the code you write properly supports key-value coding and key-value observing. However, since the release of Mac OS X 10.4 Tiger, the necessary APIs have been available to easily do test-driven development of your application’s use of Cocoa bindings, following a trust, but verify approach. (It’s also been quite easy from the start to test your support for key-value coding and key-value observing, to ensure that your code meets the necessary prerequisites for supporting bindings. I can write more on this topic in another post if anyone is interested.)

The key to writing unit tests for Cocoa bindings is the -infoForBinding: method in AppKit’s NSKeyValueBindingCreation informal protocol. Using this simple method, you can interrogate any object that has a binding for all of the information about that binding! It simply returns a dictionary with three keys:

1. NSObservedObjectKey, which is the object that the binding is bound to;
2. NSObservedKeyPathKey, which is the key path that is bound — in Interface Builder terms, this is the controller key path combined with the model key path, with a dot in between them; and
3. NSOptionsKey, which is a dictionary of additional binding options unique to the binding. These are all of those additional checkboxes and pop-ups in the Interface Builder bindings inspector for setting things like a value transformer.

By specifying what this dictionary should contain for a particular binding, you can describe the binding itself and thus start doing test-driven development of your Cocoa bindings-based user interface. Note that all of the system-supported binding names — as well as the binding option names — are specified in <AppKit/NSKeyValueBinding.h> and are documented, too!

Let’s take a simple example, like the one in last year’s target-action example, of a window controller whose window has a static text field in it. The field should have its value bound to the name of a person through an object controller for that person. Assume that I’ve already created the test case and set up some internal methods on my window controller to refer to the contents of the window via outlets, and to load the window (without displaying it) in -setUp just like in the target-action example.

First, to see that my text field has a value binding, I might write something like this:

- (void)testPersonNameFieldHasValueBinding {
    NSTextField *personNameField = [_windowController personNameField];

    NSDictionary *valueBindingInfo = [personNameField infoForBinding:NSValueBinding];
    STAssertNotNil(valueBindingInfo,
        @"The person name field's value should be bound.");
}

Of course, this tells us nothing about how the binding should be configured, so it needs some fleshing out…

Let’s check the object and key path for the binding.

- (void)testPersonNameFieldHasValueBinding {
    NSTextField *personNameField = [_windowController personNameField];

    NSDictionary *valueBindingInfo = [personNameField infoForBinding:NSValueBinding];
    STAssertNotNil(valueBindingInfo,
        @"The person name field's value should be bound.");

    NSObjectController *personController = [_windowController personController];
    STAssertEquals([valueBindingInfo objectForKey:NSObservedObjectKey], personController,
        @"The person name field should be bound to the person controller.");

    STAssertEqualObjects([valueBindingInfo objectForKey:NSObservedKeyPathKey], @"name",
        @"The person name field's value should be bound to the 'name' key.");
}

Not very exciting, and a little verbose, but it'll easily lead us through what needs to be set up in Interface Builder for this binding to work. If you want to cut down the verbosity, you can of course extract a method to do the basic checking...

- (BOOL)object:(id)object shouldHaveBinding:(NSString *)binding
            to:(id)boundObject throughKeyPath:(NSString *)keyPath
{
    NSDictionary *info = [object infoForBinding:binding];

    return ([info objectForKey:NSObservedObjectKey] == boundObject)
            && [[info objectForKey:NSObservedKeyPathKey] isEqualToString:keyPath];
}

- (void)testPersonNameFieldHasValueBinding {
    NSTextField *personNameField = [_windowController personNameField];
    NSObjectController *personController = [_windowController personController];
    
    STAssertTrue([self object:personNameField shouldHaveBinding:NSValue
                           to:personController throughKeyPath:@"name"],
    @"Bind person name field's value to the person controller's 'name' key path.");
}

If you're writing code that needs, say, a value transformer, it's a simple matter to extend this model to also check that the correct value transformer class name is specified for the NSValueTransformerNameBindingOption key in the binding options dictionary returned for NSOptionsKey.

You can even extract these kinds of checks into your own subclass of SenTestCase that you use as the basis for all of your application test cases. This will let you write very concise specifications for how your user interface should be wired to the rest of the code, that you can use to just walk through Interface Builder and connect things together — as well as use to ensure that you don't break it accidentally by making changes to other items in Interface Builder.

This is the real power of test-driven development when combined with Cocoa: Because you can trust that the framework will do the right thing as long as it's set up right, you simply need to write tests that specify how your application's interface should be set up. You don't need to figure out how to create events manually, push them through the run loop or through the window's -sendEvent: method, how to deal with showing or not showing the window during tests, or anything like that. Just ensure that your user interface is wired up correctly and Cocoa will take care of the rest.

It’s really great to see that a lot of people are adopting unit testing for their projects and dramatically improving their quality. Test-driven development and agile development methodologies built around it are really taking off. However, a lot of people still feel that their user interface is difficult to test through code, and either requires a capture-playback tool or requires a different design approach based heavily on interfaces/protocols to get right.

In last year’s post Trust, but verify. I tried to dispel some of the mystery of testing your application’s user interface when using the Cocoa frameworks. However, I’ve still had a lot of (entirely well-justified!) requests for examples of how to put it into practice. So here’s a simple example of what I’d do to write a unit test for a button in a window that’s supposed to perform some action.

First, when implementing my window, I’d follow the standard Cocoa pattern of having a custom NSWindowController subclass to manage my window. This window controller will have an outlet connected to each of the views in the window, and will also wind up with a private accessor method — used only within the class and any subclasses, and in testing — for getting the value of each of its outlets. This design flows naturally from the test which I would write to specify that the window should contain a button. First, here’s the skeleton into which I’d put tests:

// TestMyWindow.h

#import <SenTestingKit/SenTestingKit.h>

@class MyWindowController;

@interface TestMyWindow : SenTestCase {
    MyWindowController *_windowController;
    NSWindow *_window;
}
@end

// TestMyWindow.m

#import "TestMyWindow.h"
#import "MyWindowController_Private.h"

@implementation TestMyWindow

- (void)setUp {
    // MyWindowController knows its nib name and
    // invokes -initWithWindowNibName: in -init
    _windowController = [[MyWindowController alloc] init];

    // Load the window, but don't show it.
    _window = [_windowController window];
}

- (void)tearDown {
    [_windowController release];
    _window = nil; // owned by _windowController
}

@end

That’s the infrastructure into which I’d put my other test methods for this window. For example, I’ll want to specify the nib name for the window controller and ensure that it actually knows its window:

- (void)testNibName {
    STAssertEqualObjects([_windowController windowNibName], @"MyWindow",
      @"The nib for this window should be MyWindow.nib");
}

- (void)testWindowLoading {
    STAssertNotNil(_window,
      @"The window should be connected to the window controller.");
}

Now let’s check that I have a “Do Something” button in the window, and that it sends an action directly to the window controller.

- (void)testDoSomethingButton {
    // _doSomethingButton is a private method that returns the button
    // conected to the doSomethingButton outlet
    NSButton *doSomethingButton = [_windowController _doSomethingButton];
    
    STAssertNotNil(doSomethingButton,
      @"The window should have a 'Do something' button.");
    
    STAssertEqualObjects([doSomethingButton title], @"Do Something",
      @"The button should be titled accordingly.");

    STAssertEquals([doSomethingButton action], @selector(doSomething:),
      @"The button should send -doSomething: to its target.");

    STAssertEquals([doSomethingButton target], _windowController,
      @"The button should send its action to the window controller.");
}

You’ll notice something I’m not doing in the above: I’m not simulating interaction with the interface. This is the core of the trust, but verify approach to unit testing of your user interface.

I can trust that as long as I verify everything is hooked up properly that Cocoa will cause the button to send its action message to its target — whether it’s a specific object or, if the target is nil, the responder chain — whenever the button is clicked while it’s enabled and not hidden. I don’t need to simulate a user event, and I don’t even need to display the interface while running the unit tests. All I need to do is inspect, through code, that everything is wired up correctly.

Note that I can do way more than the above in testing my interface design, too. For example, I can ensure that the control layout is correct according to what my interface designer has specified, by checking bounding rectangles for example. But testing only the functionality of my interface has significant advantages, too. For example, it doesn’t matter if I wind up using a custom kind of button to achieve exactly the kind of look and feel or behavior I need. It doesn’t matter if I wind up changing the layout in response to feedback. No matter what I do, I’ll know that functionality won’t accidentally break while I’m messing around in Interface Builder — even if I completely rip out my interface and replace it with a new one!

This approach can also be used for testing Cocoa bindings using the -infoForBinding: method that was introduced in Mac OS X 10.4 Tiger. I hope to write up a post soon on how to approach Cocoa bindings using these same techniques, but it should be fairly straightforward given the above and the above documentation.

Update: I’ve struck through the check of the button’s title above, because you may or may not want to do that. For example, if you’re primarily running your unit tests against your development localization, you may want to put it in. But if you want to run your unit tests against a localized build of your application, you’ll probably want to avoid checking a localized title against an English string. A “have your cake and eat it too” strategy might be to keep a variable somewhere in your application that can be used to selectively disable checks of only localized strings.

Update July 7, 2007: I’ve finally written a post, Unit testing Cocoa user interfaces: Cocoa bindings, on how to write tests for Cocoa bindings. Now there’s no excuse for not doing test-driven development of your Cocoa user interfaces!