Unit testing Cocoa user interfaces: Use Check Methods

In the past, I’ve talked about ways to easily write unit tests for Cocoa applications, including [tests for user interfaces using target-action][1] and [tests for interfaces using Cocoa bindings][2]. There are some strategies you can apply to make writing tests for Cocoa code even easier, though. They’re just straightforward object-oriented programming, but sometimes we can forget that all the techniques you might use in your main code base can also be applied to our test code.

So here’s one trick that you can use in writing tests for Cocoa user interfaces, especially in ways that will make test-driven development easier.

### Use a Common Base Class with Check Methods

The first, probably most important thing to do is *use your own common base class* for your tests. Don’t derive your tests directly from `SenTestCase`; instead, derive them from your own `MyTestCase` class that in turn derives from `SenTestCase`. This gives you a place to put all of the customization that’s appropriate for your project.

Sometimes you might need a series of assertions to verify some particular state. However, that series of assertions will be the same every time you need to verify that state. Or the assertions themselves aren’t very *intention revealing* so you always wind up putting a comment above them describing what they’re really doing.

### Checking Target-Action

A simple example of this is checking a target-action connection in a user interface. Say you have a view controller that presents a collection of objects managed by an array controller. Its view has an Add button that should send `-addObject:` to the array controller. You might write a test for it like this:

– (void)testAddButtonSendsAddObjectToArrayController {
STAssertEquals([viewController.addButton target], viewController.arrayController,
@”The Add button should target the array controller.”);
STAssertEquals([viewController.addButton action], @selector(addObject:),
@”The Add button should send the -addObject: action.”);

That’s not too difficult to understand, but it could be made simpler — it could be done in a single assertion. You’d just write a method to check both the target and action at once and then use that method from your test, like this:

// in your test base class…

/*! Tells whether the control sends the action to the target. */
– (BOOL)checkControl:(NSControl *)control
return ([control action] == action)
&& ([control target] == target);

// in the tests specifying the view controller’s behavior…

– (void)testAddButtonSendsAddObjectToArrayController {
STAssertTrue([self checkControl:viewController.addButton
@”The Add button’s action should send -addObject: to the array controller.”);

That makes the intention behind the entire test a lot clearer, and it makes writing the test easier & safer since you can’t (say) forget to check either the target or the action.

It *does* lose a tiny bit of information: If the test fails, you’ll have to look at your xib file instead of the failure message to determine whether it’s because the target or the action isn’t set as you’ve specified. However, the trade-off in making the test easier to read and write is worth it here.

### Checking Outlets

This is even worthwhile for single assertions, such as those you’d use to test that your outlets are connected in the first place. For example, you might initially write a test that your view controller is your table view’s delegate like this:

– (void)testViewControllerIsTableViewDelegate {
STAssertEquals([viewController.tableView delegate], viewController,
@”The table view’s delegate should be the view controller.”);

Rewriting it to be more intention-revealing with a simple check method would make it look like this:

// in your test base class…

/*! Tells whether the outlet is connected to the given destination. */
– (BOOL)checkOutlet:(id)outlet connectsTo:(id)destination {
return outlet == destination;

// in the tests specifying the view controller’s behavior…

– (void)testViewControllerIsTableViewDelegate {
STAssertTrue([self checkOutlet:[viewController.tableView delegate]
@”The table view’s delegate should be the view controller.”);

You’re not saving any code by writing your test this way — you’re actually writing more — but its *complexity* has gone down because it requires less effort to see what it’s actually trying to do.

### Checking Bindings

This is even worthwhile in situations where you may still need a few extra assertions. For example, Cocoa bindings are specified using a lot more information than just outlets and target-acton connections; you won’t always want to check (and specify the value of) all of it, but you can easily make the common parts clearer.

Going back to our Add button example, as is typical its enabled state should be bound to the array controller’s `canAdd` property. Writing a test to specify this involves using `-infoForBinding:` and interpreting the results, which takes a couple lines of code and a couple of assertions:

– (void)testAddButtonEnabledStateIsBoundToArrayControllerCanAdd {
NSDictionary *bindingInfo = [viewController.addButton infoForBinding:NSEnabledBinding];
@”The Add button’s enabled state should be bound.”);

id observedObject = [bindingInfo objectForKey:NSObservedObjectKey];
STAssertEquals(observedObject, viewController.arrayController,
@”The Add button’s enabled state should be bound to the array controller.”);

NSString *observedKeyPath = [bindingInfo objectForKey:NSObservedKeyPathKey];
STAssertEqualObjects(observedKeyPath, @”canAdd”,
@”The Add button’s enabled state should be bound through the ‘canAdd’ key path.”);

This isn’t too complicated, but it does start to get tedious, especially given that you have to remember to distinguish between `STAssertEquals` (pointer equality) and `STAssertEqualObjects` (object equivalence). Let’s put the tedium in one place:

/*! Tells whether the object’s binding is connected through the given key path. */
– (BOOL)checkObject:(id)source
hasBinding:(NSString *)binding
through:(NSString *)keyPath
NSDictionary *bindingInfo = ;
id observedObject = [bindingInfo objectForKey:NSObservedObjectKey];
NSString *observedKeyPath = [bindingInfo objectForKey:NSObservedKeyPathKey];

return (bindingInfo != nil)
&& (observedObject == destination)
&& [keyPath isEqualToString:observedKeyPath];

// in the tests specifying the view controller’s behavior…

– (void)testAddButtonEnabledStateIsBoundToArrayControllerCanAdd {
STAssertTrue([self checkObject:viewController.addButton
@”The Add button’s enabled state should be bound to the array controller’s ‘canAdd’ property.”);

Much clearer!

[1]: https://eschatologist.net/blog/?p=10 “Unit testing Cocoa user interfaces: Target-Action”
[2]: https://eschatologist.net/blog/?p=12 “Unit testing Cocoa user interfaces: Cocoa Bindings”

Singletons in Cocoa/Objective-C

I’ll preface this post with the standard advice: *Don’t create singletons if you don’t absolutely have to.* In general, if you’re creating a global “manager” object of some sort, you’re doing something wrong.

That said, there’s still occasionally a reason to have such a global singleton, such as a “default something.” The sample code in the Cocoa Fundamentals Guide goes to a lot more trouble than it needs to in order to *ensure* that a class is a singleton.

This is almost **never** what you want.

First off, you probably want your class to be testable in a variety of configurations. In your unit tests, instead of getting your shared singleton instance in your `-setUp` method and “resetting” its state in `-tearDown`, you’d be better off just instantiating a *new* instance in `-setUp` and *releasing* it in `-tearDown`.

Also, the example in the Cocoa Fundamentals Guide does a lot of work that it simply doesn’t need to. This is all you *really* need to do to create a singleton in Cocoa:

@interface SomeManager : NSObject
+ (id)sharedManager;

@implementation SomeManager

+ (id)sharedManager {
static id sharedManager = nil;

if (sharedManager == nil) {
sharedManager = [[self alloc] init];

return sharedManager;


That’s it! The astute reader will notice, of course, that this isn’t thread-safe. I got rid of the `@synchronized (self)` because it won’t do the right thing; depending on what actual class is sent `+sharedManager`, the value of `self` will be different!

For the sake of argument, though, let’s say that you do want a singleton with which you can interact from multiple threads at once. One way to do this would be to create your singleton instance in `+initialize` since it will always be run, on a single thread, before any other methods in your class:

@implementation SomeManager

static id sharedManager = nil;

+ (void)initialize {
if (self == [SomeManager class]) {
sharedManager = [[self alloc] init];

+ (id)sharedManager {
return sharedManager;


By doing this, you avoid the performance bottleneck of `@synchronized` taking a recursive lock every time `+sharedManager` is invoked.

If you want to get fancier, and it’s OK to temporarily have more than one instance of your singleton created, you could even use `objc_atomicCompareAndSwapGlobalBarrier` to assign the value to return from `+sharedManager`, though this is probably also more work than it’s worth; after all, `+initialize` will only be invoked once for your class. (Though it can be re-invoked as a side-effect of initializing subclasses, hence the `if (self == [SomeManager class]) { }` idiom.)

In all of the above cases, you’ve done a whole lot less work than the example in the Cocoa Fundamentals Guide, and your code is a lot more likely to be correct as a result.

Unit testing Cocoa user interfaces: Cocoa Bindings

About a year ago, I wrote about unit testing target-action connections for Cocoa user interfaces. That covers the traditional mechanism by which user interfaces have typically been constructed in Cocoa since the NeXTstep days. However, with the release of Mac OS X 10.3 Panther we’ve had a newer interface technology available — Cocoa bindings — which has presented some interesting application design and testing challenges.

Among other hurdles, to properly use Cocoa bindings in your own applications, you need to ensure that the code you write properly supports key-value coding and key-value observing. However, since the release of Mac OS X 10.4 Tiger, the necessary APIs have been available to easily do test-driven development of your application’s use of Cocoa bindings, following a trust, but verify approach. (It’s also been quite easy from the start to test your support for key-value coding and key-value observing, to ensure that your code meets the necessary prerequisites for supporting bindings. I can write more on this topic in another post if anyone is interested.)

The key to writing unit tests for Cocoa bindings is the -infoForBinding: method in AppKit’s NSKeyValueBindingCreation informal protocol. Using this simple method, you can interrogate any object that has a binding for all of the information about that binding! It simply returns a dictionary with three keys:

  1. NSObservedObjectKey, which is the object that the binding is bound to;
  2. NSObservedKeyPathKey, which is the key path that is bound — in Interface Builder terms, this is the controller key path combined with the model key path, with a dot in between them; and
  3. NSOptionsKey, which is a dictionary of additional binding options unique to the binding. These are all of those additional checkboxes and pop-ups in the Interface Builder bindings inspector for setting things like a value transformer.

By specifying what this dictionary should contain for a particular binding, you can describe the binding itself and thus start doing test-driven development of your Cocoa bindings-based user interface. Note that all of the system-supported binding names — as well as the binding option names — are specified in <AppKit/NSKeyValueBinding.h> and are documented, too!

Let’s take a simple example, like the one in last year’s target-action example, of a window controller whose window has a static text field in it. The field should have its value bound to the name of a person through an object controller for that person. Assume that I’ve already created the test case and set up some internal methods on my window controller to refer to the contents of the window via outlets, and to load the window (without displaying it) in -setUp just like in the target-action example.

First, to see that my text field has a value binding, I might write something like this:

- (void)testPersonNameFieldHasValueBinding {
    NSTextField *personNameField = [_windowController personNameField];

    NSDictionary *valueBindingInfo = [personNameField infoForBinding:NSValueBinding];
        @"The person name field's value should be bound.");

Of course, this tells us nothing about how the binding should be configured, so it needs some fleshing out…

Let’s check the object and key path for the binding.

- (void)testPersonNameFieldHasValueBinding {
    NSTextField *personNameField = [_windowController personNameField];

    NSDictionary *valueBindingInfo = [personNameField infoForBinding:NSValueBinding];
        @"The person name field's value should be bound.");

    NSObjectController *personController = [_windowController personController];
    STAssertEquals([valueBindingInfo objectForKey:NSObservedObjectKey], personController,
        @"The person name field should be bound to the person controller.");

    STAssertEqualObjects([valueBindingInfo objectForKey:NSObservedKeyPathKey], @"name",
        @"The person name field's value should be bound to the 'name' key.");

Not very exciting, and a little verbose, but it'll easily lead us through what needs to be set up in Interface Builder for this binding to work. If you want to cut down the verbosity, you can of course extract a method to do the basic checking...

- (BOOL)object:(id)object shouldHaveBinding:(NSString *)binding
            to:(id)boundObject throughKeyPath:(NSString *)keyPath
    NSDictionary *info = [object infoForBinding:binding];

    return ([info objectForKey:NSObservedObjectKey] == boundObject)
            && [[info objectForKey:NSObservedKeyPathKey] isEqualToString:keyPath];

- (void)testPersonNameFieldHasValueBinding {
    NSTextField *personNameField = [_windowController personNameField];
    NSObjectController *personController = [_windowController personController];
    STAssertTrue([self object:personNameField shouldHaveBinding:NSValue
                           to:personController throughKeyPath:@"name"],
    @"Bind person name field's value to the person controller's 'name' key path.");

If you're writing code that needs, say, a value transformer, it's a simple matter to extend this model to also check that the correct value transformer class name is specified for the NSValueTransformerNameBindingOption key in the binding options dictionary returned for NSOptionsKey.

You can even extract these kinds of checks into your own subclass of SenTestCase that you use as the basis for all of your application test cases. This will let you write very concise specifications for how your user interface should be wired to the rest of the code, that you can use to just walk through Interface Builder and connect things together — as well as use to ensure that you don't break it accidentally by making changes to other items in Interface Builder.

This is the real power of test-driven development when combined with Cocoa: Because you can trust that the framework will do the right thing as long as it's set up right, you simply need to write tests that specify how your application's interface should be set up. You don't need to figure out how to create events manually, push them through the run loop or through the window's -sendEvent: method, how to deal with showing or not showing the window during tests, or anything like that. Just ensure that your user interface is wired up correctly and Cocoa will take care of the rest.

Unit testing Cocoa user interfaces: Target-Action

It’s really great to see that a lot of people are adopting unit testing for their projects and dramatically improving their quality. Test-driven development and agile development methodologies built around it are really taking off. However, a lot of people still feel that their user interface is difficult to test through code, and either requires a capture-playback tool or requires a different design approach based heavily on interfaces/protocols to get right.

In last year’s post Trust, but verify. I tried to dispel some of the mystery of testing your application’s user interface when using the Cocoa frameworks. However, I’ve still had a lot of (entirely well-justified!) requests for examples of how to put it into practice. So here’s a simple example of what I’d do to write a unit test for a button in a window that’s supposed to perform some action.

First, when implementing my window, I’d follow the standard Cocoa pattern of having a custom NSWindowController subclass to manage my window. This window controller will have an outlet connected to each of the views in the window, and will also wind up with a private accessor method — used only within the class and any subclasses, and in testing — for getting the value of each of its outlets. This design flows naturally from the test which I would write to specify that the window should contain a button. First, here’s the skeleton into which I’d put tests:

// TestMyWindow.h

#import <SenTestingKit/SenTestingKit.h>

@class MyWindowController;

@interface TestMyWindow : SenTestCase {
    MyWindowController *_windowController;
    NSWindow *_window;

// TestMyWindow.m

#import "TestMyWindow.h"
#import "MyWindowController_Private.h"

@implementation TestMyWindow

- (void)setUp {
    // MyWindowController knows its nib name and
    // invokes -initWithWindowNibName: in -init
    _windowController = [[MyWindowController alloc] init];

    // Load the window, but don't show it.
    _window = [_windowController window];

- (void)tearDown {
    [_windowController release];
    _window = nil; // owned by _windowController


That’s the infrastructure into which I’d put my other test methods for this window. For example, I’ll want to specify the nib name for the window controller and ensure that it actually knows its window:

- (void)testNibName {
    STAssertEqualObjects([_windowController windowNibName], @"MyWindow",
      @"The nib for this window should be MyWindow.nib");

- (void)testWindowLoading {
      @"The window should be connected to the window controller.");

Now let’s check that I have a “Do Something” button in the window, and that it sends an action directly to the window controller.

- (void)testDoSomethingButton {
    // _doSomethingButton is a private method that returns the button
    // conected to the doSomethingButton outlet
    NSButton *doSomethingButton = [_windowController _doSomethingButton];
      @"The window should have a 'Do something' button.");
    STAssertEqualObjects([doSomethingButton title], @"Do Something",
      @"The button should be titled accordingly.");

    STAssertEquals([doSomethingButton action], @selector(doSomething:),
      @"The button should send -doSomething: to its target.");

    STAssertEquals([doSomethingButton target], _windowController,
      @"The button should send its action to the window controller.");

You’ll notice something I’m not doing in the above: I’m not simulating interaction with the interface. This is the core of the trust, but verify approach to unit testing of your user interface.

I can trust that as long as I verify everything is hooked up properly that Cocoa will cause the button to send its action message to its target — whether it’s a specific object or, if the target is nil, the responder chain — whenever the button is clicked while it’s enabled and not hidden. I don’t need to simulate a user event, and I don’t even need to display the interface while running the unit tests. All I need to do is inspect, through code, that everything is wired up correctly.

Note that I can do way more than the above in testing my interface design, too. For example, I can ensure that the control layout is correct according to what my interface designer has specified, by checking bounding rectangles for example. But testing only the functionality of my interface has significant advantages, too. For example, it doesn’t matter if I wind up using a custom kind of button to achieve exactly the kind of look and feel or behavior I need. It doesn’t matter if I wind up changing the layout in response to feedback. No matter what I do, I’ll know that functionality won’t accidentally break while I’m messing around in Interface Builder — even if I completely rip out my interface and replace it with a new one!

This approach can also be used for testing Cocoa bindings using the -infoForBinding: method that was introduced in Mac OS X 10.4 Tiger. I hope to write up a post soon on how to approach Cocoa bindings using these same techniques, but it should be fairly straightforward given the above and the above documentation.

Update: I’ve struck through the check of the button’s title above, because you may or may not want to do that. For example, if you’re primarily running your unit tests against your development localization, you may want to put it in. But if you want to run your unit tests against a localized build of your application, you’ll probably want to avoid checking a localized title against an English string. A “have your cake and eat it too” strategy might be to keep a variable somewhere in your application that can be used to selectively disable checks of only localized strings.

Update July 7, 2007: I’ve finally written a post, Unit testing Cocoa user interfaces: Cocoa bindings, on how to write tests for Cocoa bindings. Now there’s no excuse for not doing test-driven development of your Cocoa user interfaces!

Making Better Games with Test-Driven Development

Noel Llopis (Games from Within) and Sean Houghton, Backwards Is Forward: Making Better Games with Test-Driven Development:

> One of the questions we had when we jumped into TDD is
> whether it was going to hold for high-level code. We had
> seen in practice from previous projects that we can
> certainly do TDD to create low-level and intermediate-level
> libraries (math, collision, messaging, etc). But would it
> really work for high-level code that would build on
> low-level code?
> The answer is an unconditional yes. We have developed a
> full codebase doing TDD from the start, and we had no
> difficulty writing high-level code with TDD. Things like
> character state machines, game flow, or specific game
> entities were done through TDD without any problems, and
> greatly benefited from the TDD approach.

Noel’s blog is great, and this paper is being presented at the 2006 Game Developers Conference.

Noel and High Moon Studios have been a lot of pioneering work using Extreme Programming in the game development space. Game development can definitely make good use of Extreme Programming to manage the development process, dramatically improve the quality of code, and do wonders for scheduling accuracy and schedule predictability.

Extreme Programming in game development also presents special challenges due to the exploratory nature of a lot of the work, as well as the highly-interactive nature of the software itself. As Noel points out, you wind up writing much more finely-factored code when doing TDD, which will be alien to a lot of game developers but will help greatly with maintenance, debugging, and the ever-more-important portability. (Not just between Windows and Mac OS X, but also between Xbox and PlayStation 2 and Nintendo GameCube and Xbox 360 and PlayStation 3 and Nintendo Revolution…)

Xcode: Debugging Cocoa application unit tests

A couple weeks ago as part of my Unit Testing Series I talked about how to use Xcode to write unit tests for Cocoa frameworks, debug unit tests in Cocoa frameworks, and write unit tests for Cocoa applications. However, I haven’t yet described how to debug your unit tests in Objective-C Cocoa applications. I’ll take care of that tonight.

After you’ve set up unit testing in your Cocoa application, debugging your unit tests is similar to debugging them in a Cocoa framework. All you have to do is adjust the arguments and environment variables your application’s Executable is configured to use in Xcode. You don’t even have to create a new executable.

To start, bring up the Info window for your application’s executable (which is its entry in Xcode’s Executable smart group). In the Arguments tab, add the argument -SenTest All. This tells the unit testing infrastructure that you want to run all of the unit tests, not just the ones that are built in to the executable. (After all, you don’t have any unit tests in your executable itself.)

Now we’ll need to engage in a little bit of environment variable magic. When you test an application, what you’re really doing is injecting your unit test bundle into the application and telling it to run its tests at its first opportunity. This is accomplished through by telling dyld to insert a private framework, DevToolsBundleInjection.framework, into your application on launch, and telling that framework via an environment variable, XCInjectBundle, the path of the test bundle to inject.

You also have to tell the injection framework itself the full path to the application executable you want to inject the bundle into, via the XCInjectBundleInto environment variable. This is needed to avoid injecting your test bundle into other executables that are run by the application you’re testing, or that are run as a side-effect of running the application you’re testing. (For example, gdb generally wants to run applications from within a shell, so that environment variables are expanded in its environment and command-line parameters.)

In the Arguments tab of your application executable, first add an environment variable named DYLD_INSERT_LIBRARIES. Set its value to the path to the DevToolsBundleInjection.framework/DevToolsBundleInjection library in the framework of the same name that’s included in the developer tools. Prior to Xcode 2.5, this was in $(SYSTEM_LIBRARY_DIR)/PrivateFrameworks but as of Xcode 2.5 and Xcode 3.0, it has been moved to $(DEVELOPER_LIBRARY_DIR)/PrivateFrameworks

Then add a second environment variable, XCInjectBundle. Set its value to $(BUILT_PRODUCTS_DIR)/MyTestBundle.octest.

Add a third environment variable, XCInjectBundleInto. Set its value to the full path to your application’s executable — not just the application bundle — e.g. $(BUILT_PRODUCTS_DIR)/MyApplication.app/Contents/MacOS/MyApplication. This is the debugging equivalent of the Test Host build setting you used to tell Xcode what executable to inject your tests into when running them.

For Xcode 3.0 and later, add a final environment variable, DYLD_FALLBACK_FRAMEWORK_PATH to your executable. Set its value to $(DEVELOPER_LIBRARY_DIR)/Frameworks.

Why do you need to do this? In order to support moving and renaming the Developer folder, all of the frameworks within it — including OCUnit — use runpath search paths. This means that the internal name of the framework, including the one copied into your test bundle, will start with @rpath rather than an absolute path starting with /Developer/Library/Frameworks. Unfortunately this means that your unit tests won’t find SenTestingKit.framework without some extra help. That’s what DYLD_FALLBACK_FRAMEWORK_PATH does: It tells dyld to try an additional set of directories in place of @rpath when it can’t be resolved. (More information on runpath-relative install names can be found in the ld(1) man page.)

Make sure the check marks next to all three of these environment variables — and your -SenTest All argument, of course — are set.

Troubleshooting note: Troubleshooting note: If this doesn’t work — that is, if your test bundle isn’t found and run — change the executable’s working directory (in the General tab) to Built Products Directory and remove $(BUILT_PRODUCTS_DIR) above. Generally this is caused by $(BUILT_PRODUCTS_DIR) not being expanded to a full path, but rather to a partial path relative to your project directory.

Now if you choose Run Executable from the Debug menu, your application should launch, you should see the results of executing your unit tests in the Run Log, and as soon as your unit tests are complete your application should quit!

To debug a failing test, build your tests and set a breakpoint on the line where the failure occurs. Now choose Debug Executable from the Debug menu. As with a Cocoa framework, do not choose Build and Debug from the Build menu. You need to use Debug Executable because your build will fail due to the failing test. Debug Executable will work as long as your executable itself is actually present.

Having done all this, you should be stopped at the breakpoint!

Just as any other time you use OCUnit, instead of -SenTest All you can specify -SenTest MyTestCaseClassName to run just the tests in the test case class MyTestCaseClassName, or -SenTest MyTestCaseClassName/testMethodName to run just a single test.

Update July 7, 2007: Added the troubleshooting note about removing $(BUILT_PRODUCTS_DIR) if you get errors about not being able to load the bundle.

Update March 17, 2008: I’ve updated this a bit to handle some changes in the process introduced with Xcode 3.0.

Update September 5, 2008: I’ve updated this again to cover the changes that were made to bundle injection for Xcode 3.1; the change is the introduction of the XCInjectBundleInto environment variable.

Xcode: Unit Testing Cocoa Applications

Yesterday, I talked about how to add unit tests to Cocoa frameworks using Xcode. There’s only a little more set-up required to add tests to Objective-C Cocoa applications.

First, turn off ZeroLink for the application target you want to test. Just as with a framework, your unit tests will be built as a test bundle containing only the tests, completely separate from the code being tested. The test bundle will access the code in your application by linking against it, and you can’t link against something built with ZeroLink. Note: You only need to do this for Xcode 2.1 through Xcode 2.5. Xcode 3.0 removed support for ZeroLink, since the linker is now sufficiently fast as to obviate it.

Next, add a new Cocoa Unit Test Bundle target to your application project. This is the target that will actually contain your tests. When you add the test bundle target, Xcode shows its Info window. In the General tab of this window, press the + button and choose your application target from the sheet that appears. This will make your test bundle target depend on your application target, so you can just build your test bundle all the time and have your application automatically rebuild first.

Now you actually need to make your test bundle link against your application. You do this using the Bundle Loader property under Linker Options in your test bundle target’s configurations. You need to set the value of this option to the path of your application executable and not just its wrapper: $(BUILT_PRODUCTS_DIR)/MyApplication.app/Contents/MacOS/MyApplication. The $(BUILT_PRODUCTS_DIR) variable is expanded at build time to be the path to the build products for the currently-selected configuration, which lets you avoid using an absolute path.

The final step is to tell the unit testing infrastructure how to run your tests. For a framework, the dynamic loader does the heavy lifting; when the test rig loads your test bundle, the loader will automatically load the frameworks it depends on including the one you’re testing. However, for an application to be tested the application must be launched and the test bundle injected into it. You can specify that this should happen using the Test Host property under Unit Testing in your test bundle target’s configurations. Just as with the Bundle Loader property, you need to set the value to the full path of your application executable. Since you’ve already done that once, you can just re-use the value of the Bundle Loader setting by specifying $(BUNDLE_LOADER) — this is the underlying variable that the property is associated with.

Now just like with frameworks you can add test cases to your test bundle just by adding new files based on the Cocoa Objective-C test case class template. Your application code can remain in your application target and your unit testing code can remain in your test bundle target. Whenever you build your test bundle target, once everything is built your application will launch, its tests will be run, and it will quit — and the test results will be reported via the Build Results window just like compiler and linker errors.

Xcode: Debugging Cocoa framework unit tests

So you’ve set up unit testing for your Objective-C Cocoa framework and it’s been working great. But now you’ve written a test and it fails, and you can’t figure out why. It’s time to break out the debugger, but how, exactly, do you do that? Your unit tests are built as a bundle, and you can’t run a bundle.

It’s simple. All you have to do is set up an appropriate Executable in Xcode to run the test rig that runs your bundle, and then debug that. To get started, choose the Project > New Custom Executable menu item. For its name, specify otest — this is the name of the test rig used by OCUnit. Specify /Developer/Tools/otest as the path to your tool.

When you add the custom executable, Xcode will bring up its info window. Switch to the Arguments tab. Here you’ll need to enter some command-line arguments to pass to the test rig and some environment variables to affect how it’s run.

First, add the argument -SenTest Self to your executable. This tells otest to run all of the tests in your bundle. (It’s actually a pair of arguments, but you can add it as one as a convenience.) Then for your second argument specify $(BUILT_PRODUCTS_DIR)/MyTestBundle.octest where MyTestBundle is the name of your test bundle. This tells otest the path of the test bundle to load; $(BUILT_PRODUCTS_DIR) will be expanded to the appropriate build products directory for your project at run time. (If you get the order wrong, just drag the arguments around in the table.)

Troubleshooting note: If this doesn’t work — that is, if otest complains it can’t find your test bundle — change the executable’s working directory (in the General tab) to Built Products Directory and remove $(BUILT_PRODUCTS_DIR) above. Generally this is caused by $(BUILT_PRODUCTS_DIR) not being expanded to a full path, but rather to a partial path relative to your project directory.

Next add a pair of environment variables named DYLD_FRAMEWORK_PATH and DYLD_LIBRARY_PATH to your executable. These will tell the loader to check your build products directory first for frameworks and libraries whenever the executable is run from within Xcode. Specify $(BUILT_PRODUCTS_DIR) for the value of each variable.

Finally, from the Project > Set Active Executable menu choose your new otest executable. This way any time you run or debug within Xcode, it will run otest with the arguments and environment you’ve specified.

To actually debug a failing test, build your tests and set a breakpoint on the line where the failure occurs. Now choose Debug Executable from the Debug menu. Do not choose Build and Debug from the Build menu. You need to use Debug Executable because your build will fail due to the failing test; Debug Executable doesn’t try to build first, it only cares whether an executable is present.

You should successfully stop at your breakpoint!

If your tests take a long time to run — they shouldn’t, they’re unit tests after all, but it can still happen — you may want to just run the tests for one test case, or just one individual test. This is easy too. Rather than specifying -SenTest Self in your arguments to otest, you can specify -SenTest MyTestCaseClassName to run the all the tests in the specified test case. To run just a single test, use -SenTest MyTestCaseClassName/testMethodName instead.

Update July 7, 2007: Added the troubleshooting note about removing $(BUILT_PRODUCTS_DIR) if you get errors about not being able to load the bundle.

Xcode: Unit Testing Cocoa frameworks

It’s straightforward to write unit tests for Objective-C Cocoa frameworks with Xcode 2.1 and later.

First, turn off ZeroLink when building your framework. ZeroLink is a great technology, but you can’t link against something that’s built with ZeroLink, and that’s exactly what your unit tests are going to do. Note: You only need to do this for Xcode 2.1 through Xcode 2.5. Xcode 3.0 removed support for ZeroLink, since the linker is now sufficiently fast as to obviate it.

Next, add a new Cocoa Unit Test Bundle target to your framework project. This is the target that will actually contain your tests.

When you add the new test bundle target Xcode should bring up its Info window. Switch to its General tab and press the + button in the lower-left to add a new dependency. In the resulting sheet, choose your framework target. This tells the build system that your framework target needs to be built before your test bundle target.

Now that you’ve declared the dependency between the targets, you need to make sure your test bundle actually links against your framework. Make your test bundle target the active target by choosing it from the Project > Set Active Target submenu. Then highlight the Products group in the Groups & Files view at the left of the Xcode project window. In the detail view to the right, you’ll see all of the products that are built by the various targets in your project; one of the rows will be for your framework. Click the checkbox at the very right of its row, in the Target Membership column, to make your test bundle link against your framework.

Now you can add test cases to your test bundle by just adding new files based on the Cocoa Objective-C test case class template. You don’t need to add any of your framework code to your test bundle, and you don’t need to add any testing code to your framework. And you’re not restricted to using only header files with public visibility in tests, either; you should be able to use anything from your framework target, so long as it’s exported at link time.

Trust, but verify.

President Reagan, for all his faults, gave us a very useful aphorism in describing his approach to diplomacy with the Soviet Union: “Trust, but verify.” This is also a very useful approach to take when writing unit tests when you’re working with a framework, particularly when you’re developing a human interface.

For example, Cocoa uses the target-action pattern for controls. When you create an NSButton you can specify which object to send a message to when it’s clicked — the target — and what message to send — the action.

You can trust that when you click the button, Cocoa will cause the action message to be sent to the target object so long as they have been properly specified. Therefore you probably don’t need to write a unit test that simulates clicking on the button. However, you should verify that your button has had its target and action properly specified, and you can write a test for this.

You can also apply this principal in your own code. Let’s say you’re implementing a new type of control that also has to follow the target-action pattern. In the tests for your control itself, you probably will want to simulate the appropriate user-interface events and see that a testing instance of your control behaves appropriately. However, you don’t need to do this in the code that uses the control — you can trust that the control behaves correctly due to the tests you wrote for the control itself, and just verify that it’s been properly configured.