Asynchronous testing for iOS

After a long time thinking and talking about it, I’ve started to add some unit testing to WordPress for iOS. Not long after I started, I hit my first roadblock: apparently the included ¬†SenTestingKit framework doesn’t support asynchronous testing.

I had a piece of test code like this:


[_blog validateJetpackUsername:@"test1" password:@"test1" success:^{
STFail(@"User test1 shouldn't have access to test.blog");
} failure:^(NSError *error) {
STAssertEquals(error.domain, BlogJetpackErrorDomain, nil);
STAssertEquals(error.code, BlogJetpackErrorCodeNoRecordForBlog, nil);
}];

Soon I noticed those assertions were never being triggered, and the reason is the test finished before those callback blocks were called.

After a bit of googling I came across SenTestingKit in Xcode 4: Asynchronous testing?, and tried GHUnit, since it supports asynchronous testing. But I gave up too quickly.

GHUnit is not a drop-in replacement for SenTestingKit, and while the assertion macros are similar, the way of running the tests is not and I was happy with SenTestingKit so far.

This is how the GHUnit runner looks
This is how the GHUnit runner looks

So I went for the next best thing: using semaphores to wait for those blocks. And I wrote some helper macros for this


//
// AsyncTestHelper.h
// WordPress
//
// Created by Jorge Bernal on 2/12/13.
// Copyright (c) 2013 WordPress. All rights reserved.
//
#import <Foundation/Foundation.h>
#import <SenTestingKit/SenTestingKit.h>
extern dispatch_semaphore_t ATHSemaphore;
extern const NSTimeInterval AsyncTestCaseDefaultTimeout;
#define ATHStart() do {\
ATHSemaphore = dispatch_semaphore_create(0);\
} while (0)
#define ATHNotify() do {\
dispatch_semaphore_signal(ATHSemaphore);\
} while (0)
#define ATHWait() do {\
BOOL timedOut;\
NSDate *timeoutDate = [NSDate dateWithTimeIntervalSinceNow:AsyncTestCaseDefaultTimeout];\
long lockStatus = 0;\
while ((lockStatus = dispatch_semaphore_wait(ATHSemaphore, DISPATCH_TIME_NOW)) && [timeoutDate compare:[NSDate date]] == NSOrderedDescending)\
[[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode\
beforeDate:[NSDate dateWithTimeIntervalSinceNow:10]];\
timedOut = (lockStatus != 0);\
dispatch_release(ATHSemaphore);\
STAssertFalse(timedOut, @"Lock timed out");\
} while (0)


//
// AsyncTestHelper.m
// WordPress
//
// Created by Jorge Bernal on 2/12/13.
// Copyright (c) 2013 WordPress. All rights reserved.
//
#import "AsyncTestHelper.h"
dispatch_semaphore_t ATHSemaphore;
const NSTimeInterval AsyncTestCaseDefaultTimeout = 10;

And the new (working) test looks like this:


ATHStart();
[_blog validateJetpackUsername:@"test1" password:@"test1" success:^{
STFail(@"User test1 shouldn't have access to test.blog");
ATHNotify();
} failure:^(NSError *error) {
STAssertEquals(error.domain, BlogJetpackErrorDomain, nil);
STAssertEquals(error.code, BlogJetpackErrorCodeNoRecordForBlog, nil);
ATHNotify();
}];
ATHWait();