Simon would not be much good if it could not tell you how the stories went. At this time, Simon does two things – Firstly it logs the results of the run to XCode’s console via NSLog(…) statements. Secondly it displays a View on top of your app where you can view and re-run tests.

Here’s an example of the log output from a test run:

Simon's run report
====================================================
 
Story: Goto second page
   Step: "Given I am on the first page" (BDDImplementations::startOnFirstScreen) - Success
   Step: "Then goto the second page" (BDDImplementations::gotoSecondScreen) - Success
Result: Success
 
Story: Test Tapping a button
   Step: "Given I am on the second page" (BDDImplementations::startOnSecondScreen) - Success
   Step: "Then I can tap a button" (BDDImplementations::tapSecondPageHelloButton) - Success
   Step: "and see "hello" in the label" (BDDImplementations::verifyLabel:) - Success
Result: Success
 
Story: Entering some text
   Step: "Given I am on the first page" (BDDImplementations::startOnFirstScreen) - Success
   Step: "then I can enter "abc" into the name field" (BDDImplementations::enterTextIntoNameField:) - Failed!
Result: Failed: Exception caught: Path //UIKBKeyplaneView failed to find anything.
 
Story: SIFail macro handles custom message
   Step: Given the interface is up, not mapped
   Step: "and call the SIFail macro with a message" (MacroStoryImplementations::doSIFailWithMessage) - Not executed
Result: Not mapped
 
Story: SIFail macro passes exception back to story
   Step: Given the interface is up, not mapped
   Step: "and call the SIFail macro" (MacroStoryImplementations::doSIFail) - Not executed
Result: Not mapped
  
Story: SIAssertNil should pass when not nil
   Step: Given the interface is up, not mapped
   Step: "and reset test flags" (MacroStoryImplementations::setUp) - Not executed
   Step: and call the SIAssertNil macro with "abc", not mapped
   Step: "and check the flags" (MacroStoryImplementations::checkFlags) - Not executed
Result: Not mapped
  
Story: SIAssertNil should throw an exception
   Step: "given reset test flags" (MacroStoryImplementations::setUp) - Success
   Step: "then calling SIAssertNil with nil should work" (MacroStoryImplementations::doSIAssertNilShouldPassNilOk) - Success
Result: Success
 
Final Report:
Total stories    : 7
Not run          : 0
Not fully mapped : 3
Successfully run : 3
Ignored          : 0
Failures         : 1

The in-app results UI

When Simon’s UI is enabled with the -ui argument, the UI is automatically displayed. Each story gets it’s own row and the colour of the story name indicates the state – Green – a successfully executed story, Red – a story that had an execution failure and Grey – for any story that was ignored for some reason. Here are what the buttons and controls do:

Close Simon

This function will close Simon’s results window and remove Simon from memory. This is so that you can return to you app without having to restart it.

Run

Executing Run from this screen will re-run all the stories. If you don’t want all of them to run, you can use the search bar to show only the stories you want to run.

Search

This lets you filter the list of stories. It will search both on the name of the file that the stories were loaded from and also the story name. if a file name matches, all the stories in that file are automatically selected. Otherwise only the stories with matching names.

Individual stories can be run from the Details screen.

Viewing details

You can tap a story to view the details of it. This opens a new screen that looks like this:

Details screen

The details screen is designed to give you a run down of a single story and what happened when it executed. As you can see from the screen dump, it shows you the following information

• The stories title.
• The story file which the story came from.
• The story steps and the status of each step.

You can also use the [Run] button to re-run just this story and when Simon reloads the UI after running it, Simon will return to this screen.

Details screen (Failure!)

Here we have a screen shot of a story which failed. Apart from the step status showing which steps failed, you also get a processed stack trace which hopefully will give you an idea of just where the error originated.

Unfortunately at this time Simon is unable to deduce the exact line numbers. THis is a limitation within Apple’s runtime.

Index

Download latest
static library
v0.2.0