How to Re-run RSpec Tests Again After Failures

In the course of working with Selenium and Appium via RSpec tests (while testing iOS apps), I occasionally was running into issues where Appium would lose its connection with the iOS simulator and the RSpec test would fail.

The test failures had nothing to do with the iOS app itself, or even with the RSpec tests. The problems existed somewhere between Selenium, Appium, and the iOS simulator. Something would just randomly break on one of those layers, and my code’s connection to the iOS simulator would essentially be lost. The only way to fix this problem is to manually close the iOS simulator and re-launch it via Appium. After that, all the RSpec tests would proceed merrily along their way without problem… or at least until Appium glitched again.

As you can imagine, with a large test suite, this can be rather annoying. If the driver glitches halfway through, you end up having to reset everything and re-run the entire suite from scratch.

This made me wonder… Would it be possible to catch the random driver errors and random Appium glitches, then programmatically reset the iOS simulator and re-run the failing test?

I did a bit of research and came up with a solution, which I will share here. Hopefully someone out there will get some benefit out of it:

spec_helper.rb (partial):

Essentially, what we are doing is running each RSpec test inside of an “around” block, and if the example fails for some driver-related reason, we reset and retry (up to three times) before moving on to the next test in the suite. Nifty, eh?

————————————-

01/30/2015 – Update to account for recent rspec changes

There have been some recent changes to the way how rspec handles examples and exceptions, so I have updated my code accordingly:

6 comments

  • Are you re-setting the example result anywhere? I’m curious when an example is re-ran and it passes whether the suite result is pass or fail.

    • Yes, it resets the exception on the rspec example to nil before it retries, as seen in this line of code:

      @example.instance_variable_set(‘@exception’, nil)

      This means that if the retry succeeds, there is no exception on the example, and the suite passes.

  • Thanks for the great code.
    I am just starting to use rspec and webdriver.

    Thanks again for sharing

  • Thanks for this. It helped me out greatly without having to incorporate a gem or something else.

  • Hi
    Thanks for the awesome code.
    I have just started programming and this has been really helpful.
    It has stopped my allure reporting working.
    In allure I get – undefined method ‘example’ for #

    Sorry to ask but would you be able to explain why that is happening?

    thanks

    Mick

  • I was having some troubles recently where retries would succeed but still be counted as failures, showing at the error report at the end. The trouble seemed to be coming from resetting @excecption to nil – the code “example.instance_variable_set(‘@exception’, nil)” didn’t seem to be working. When I changed it to “real_example.instance_variable_set(‘@exception’, nil)”, it worked fine.

    Was anyone else having this issue?

Leave a Reply

Your email address will not be published. Required fields are marked *

Are you a real person? *