Why Automate Software Testing?

• Save time by automating manual tests that need to be run frequently
• Make it possible to monitor the behavior of systems with thousands of features (or more)
• Reduce risk by testing mission critical or high-risk business processes whenever there is a change
• Reduce time to deliver new software functionality to users (since developers can immediately find out whether new features will break old ones)
• Support agile processes that demand frequent updates of tests in response to new and enhanced features

Which Cases are Good for Automation?

• Simple or Easily Assessed: Tests that don't require human judgment
• Frequent: Tests that need to be run often
• Tedious: Tests that are painstaking for humans to run
• Multiple Examples Required: When a single scenario needs to be tested with multiple combinations of data
• Mission Critical: When a business depends on a process and needs to continuously monitor it

When to Avoid Automating Tests

• When the amount of time to create and run a manual test (over a period of time) is much less than the amount of time to automate.
  • Example 1:
    • It takes 2 hours to run a complex manual test. That test only needs to be run once every three months, and it rarely changes. In a year, that's 2 hours x 4 times/year = 8 hours/year
    • It would take 80 hours to automate that same complex test. Over the course of a year, 8 hours of labor are saved. 80 hours invested - 8 hours saved = 72 hour cost
    • In this case, manual testing is more efficient.
  • Example 2:
    • It takes 2 hours to run a complex manual test. The underlying process changes frequently, so the test has to be run at least once a day. In a year, that's 2 hours x 260 workdays/year = 520 hours/year
    • It would take 80 hours to automate that same complex test. Over the course of a year, 520 hours of labor are saved. 80 hours invested - 520 hours saved = 440 hour benefit (or 10 person-weeks)
    • In this case investing in test automation makes sense.
• When you can drive the testing process deeper into the system.
  • Example: A client has asked you to write automated tests for their web UI. The web UI calls APIs to deliver the functionality. If you test at the API level instead of the UI level, you can still continuously assess the behavior and performance of the system, but you won't bear the cost of having to change the tests whenever the UI changes. (Note: the client may still want you to write some UI tests.)