Details
Name
(required) A given name to the execution. It is recommended to keep the name predictable and descriptive what it is testing (for example what version and build number of application this execution is testing).
Tags
User can specify multiple tags by typing in a tag value and pressing enter or select them from drop down list. This will identify to which tag type the execution belongs to (for example that this is ‘regression’ execution).
Test Set
(required) When creating an Execution you have to point at a Test Set which points at a set of test cases (see more details about Test Set creation here). Once you save a new Execution you can never switch to a different set.
State
There are two states: ‘Ready To Run’ (execution is ready to be executed) and ‘Running’ (execution is currently executing).
Lock
An un-locked execution (default state) means that Execution can be run but if user clicks on Lock icon and it means that running it is disabled. This is typically done for executions which are done and no longer need to be executed but we want to keep its results for historical purposes (since lock will also keep all of Test Cases frozen for this execution even if test set and test cases were modified down the line).
Email Notification
You can select multiple users from Email Notification drop down who will receive an email after every time Execution finishes running if ‘Send Email’ check box is checked (for more information on setting up Email settings go here).
Settings
Retry Count
If a test case fails how many times should it be tried again. Specifying value of 2 means it will try test case up two more times before reporting failure, if it does Pass within these first tries then it will not try again and report passing result.
Ignore Status
If a Test Case (or an Action that is being used by the Test Case) has Status as ‘To be Automated’/‘Needs Maintenance’ then the execution will not run it but if you check ‘Ignore Status’ check box then it will run this test case no matter what its Status is.
No Screen Shots
Every time a test case fails it will generate a screen shot for the action that has failed (which can be viewed in ‘Test Details’ tab), but if this check box is set then it will never generate screenshots (useful if running test cases which are not UI based like REST/SOAP, API, CLI, etc.).
UI Snapshots
Checking this option will take a screen shot for every single action regardless if it has failed or not (which can be viewed in ‘Test Details’ tab). This is a useful feature to be able to analyze all the steps visually that led up to the failure but is not recommended to be used for all large executions since they can fill up RedwoodHQ server (but deleting execution will delete all screenshots on back-end as well).
No After State
If set then do not execute After State if it’s specified in a test case (for example After State closes the browser but I want to keep it open for current test case execution after test is done to analyze potential issue).
Set Variables
If a variable was set to be an ‘Execution Var’ in Variables screen, then it will appear in this list here and execution will use the value that was set in this execution (for more information on setting up Variables go here). This is useful to setup values specific to this execution run like which Browser to use, URL of test application to use, etc.
Name
Name of the variable as defined on Variables page.
Value
The value that will be assigned to the variable for this execution. If Possible Values are specified for this variable then they can be selected via drop down.
Tags
Variables can also be Taged for ease of search and selection.
Execution Totals and Chart
Execution Totals and Execution Chart will show a current snapshot of the execution results and how it stands right now.
Total
Total amount of test cases in this execution as defined in Test Set.
Passed
How many test cases have Passed the execution run.
Failed
How many test cases have Failed the execution run.
Not Run
How many test cases have not been executed yet OR are in ‘Need Maintenance/To be Automated’ status.
Run Time Sum
The total sum time of all test case ‘Elapsed Time’ column. This shows how much it would take if 1 machine with 1 thread it would’ve taken to execute so far (so can potentially be used as a multiplier to determine how much time it would take if more machines/threads are assigned for execution).
Export Results
User can easily export current execution results in to PDF file by clicking on ‘View Results as PDF’ button (located on top). After clicking on the button a PDF file will be downloaded that will list execution name, total’s overview and details of each test case whether it passed or failed and with what error.
Post your comment on this topic.