A new capability available in 2.40 is the ability to significantly reduce the “noise” of false positives during playbacks. This is achieved by running a “base-line” playback into a precise database copy that allows the customer to understand the “noise” in the system. A second playback is then required (separate execution queue) that is done into a system that contains the changes to be tested. The failures in the second (latter) queue are compared to the base-line. If they happened there for the same reason, these are excluded from defect proposal generation for the second queue – thus reducing the noise.
The following process is required to achieve this:
- Run a playback into an exact copied system (same database, no changes)
- Copy the execution queue within the same test plan
- Restore the target system to the snapshot
- Deploy changes that are to be tested
- Re-run the playback of the second queue
- Run defect proposal with new parameters to activate double playback