Test status and reporting

Following the successful completion of agreed test cases, the test lead, test manager and project manager must know exactly where the Project / Change stands in terms of testing activities.

Statuses

Each test should be given one of the following statuses: 

  1. Not Started - The test has not started and should be the default status for planned tests at the very start of test execution.
  2. Passed - The expected result defined in the test step matches the actual result experienced by the tester.
  3. Failed - The expect result defined in the test step does not match the actual result experienced by the tester.

It is important to state that even though the failure of the test step may be suspected at that moment in time (e.g., a suspicion a requirement has been changed), whilst the root cause of the unexpected behavior is still unknown the test step / test must be marked as failed.

It is the responsibility of the test lead (with the assistance of the testing manager) for that test phase to communicate the test execution summary on a regular basis throughout the project. Details on how regular and what test metrics will be reported should be described in the lower-level test plan as different Projects / Changes may require differing levels of reporting.

Key metrics that should be reported include:

Details of defects discovered including:

 code, configuration, requirements).

Evaluation of test processes and test improvement (Continuous Improvement)

At each point of the software development lifecycle, including the go-live, the project team should evaluate the effectiveness of testing. Various techniques, such as trend analysis from live service monitoring, will be used to measure the testing. It is crucial to have a clearly defined, comprehensive set of testing processes/standards because the effectiveness of the testing effort will be determined by the quality of the testing process. This will be accomplished via techniques like holding workshops at the conclusion of projects to determine how the procedure may be improved.

This section suggests activities to be undertaken to evaluate current testing practices and to ensure the testing process, associated test procedures and guidelines are regularly reviewed, refined and continually improved.

By identifying areas for improvement and measuring results from the testing effort this will allow testing to be conducted in the most efficient way possible at that point in time. The points below can be undertaken by any member of a testing project however RISA Testing Services will be reviewing any outcomes from the tasks and assist with implementation of any improvements.

Project Lesson Learned workshops

  1. Ensure there is a testing representative involved in any planned overall project lessons learnt meetings.
  2. Identify and review any areas of improvement that have been raised during the test process (test planning, test design, test execution and test closure)
  3. Raise any recommendations to the wider project that may fall outside the testing subject but that as a whole may improve the delivery of future Project / Change delivery.
  4. Arrange lessons learned workshop focusing solely on testing. This should take place at the end of a Project / Change delivery.
  5. The outputs should be shared and reviewed by the RISA Testing Services and where appropriate:
  6. Test Process and low levels procedures will be updated with the actions arising from the workshop
  7. Non-testing relating actions will be communicated to the relevant Project Manager / Team Leader for future projects
  8. Regular meetings held with all key testing stakeholders to agree areas for improvement
  9. Post “Go-Live” support requests / incidents monitoring:
  10. The purpose of this is to identify incidents which could have been prevented during the development and testing effort and to refine future procedures accordingly.