CASA software quality control consists of automated and manual testing, which are performed in separate, sequential stages of the work request lifecycle. Within the CASA JIRA workflows automated testing is referred to as "verification" testing and manual testing is referred to as "validation" testing. However, this use of terminology is not consistent with software project management standards. Formally, verification tests tell us if the software meets requirements, while validation tests tell us if the software satisfies the user's needs and expectations. CASA's automated tests may contribute to both verification and validation; likewise, manual tests may contiribute to both verification and validation.

The following testing issues will be discussed with the automated and manual testing teams in FY17 Q2. A more detailed testing plan will be developed based on the outcome of this discussion.

• Agree on and document automated test nomenclature. Are the current test categories, described in Automated Test Definitions, appropriate? Should we consider using different categories as described in Recommendations for Improvement? How do CASA's test categories map to Miranda's test categories?
• Measure current branch coverage, or establish plan for measuring coverage. 
• Agree on and document test ownership. Who is responsible for maintaining each test? Does the test maintainer have authority to make changes to test?
• How is information communicated from automated to manual test groups, and vice a versa? Do validators know if an automated test failed? Can they see the test results?

Automated Tests

The CASA team will build a set of CASA functional capabilities and map those capabilities to both files and tests (see Traceability of Functions, Files, and Tests). The list of software functional capabilities will then serve as a high-level list of requirements, and the automated tests will verify that functional capabilities are maintained with each code modification. By tracing function to files and tests, the test team will be able to selectively run only the tests that are applicable to a given code change. This capability should be planned and developed in conjunction with the traceability tool.

• When selective automated testing is in place, developers will increase test branch coverage with every work request.

Measurement and Analysis

The following measurements will be used for CASA quality assessment:

• Branch coverage as a percentage of total code branches.