Version 1.9 Build 1556

News	FAQ
Search	Home

Next: Management and Oversight Up: NOTE 209 - AIPS++ OPERATIONS Previous: Astronomer and Programmer support

Subsections

Quality Assurance

Traditionally quality assurance for software packages has been either diffused over an entire organization or, more commonly, done by the users. In keeping with the large scope of the AIPS++ Project, we have decided to designate a Quality Assurance Group that has special responsibility for ensuring the quality of official AIPS++ products. The main task for the QAG is assuring the high quality of AIPS++ products, principally via testing, including execution of software, reviewing code and documentation.

Testing

Unit testing: Currently the code review process requires that along with submitted code, unit-test software is also submitted. The complete suite of unit tests is run once a week as a check of the basic integrity of the system. We have found this to be extremely useful, and see no reason not to continue it.
Application testing: Application testing is currently weak. Some test scripts are present in the system but these are few in number. We think that two orthogonal approaches are needed. First we must perform the equivalent of periodic unit testing checking known inputs against expected outputs. Second, we should assign a group to perform blind-testing of applications, to trap unexpected behavior not caught by unit-testing.
Package testing: Does a given package do all that is required? Is the model used an appropriate one? Are there better models available? This type of high-level review will require a collaboration between the QA group and an expert in the field. One might think of this as being similar to a review of a technical textbook.
Benchmarking: The AIPS Project found that benchmarking of new systems using a pre-determined set of applications was extremely useful both for determining possible computer purchases and for uncovering systematic performance problems. They use a set of Dirty Dozen Tasks to characterize the performance.

The Quality Assurance Group will be initially consist of three people, with the following division of responsibility:

Code Cop

Ensuring that code is reviewed in a timely manner
Ensuring that module and code documentation is kept up to date
Ensuring that unit testing is performed
Updating external libraries when necessary
Maintaining the RCS and Change log entries so that they are useful
Maintaining the directory hierarchy

Rules Boss

Establishing new coding standards and removing outdated ones
Maintaining templates used for aips++ development
Controlling development of the AIPS++ directory hierarchy
Maintaining and policing the file extensions used
Controlling inclusion of external libraries

Chief Tester

Responsible for user level documentation
Ensuring that that applications testing, both automated and interactive, is performed
Maintaining benchmarks and performing new ones
Has final sign-off on AIPS++ releases

The head of the QAG will report directly to the Project Manager.

Code Reviews

Our code review process has been in place for some time now and seems to work well for the current size of the Project. The process is documented in Note 167. A code cop is currently responsible for overseeing the code review process. As the Project grows, this role would probably better be served by a group of people, perhaps in the above-mentioned Quality Assurance Group.

We need to augment the code review process with design reviews to catch problems early on in the design-implement-review-use cycle. We may also wish to use code walkthroughs or inspections.

Next: Management and Oversight Up: NOTE 209 - AIPS++ OPERATIONS Previous: Astronomer and Programmer support Contents