CASA Test Framework Primer


This document is a discussion on how the test framework for CASA is deployed. It's intended audience is developers who need to learn, or are otherwise curious about how the CASA test framework is deployed. Links to references are included largely to flesh out details that are a bit too low level for this document. 


Software Test Infrastructure Technologies

CASA uses primarily a couple of test framework technologies: the Robot Framework and PyUnit. The Robot Framework is a general test framework that can be used for Python or Java. CASA uses it for Python. Please refer to as an introduction to this technology. The more technical User Guide can be found here: PyUnit is the standard unit testing framework for Python. It is used for what in CASA are considered functional tests (as opposed to regression tests). A helpful website to gain some background on PyUnit is:

Gradle is the technology used to build CASA. Gradle is written in Groovy. The Gradle User Guide is here: You can find an overview of Gradle in the User Guide. For some more background on Groovy:

Gradle can be run from the command line (using the build.gradle command), at which point it will run Groovy to build and run the tests. The output from this will be both the log.html and report.html pages to show test results in your favorite browser. Pybot can also be run on the command line (when run in the same directory as the HTML files in the pybot subdirectory). Likewise the output from Pybot will be both the log.html and report.html test results.  


Continuous Integration Technology Employed

Though this document is defining the automated test frameworks being employed by CASA, it is worthwhile to briefly mention the technology which is using these during the build and test process. Currently CASA is using Atlassian's Bamboo to build and test the software. You can read more about Bamboo here: How CASA uses Bamboo is outlined  here: CASA's Project Wallboard is at: This outlines build and test results for the various plans defined for CASA. Note - this requires authentication using your NRAO credentials. 

The sequence of events is that Bamboo initiates Gradle, which in turn executes Groovy scripts, which kicks off Pybot. Pybot then runs the tests dictated by the html files, and using the script, that finally runs and scripts, executing the tests in the system, and finally producing the reports with the test results.


Regression and Unit (Functional) Tests

Regression Tests

The file which dictates how regression tests are run is: It currently can be found at ~/gcwrap/python/scripts/regressions/admin/ An example of how to run your regression test is: 

casa --nologger --log2term -c `echo $CASAPATH | awk '{print $1}'`/gcwrap/python/scripts/regressions/admin/

Where testname is the Running this will only produce terminal line messages/errors: no reports are produced running this command. 

Unit (Functional) Tests

There is documentation covering Functional Tests: For the most part, this documentation is pertinent. 


Automated Tests Defined

An article that fleshes out the details of how tests are broken down is available: Automated Test Definitions


Requesting Tests Be Added or Removed

If you've written your test, and would like it included in the framework, directions are here:

If you want an existing test removed from the framework, please give this a look: