Pipeline Development Processes

Development and Deployment Model

The CASA pipeline has a single release per year.  There are several reasons for this model:

  • Time is required after each release to evaluate the performance of the pipeline against data and formulate new requirements.
  • The ALMA project would prefer to have a single pipeline version throughout an observing cycle.
  • The amount of time required by the scientific staff of both telescopes to validate and document a new pipeline version make more frequent releases prohibitive.

The figure below shows the deployment banches created by the pipeline and the approximate timescales.  The production pipeline branch in created in the late summer to be deployed with the Fall release of CASA.  The trunk of the pipeline at that point is available for development intended for delivery the following year.  Bugs that are descovered in production can be fixed on the production branch, and put into deployment.  In the Spring a "compatability" branch will be created from the production branch, any interface changes or backwords incompatabilities for the pipeline relative to the upcoming release version of CASA will be corrected on this branch and this variant of the pipeline will be delivered as part of the Spring CASA release.   Finally as we return to late summer the next release of the pipeline will be prepared, and the production branch produced.

The yearly calendar is summarized below, this schedule may need to be adjusted from year to year based on the exact timing of other events at the observatory.


Time Period  Development Team Science Operations
Sept. 15 CASA and Production Pipeline Release

Oct.- Nov. 

Bug fixes for production pipeline

Infrastructure development decrease technical debt

Heuristics definition for following release
Dec. 1 Deadline for feature and heuristics definition for next yearly release.
Dec. - Mar. Development of new features and heuristics  
Mar. 1 Initial release (R1) of functionality to Science Operations for feedback, heuristic clarification, and initial verification.

Mar. - April Responding to bugs found in testing of R1, addressing bugs and additional features. Verification of R1 functionality, development of punchlist items 
May 1 Final deadline for release requirements (Initial punchlist, clarification)
May - Jun. Improvements to R1, addressing items on punchlist.  
July 1. Second release (R2) to Science Operations 
 July  Respond to issues identified in testing  Final verificiation testing, initial validation testing
 Aug 1  Final punchlist from verification testing complete 
 Aug - Sept.  Final bug fixes, release preparation  End to end testing, validation testing
 Sept. 15  CASA and Production Pipeline Release  


Version Control System

The pipeline subsystem uses Git as it's version control system.  The pipeline should be included in the continuous integration process in two seperate ways, to accomodate two distinct use cases.

  • Use a released and well tested version of the pipeline to monitor the evolution of the CASA packages and identify any changes that affect the pipeline at the earliest possible point.  This use case also provides significant regression testing for the CASA package.
  • Use a selected version of the CASA package with the latest updates from the pipeline development team for testing of the pipeline development.  The goal is to isolate the pipeline development from changes in the underlying CASA implementation.

The CASA pipeline is maintained as a submodel of the CASA tree in Git, allowing independent development with a simple path to publish new versions of the pipeline as part of standard CASA artifiacts.  Below are recipies for many of the most common tasks that a developer or pipeline lead needs to do.

Developer Tasks

Begin Work on a New Feature

Complete Work on a New Feature

Provide a Fix to a Released Pipeline Version

Administrative Tasks

Change the Version of Pipeline CASA Uses

Change the Version of CASA Used in Regressions