Release Notes CASA 5.4.1

Summary of new features in this release of CASA

CASA 5.4.1 is intended for general use, and is also the official VLA pipeline release. The official ALMA pipeline release is CASA 5.4.0. The CASA 5.4 releases build on CASA 5.3. Changes compared to the CASA 5.3 release are listed below. The differences between 5.4.1 and 5.4.0 are labeled "5.4.1 only" on this page.

IMPORTANT: NEW VERSION CASA 5.4.1-32

A patch to CASA 5.4.1 with the version number CASA 5.4.1-32 is officially released by CASA and approved by the VLA, and can be downloaded from the CASA download page. The 5.4.1-32 version solved a critical bug for large mosaic cubes, where the signal-to-noise in the first pointing of a field could be noticeably higher than in the rest of the cube.

Problem: The problem occurred when the chanchunk parameter in tclean is put to a value greater than the default value of 1. Note that for chanchunks = -1 (which is often used as default for ALMA), chanchunks > 1 is triggered whenever the estimated memory requirement is > 80% of the system memory. The value of chanchunks used to create a cube is printed in the casa .log file, shortly after tclean starts. The chanchunk feature is useful for large images cubes, where the gridders can run into memory limits as they loop over all available image planes for each row of data accessed. To prevent this, the chanchunk parameter can be used to grid subsets of channels in sequence, so that at any given time only part of the image cube needs to be loaded into memory. The bug found for values of chanchunk >1 affects primarily mosaic images, where each "source" is made up of more than one "field", but technically also affects single field observations, where each "source" is made up of one "field". The specifics of the bug in tclean are:

gridder='mosaic': the first integration of the first field of each source included in the mosaic is dropped.

gridder='standard': the first integration of each source (i.e. each field in this case) included in the image is dropped (though typically the 'standard' gridder is used when there is only one field).

The loss in signal-to-noise in the affected pointing scales with sqrt(#integrations -1)/ #integrations, so gets less problematic when a pointing is visited with a large number of integrations. For example, if a field has > 5 integrations throughout the data selected for the image, the loss in rms noise will be <10%. So only images with few integrations per field are adversely affected - typically these will be large mosaics and very short snapshot observations.

Solution: We recommend VLA users to use the CASA 5.4.1-32 version, in particular for large image cube mosaics.

ALMA users are recommended to use CASA 5.4.0-70, which also has this bug-fix for chanchunk included. Compared to CASA 5.4.1-32, the version CASA 5.4.0-70 excludes several features that were implemented for the VLA and not yet testing by ALMA (these features are not relevant for ALMA Cycle-6 production).

All new features and bug fixes throughout the various 5.4 release versions will be included and tested for both ALMA and VLA in the next CASA 5.5 release. To the best of our knowledge, this chanchunk bug was introduced in CASA 5.0 and does not affect single dish imaging tasks (tsdimaging)

General

  • During testing for the CASA 5.4 releases, several issues involving mosaicing in clean and/or tclean for CASA versions 4.7 - 5.1 were uncovered, as explained in more detail below. These issues have been fixed in CASA 5.4.
  • A number of tasks have been deprecated in CASA 5.4: autoclean, boxit, clearplot, csvclean, importevla (use importasdm instead), mosaic, oldhanningsmooth, oldpartition, oldplotants, plotuv (capabilities transferred to plotms), plotxy (capabilities transferred to plotms), ssoflux, widefield, visstatold and visstat2.
  • The deprecated task importevla is now replaced by importasdm, but several additional step in parameters specification and flagging of zero and shadowed data are required to replicate the functionality of importevla, as described in the CASA Docs task pages of importasdm.
  • The task statwt2 is now statwt (the previous startwt is now oldstatwt)
  • 5.4.1 only: statwt has changed its syntax for choosing which channels to take into account regarding the weights calculation. While in CASA 5.4.0, the parameter excludechannels was introduced to exclude channels from the weights calculation, CASA 5.4.1 uses fitspw to specify channels to be included in the weights calculation.
  • Starting CASA 5.4.1, telemetry data is by default generated for each CASA run and sent anonymously to NRAO to measure statistics on CASA task useage. This feature can be disabled by setting the CASA_ENABLE_TELEMETRY environment variable to "false".

Documentation

  • We continue to update CASA Docs, but are still in a transitioning phase of making CASA Docs the main source of information on CASA. All task descriptions are now in CASA Docs, and we will gradually phase out the inline help descriptions. We are slowly rolling out additional documentation on remaining areas of CASA as we are able to test and verify the functionality matches the description.

Data Import/Export

  • Ephemeris data for the Moon, based on JPL-Horizons with a cadence of 1hour, has been added as a standard DE200 ephemeris table.
  • importnro now imports Nobeyama data with data stored in UTC, rather than JST (UTC+9).

MS Viewing, Editing, Manipulation

  • flagmanager now returns a dictionary for mode='list', returning the flag version names and comments. 

Calibration

  • 5.4.1 only: gaincal now has the option to use the median values for normalizing amplitude solutions
  • plotms inherited the functionality of plotcal and now supports all calibration tables. plotms now also offers plotting scalar residuals. In addition, plotms supports simple text export, which just exports X and Y values, in addition to the default verbose export.
  • 5.4.1 only: plotms now also the capability to connect points for calibration tables (support for MeasurementSets will be added later).

Imaging

  • During testing for the CASA 5.4.0 release, several unknown issues were uncovered that affected mosaic imaging in clean and/or tclean for CASA versions 4.7 - 5.1. The problems have been investigated and quantified for ALMA by the North American ALMA Science Center (NAASC) Software Support team, and can be found in this memo. In short, these are the problems uncovered:
    • ALMA 7m-array mosaics imaged with CASA versions up to and including 4.7.2 (Cycle 0-4): The ALMA 7m-array mosaic imaging used a non-optimal definition for the antenna primary beam in both clean and tclean up to CASA 4.7.2. For Nyquist-sampled mosaics, after primary beam correction, within the 50% mosaic primary beam response, the result is flux densities that are too high by ∼10%. At the 20% mosaic primary beam response level, the flux densities are too large by ∼30%.
    • All mosaics imaged with gridder='mosaic' in tclean in CASA 5.0 and 5.1 (this includes all ALMA mosaics made with Cycle-5 pipeline version 5.1.1): In CASA 5.0 and 5.1, a truncated version of the PB model was erroneously used for gridding the data in tclean. This truncation causes aliasing (at some level) in all CASA mosaic images made with the mosaic gridder. The manifestation of the problem in the image plane depends sensitively on a range of factors including: mosaic shape, mosaic spatial sampling, and uniformity of mosaic sensitivity per pointing. In extreme cases errors as large as 15% have been observed inside the 50% mosaic primary beam response, but the median error for most mosaics is a few percent inside the 50% mosaic primary beam response. In particular for rectangular mosaics, the situation rapidly deteriorates beyond the 50% mosaic primary beam response, with flux density errors as large as 30% seen at the 20-50% primary beam response for mosaics with size-ratio in the actual sky-coverage pattern of 1.5 or larger.

The latter issue affects images made with the mosaic gridder for all telescopes (including the VLA) at some level, although the effect of the errors is expected to be smaller for larger antennas. VLA observers who created image mosaics using the mosaic gridder in previous CASA 5 versions, especially in the case of non-square image sizes, are encouraged to re-image using the latest CASA version.

     The reported problems only affected data imaged with the gridder='mosaic' mode. These known issues have been fixed in CASA 5.4.0.

  • Automasking has been further improved with refined performance.
  • tclean now fully supports mosaics of ephemeris objects spanning one or more execution blocks through the new parameter phasecenter. For ALMA data, phasecenter='TRACKFIELD' uses the ephemeric tables attached to each MS. For the latest ephemeric information, or for non-ALMA data, one may set phasecenter to the name of the ephemeris file, or a body for which a DE200 ephemeris table is distributed with CASA ('MERCURY', 'VENUS', 'MARS', 'JUPITER', 'SATURN', 'URANUS', 'NEPTUNE', 'PLUTO', 'SUN', 'MOON')
  • For mosaic imaging with Briggs/uniform weighting in tclean, the mosweight sub-parameter of gridder='mosaic' has been added to tclean with default setting mosweight = True (compared to the old mosweight = False default in clean). There is the current Known Issue that mosweight=True is now silently invoked also for other multi-field imaging. The default mosweight = True optimizes noise characteristics for ALMA mosaics. However, it may potentially cause memory issues for large VLA mosaics, and may cause the major and minor axis of the synthesized beam to be up to ~10% larger than with mosweight=False. Please change to mosweight=False to get around these issues.

Analysis

  • To calculate robust statistics in imstat, an algorith = 'biweight' option has been added to determine the center and width of a distribution while downweighting points that are far off, e.g. due to the presence of significant, wide spread signal.
  • Most image analysis tasks in CASA can now handle complex-double and double precision images.

Bug-fixes (selected)

  • In imregrid, the conversion between galactic and J2000 coordinates that previously caused a wrong tilt to high declination targets has been fixed.
  • In simobserve and  the underlying tool ia.modify, point-sources from a component-list were assigned the wrong WCS coordinates in CASA 5.3, because the x and y pixel positions were swapped. This issue has been fixed for CASA 5.4, and only affected CASA version 5.3.
  • A bug was fixed where multi-EB ephemeris data sets produced a wrong image with tclean parameter settings deconolver='mtmfs' and nterms=2
  • In rflag, the parameters 'timedevscale' and 'freqdevscale' now operate in the 'apply' mode, whereas previously the values were written to the flag file during the 'calculate' mode only, and any settings during the 'apply' stage were silently ignored.
  • In split, specifying a spw selection with multiple channel ranges within a single spw (e.g., spw='1:2~6;10~12') now creates the correct DATA_DESCRIPTION subtables
  • In sdimaging, a bug was fixed that caused wrong computation of pointing directions when multiple fast-scan MS files are given as input.
  • In importnro, several bugs that causes data inconsistency in output MS have been fixed
  • 5.4.1 only: In tclean, the default of conjbeams is set to False for all applicable gridders, including mosaic and awproject (was mistakenly set to conjbeams=True in CASA 5.3 and 5.4.0).
  • 5.4.1 only: In statwt, the default of the parameter timebin has temporarily been set to 0.001s to work around a bug detailed in the CASA 5.4.1 Known Issues.