Overview

The importasdm task will fill SDM1.2 and SDM1.3 format data into a CASA visibility data set (MS).  ALMA data were recorded in SDM1.2 format from October 2009 until May 2011. Since May 2011, ALMA is using the SDM1.3 format. In particular, all science data from Cycle 0 will be in SDM1.3. The JVLA also started using SDM1.2 in October 2009 and continues to use this format as of October 2011.  The importasdm task can read all of the above formats.

The basic modes of importing data utilizing importasdm can be set by using various parameters.  For example,

• The parameter useversion can be used to enable the options 'process_syspower', 'process_caldevice', and 'process_pointing'.
• If the parameter scans is set, then importasdm processes only the scans specified in the option’s value. This value is a semicolon-separated list of scan specifications. A scan specification consists in an execution (exec) block index followed by the character ’:’ followed by a comma-separated list of scan indices or scan index ranges. A scan index is relative to the exec block it belongs to. Scan indices are 1-based while exec blocks are 0-based. The expressions:
  • "0:1"
  • "2:2~6"
  • "0:1,1:2~6,8;2:,3:24~30"
  • "1,2"
  • "3:"
• are all valid values for the selection. The "3:" selector will be interpreted as ’all the scans of the exec block 3’. A scan index or a scan index range not preceded by an exec block index will be interpreted as ’all the scans with such indexes in all the exec blocks’. By default, all the scans are considered.
• When process_flags=True, the task will create online flags based on the Flag.xml, Antenna.xml and SpectralWindow.xml files and copy them to the FLAG_CMD sub-table of the MS. The flags will NOT be applied unless the parameter applyflags is set to True. Optionally, the flags can also be saved to an external ASCII file if savecmds is set to True. The flags can later be applied to the MS using task flagdata in list mode.
• When bdfflags=True, the task will apply online flags contained in the ASDM BDF data by calling the executable bdflags2MS which the user can also do from the OS prompt. Setting bdfflags=True is recommended for ALMA data.
• If singledish=True, the output data format is a scantable (single-dish data format, see the chapter on single-dish calibration) instead of MS. In that case, you must specify the name or id of the antenna that you want to obtain data. This can be done by using the antenna parameter that is defined as a subparameter of singledish. For single-dish mode, only auto-correlation data are filled, i.e. ocorr mode is forcibly set to ’ao’. The option createmms prepares the output file for parallel processing and creates a multi-MS.

Import of ASDM data with option lazy=True

With release 4.3, the parameter lazy (default = False) is fully tested and operational for interferometric data. For total power data, lazy=True will work starting with version 5.0 of CASA. If the default value False is chosen, importasdm will (as in previous versions) fill the visibilities into a newly created DATA column (FLOAT_DATA for total power data) of the MS converting them from their binary format in the ASDM to the CASA Table format. If lazy is set to True, the task will create the DATA/FLOAT_DATA column with an ASDM-specific storage manager, the (asdmstman), which enables CASA to directly read the binary data from the ASDM with on-the-fly conversion. No redundant copy of the raw data is created.

This procedure has the advantage that it saves more than 60% disk space and at least in some cases makes the access to the DATA column ≥ 10% faster because the data I/O volume is decreased. For the same reason, it also accelerates the import itself by ca. a factor 2. The acceleration is particularly large in the applycal task and here particularly on standard SATA disks. E.g., if your ASDM has a size of 36 GB, the import with default parameters will turn this into an MS of 73 GB size (total disk space consumption = 36 GB + 73 GB = 109 GB). With lazy=True, the imported MS has a size of only 2 GB (total disk space consumption = 36 GB + 2 GB = 38 GB). I.e. your total disk space savings are ca. 65%. Even when you compare to the case where you delete the ASDM after normal import, the solution with lazy import and keeping the ASDM will save you ca. 48% disk space (in the example above 38 GB compared to 73 GB). The only caveats are the following:

1. You must not delete your ASDM. You can, however, move it but you have to update the reference stored in the MS. Symbolic links will work. See below on how to use the tool method ms.asdmref to manipulate the ASDM reference.
2. The lazily imported DATA/FLOAT_DATA column is read-only. But in any normal data reduction, the DATA/FLOAT_DATA column (as opposed to CORRECTED DATA) is treated as read-only anyway.

The lazily imported MS is numerically identical with the traditionally imported MS and so are all results derived from the MSs. The setting lazy=True might be made the default setting in future CASA releases. An important additional tool to manipulate lazily imported MSs is the method ms.asdmref in the MS tool. If the MS is imported from an ASDM with option lazy=True, the DATA/FLOAT_DATA column of the MS is virtual and directly reads the visibilities from the ASDM. A reference to the original ASDM is stored with the MS. If the ASDM needs to be moved to a different path, the reference to it in the MS needs to be updated. This can be achieved with ms.asdmref. The method takes one argument: abspath. When called with abspath equal to an empty string (default), the method just reports the currently set ASDM path or an empty string if the ASDM path was not set, i.e. the MS was not lazily imported. If you want to move the referenced ASDM to a different path, you can set the new absolute path by providing it as the value of abspath to the method.

will set the new location of the referenced ASDM to /home/alma/myanalysis/uid___A12345_X678_X910.