Concatenating multiple datasets (concat)

NRAO Home > CASA > CASA Cookbook and User Reference Manual

2.2.10 Concatenating multiple datasets (concat)

Once you have your data in the form of CASA Measurement Sets, you can go ahead and process your data using the editing, calibration, and imaging tasks. In some cases, you will most efficiently operate on single MS for a particular session (such as calibration). Other tasks will (eventually) take multiple Measurement Sets as input. For others, it is easiest to combine your multiple data files into one.

If you need to combine multiple datasets, you can use the concat task. The default inputs are:

#  concat :: Concatenate several visibility data sets.
vis                 = ’IRC10216.cont’   #  Name of input visibility files to be concatenated
concatvis           =         ’’        #  Name of output visibility file
freqtol             =         ’’        #  Frequency shift tolerance for considering data as the same spwid
dirtol              =         ’’        #  Direction shift tolerance for considering data as the same field
timesort            =      False        #  If true, sort by TIME in ascending order
copypointing        =       True        #  Copy all rows of the POINTING table.
visweightscale      =         []        #  List of the weight scaling factors to be applied to the individual MSs
createmms           =      False        #  Should this create a multi-MS output
async               =      False        #  If true the taskname must
be started using concat(...)

The vis parameter will take a list of one or more MS. Usually, this will contain all the MS to combine.

With visweightscale, a list of weights can be manually specified for the respective input data sets. They will be applied at the time of the combination. To determine the appropriate weights for this procedure, one can inspect the weights (Wt axis parameter) of the input datasets in plotms.

The concatvis parameter contains the name of the output MS. If this points to an existing file on disk, then the MS in vis will appended to it, otherwise a new MS file is created to contain the concatenated data. Be careful here!

The timesort parameter can be used to make sure the output MS is in time order (e.g. if your input MS have concurrent times). This can possibly speed up some subsequent calibration operations.

Furthermore, the parameter copypointing can be used to control whether the POINTING table will be carried along in the concatenation process or if the output MS should not contain a POINTING table. This table is quite large for some data (e.g. ALMA) and is mainly needed for mosaic imaging. If you are certain that you will not need it, you can save time and diskspace by setting copypointing to False.

The parameters freqtol and dirtol control how close together in frequency and angle on the sky spectral windows or field locations need to be before calling them the same.

ALERT: Note that if multiple frequencies or pointings are combined using freqtol or dirtol, then the data are not changed (ie. not rephased to the single phase center). Use of these parameters is intended to be tolerant of small offsets (e.g. planets tracked which move slightly in J2000 over the course of observations, or combining epochs observed with slightly different positions).

For example:

   default(’concat’)
   vis = [’n4826_16apr.split.ms’,’n4826_22apr.split.ms’]
   concatvis = ’n4826_tboth.ms’
   freqtol = ’50MHz’
   visweightscale=[’1’,2’]
   concat()

combines the two days in ’n4826_16apr.split.ms’ and ’n4826_22apr.split.ms’ into a new output MS called ’n4826_tboth.ms’, and the second MS is weighted twice the first one.

ALERT: Note that if you are concatenating MSs which use antennas which were moved between observations, the MS definition does only foresee a unique antenna ID, but not a unique name(!). The moved antenna will appear twice in the antenna list under the same name but on different stations and with two different IDs. The pair (’NAME@STATION’) will be the unique identifier.

If you would like to only concatenate the subtables of several MSs, not the bulk visibility data, you can use the task testconcat instead of concat to save time and diskspace. testconcat has the same parameters as concat. It produces an output MS with the concatenated subtables and an empty Main table.

[next] [prev] [prev-tail] [front] [up]

More information about CASA may be found at the CASA web page

This code is available under the terms of the GNU General Public Lincense