Data Flagging using tflagdata

NRAO Home > CASA > CASA Cookbook and User Reference Manual

3.4 Data Flagging using tflagdata

Alert: tflagdata is experimental for CASA 3.4. For CASA 3.5. we will remove the current tasks flagdata, flagdata2, flagautocorr, and testautoflag and rename tflagdata to flagdata. So we recommend to become familiar with tflagdata and recommend that scripts may be written to use the new tflagdata syntax. tflagdata is substantially faster, more flexible, and more comprehensive than the current non-interactive flagging tasks. tflagdata has many modes to derive flags, based on data selections, derived quantities such as shadowing. We also implemented automatic flagging algorithms, tfcrop and rflag.

For a full description of tflagdata please visit:

http://www.aoc.nrao.edu/\~rurvashi/FlaggerDocs/FlaggerDocs.html

The inputs to tflagdata are:

#  tflagdata ::  All-purpose flagging task based on data-selections and flagging modes/algorithms
vis                 =         ’’        #  Name of MS file to flag
mode                =   ’manual’        #  Flagging mode (list/manual/clip/shadow/quack/elevation/tfcrop/rflag/
                                        #   extend/unflag/summary)
     field          =         ’’        #  Field names or field index numbers: ’’==>all, field=’0~2,3C286’
     spw            =         ’’        #  spectral-window/frequency/channel
     antenna        =         ’’       #  antenna/baselines: ’’==>all, antenna =’3,VA04’
     timerange      =         ’’        #  time range: ’’==>all,timerange=’09:14:0~09:54:0’
     correlation    =         ’’        #  Select data based on correlation
     scan           =         ’’        #  scan numbers: ’’==>all
     intent         =         ’’        #  Select data based on observation intent: ’’==>all
     feed           =         ’’        #  multi-feed numbers: Not yet implemented
     array          =         ’’        #  (sub)array numbers: ’’==>all
     uvrange        =         ’’        #  uv range: ’’==>all; uvrange =’0~100klambda’, default units=meters
     observation    =         ’’        #  Select data based on observation ID: ’’==>all
     autocorr       =      False        #  Flag auto-correlations

action              =    ’apply’        #  Action to perform in MS and/or in inpfile (none/apply/calculate)
     display        =         ’’        #  Display data and/or end-of-MS reports at runtime (data/report/both).
     flagbackup     =       True        #  Back up the state of flags before the run

savepars            =      False        #  Save the current parameters to the FLAG_CMD table or to a file
async               =      False        #  If true the taskname must be started using tflagdata(...)

The mode parameter (§3.4.2) selects the flagging algorithm and the following are available:

list        = list of flagging commands to apply to MS
manual      = flagging based on specific selection parameters
clip        = clip data according to values
quack       = remove/keep specific time range at scan beginning/end
shadow      = remove antenna-shadowed data
elevation   = remove data below/above given elevations
tfcrop      = automatic identification of outliers on the time-freq plane
rflag       = automatic detection of outliers based on sliding-window RMS filters
extend      = extend and/or grow flags beyond what the basic algorithms detect
summary     = report the amount of flagged data
unflag      = unflag the specified data

Flagging will only be applied to the data selection that is performed with the usual selection parameters (§ 2.3). The dataset is iterated-through in chunks (small pieces of data) consisting of one field, one spw, and a user-defined timerange (default is one scan). In addition to the typical antenna, spw, timerange, etc. selections, we would like to point out some addition of the correlation syntax for modes clip, tfcrop, and rflag. One can combine correlation products with simple mathematical expressions

’ABS’, ’ARG’, ’RE’, ’IM’, ’NORM’

followed by the polarization products (using an underscore in between “_” )

’ALL’, ’I’, ’XX’, ’YY’, ’RR’, ’LL’, ’WVR’

’WVR’ refers to the water vapour radiometer of ALMA data. Note that the operators ABS,ARG,RE, etc are written only once as the first value. if more than one correlation is given, the operator will be applied to all of them. An example would be

correlation=’RE_XX,XY’

which would select all real XX and XY polarization for flagging.

  3.4.1 The action parameter
  3.4.2 Flagging Modes
   3.4.2.1 Manual Flag/Unflag
   3.4.2.2 List
   3.4.2.3 Clip
   3.4.2.4 Shadow
   3.4.2.5 Quack
   3.4.2.6 Elevation
   3.4.2.7 Tfcrop
   3.4.2.8 Rflag
   3.4.2.9 Extend
   3.4.2.10 Unflag
   3.4.2.11 Summary

[next] [prev] [prev-tail] [front] [up]

More information about CASA may be found at the CASA web page

This code is available under the terms of the GNU General Public Lincense