This task can flag a MeasurementSet or a calibration table. It has two main types of operation. One type will read the parameters from the interface and flag using any of the various available modes. The other type will read the commands from a text file, a list of files or a Python list of strings, containing a list of flag commands (each line containing data selection parameters and any parameter specific for the mode being requested). Please see examples at the end of this help.

It is also possible to only save the parameters set in the interface without flagging. The parameters can be saved in the FLAG_CMD sub-table or in a text file. Note that when saving to an external file, the parameters will be appended to the given file.

The available flagging modes are: 'manual', 'clip', 'shadow', 'quack', 'elevation', 'tfcrop', 'rflag', 'extend', 'unflag' and 'summary'. For automatic flagging, it is recommended to combine auto-flag modes with 'extend', via the list mode.

The current flags can be automatically backed up before applying new flags if the parameter flagbackup is set. Previous flag versions can be recovered using the flagmanager task.

The task will flag a subset of data based on the following modes of operation:

• 'list' = list of flagging commands to apply to MS/cal table
• 'manual' = flagging based on specific selection parameters
• 'clip' = clip data according to values
• 'quack' = remove/keep specific time range at scan beginning/end
• 'shadow' = remove antenna-shadowed data
• 'elevation' = remove data below/above given elevations
• 'tfcrop' = automatic identification of outliers on the time-freq plane
• 'rflag' = automatic detection of outliers based on sliding-window RMS filters
• 'antint' = flag integrations if all baselines to a specified antenna are flagged
• 'extend' = extend and/or grow flags beyond what the basic algorithms detect
• 'summary' = report the amount of flagged data
• 'unflag' = unflag the specified data 

 

Parameter descriptions

vis

Name of input visibility file or calibration table. Default: '' (none). Examples: vis='uid___A002_X2a5c2f_X54.ms', vis='cal-X54.B1'

mode

Mode of operation. Options: 'list', 'manual', 'clip', 'quack', 'shadow', 'elevation', 'tfcrop', 'extend', 'unflag', 'summary'. Default: 'manual'

mode expandable parameters (except mode='list')

field

Select fields in mosaic. Use field id(s) or field name(s). [go listobs to obtain the list id's or names] Default: '' = all fields. If field string is a non-negative integer, it is assumed to be a field index otherwise, it is assumed to be a field name. Examples: field='0~2', field ids 0,1,2; field='0,4,5~7', field ids 0,4,5,6,7; field='3C286,3C295', field named 3C286 and 3C295; field = '3,4C*', field id 3 and all names starting with 4C.

spw

Select data based on spectral window and channels. Default: '' => all spectral windows and channels. Examples: spw='0~2,4', spectral windows 0,1,2,4 (all channels); spw='0:5~61', spw 0, channels 5 to 61; spw='<2', spectral windows less than 2 (i.e. 0,1); spw='0,10,3:3~45', spw 0,10 all channels, spw 3, channels 3 to 45; spw='0~2:2~6'; spw 0,1,2 with channels 2 through 6 in each; spw='0:0~10;15~60'; spectral window 0 with channels 0-10,15-60; spw='0:0~10,1:20~30,2:1;2;3'; spw 0, channels 0-10, spw 1, channels 20-30, and spw 2, channels, 1,2 and 3.

antenna

Select data based on baseline. Default: '' (all). Examples: antenna='DV04&DV06' baseline DV04-DV06; antenna='DV04&DV06DV07&DV10' baselines DV04-DV06 and DV07-DV10; antenna='DV06' all cross-correlation baselines between antenna DV06 and all other available antennas; antenna='DV04,DV06' all baselines with antennas DV04 and DV06; antenna='DV06&&DV06' only the auto-correlation baselines for antenna DV06; antenna='DV04&&*' cross and auto-correlation baselines between antenna DV04 and all other available antennas; antenna='0~2&&&' only the auto-correlation baselines for antennas in range 0~2   

timerange

Select data based on time range. Default: '' (all). Examples: timerange = 'YYYY/MM/DD/hh:mm:ss~YYYY/MM/DD/hh:mm:ss';

timerange='09:14:0~09:54:0' picks 40 min on first day; timerange='25:00:00~27:30:00' picks 1 hr to 3 hr 30min on NEXT day; timerange='09:44:00' pick data within one integration of time; timerange='>10:24:00' data after this time.

correlation

Correlation types or expression. Default: '' (all correlations). For modes clip, tfcrop or rflag, the default means ABS_ALL. If the input is cal table that does not contain a complex data column, the default will fall back to REAL_ALL. Examples: correlation='XX,YY' or options: Any of 'ABS', 'ARG', 'REAL', 'IMAG', 'NORM' followed by any of 'ALL', 'I', 'XX', 'YY', 'RR', 'LL', 'WVR'. 'WVR' refers to the water vapour radiometer of ALMA data. For calibration tables, the solutions are: 'Sol1', 'Sol2', Sol3, Sol4. Correlation selection is not supported for modes other than 'clip', 'tfcrop', or 'rflag' in cal tables.

scan

Scan number range. Default: '' (all). Examples: scan='1~5'. Check 'go listobs' to insure the scan numbers are in order.

intent

Select data based on scan intent. Intent selection is not supported for cal tables. Default: '' (all). Examples: intent='*CAL*,*BAND*'

array

Selection based on the antenna array. Array selection is not supported for cal tables. Default: '' (all).

uvrange

Select data within uvrange (default units meters). Default: '' (all). Examples: uvrange='0~1000klambda', uvrange from 0-1000 kilo-lambda; uvrange='>4klambda', uvranges greater than 4 kilo lambda. uvrange selection is not supported for cal tables.

observation

Selection based on the observation ID. Default: '' (all). Examples: observation='1' or observation=1

feed

Selection based on the feed - NOT IMPLEMENTED YET

 

mode='manual' expandable parameters

Flag according to the data selection specified. This is the default mode (used when the mode is not specified).

autocorr

Flag only the auto-correlations. Note that this parameter is only active when set to True. If set to False it does NOT mean "do not flag auto-correlations". When set to True, it will only flag data from a processor of type CORRELATOR. Default: False. Otions: True, False

 

mode='list' expandable parameters

Flag according to the data selection and flag commands specified in the input list. The input list may come from a text file, a list of text files or from a Python list of strings. Each input line may contain data selection parameters and any parameter specific to the mode given in the line. Default values will be used for the parameters that are not present in the line. Each line will be taken as a command to the task. If data is pre-selected using any of the selection parameters, then flagging will apply only to that subset of the MS.

For optimization and whenever possible, the task will create a union of the data selection parameters present in the list and select only that portion of the MS.

inpfile

Input ASCII file, list of files or a Python list of command strings. Default: ''. Options: [ ] with flag commands or [ ] with filenames or ' ' with a filename.

Example1: The following commands can be saved to a file or group of files and given to the task (e.g., save it to 'flags.txt'): 

scan='1~3' mode='manual'
mode='clip' clipminmax=[0,2] correlation='ABS_XX' clipoutside=False
spw='9' mode='tfcrop' correlation='ABS_YY' ntime=51.0
mode='extend' extendpols=True

or

Example2: The same commands can be given in a Python list on the command line to the task.

reason

Select flag commands based on REASON(s). Can be a string, or list of strings. If inpfile is a list of files, the reasons given in this parameter will apply to all the files. Default: 'any' (all flags regardless of reason). Examples: reason='FOCUS_ERROR'; reason=['FOCUS_ERROR', 'SUBREFLECTOR_ERROR']   

tbuff

A time buffer or list of time buffers to pad the timerange parameters in flag commands. When a list of 2 time buffers is given, it will subtract the first value from the lower time and the second value will be added to the upper time in the range. The 2 time buffer values can be different, allowing to have an irregular time buffer padding to time ranges. If the list contains only one time buffer, it will use it to subtract from t0 and add to t1. If more than one list of input files is given, tbuff will apply to all of the flag commands that have timerange parameters in the files.

Each tbuff value should be a float number given in seconds. Default: 0.0 (it will not apply any time padding). Example: tbuff=[0.5, 0.8] inpfile=['online.txt','userflags.txt']. The timerange parameters in the 'online.txt' file are first converted to seconds. Then, 0.5 is subtracted from t0 and 0.8 is added to t1, where t0 and t1 are the two intervals given in timerange. Similarly, tbuff will be applied to any timerange in 'userflags.txt'.   

 

mode='clip' expandable parameters

Clip data according to values of the following subparameters. The polarization expression is given by the correlation parameter. For calibration tables, the solutions are also given by the correlation parameter.

clipminmax

Range of data (Jy) that will NOT be flagged. It will always flag the NaN/Inf data, even when a range is specified. Default: [ ]. Example: clipminmax=[0.0,1.5]

clipoutside

Clip OUTSIDE the range. Default: True. Example: clipoutside=False, flag data WITHIN the clipminmax range.

clipzeros

Clip zero-value data. Default: False.

 

mode='clip', 'tfcrop', or 'rflag' expandable parameters

datacolumn

Column to use for clipping. Default: 'DATA'. Options: MS columns: 'DATA', 'CORRECTED', 'MODEL', 'RESIDUAL', 'RESIDUAL_DATA', 'WEIGHT_SPECTRUM', 'WEIGHT', 'FLOAT_DATA'. Cal table columns: 'FPARAM', 'CPARAM', 'SNR', 'WEIGHT'.                            

channelavg

Pre-average data across channels before analyzing visibilities for flagging. Partially flagged data is not be included in the average unless all data contributing to a given output channel is flagged. If present, WEIGHT_SPECTRUM/ SIGMA_SPECTRUM are used to compute a weighted average (WEIGHT_SPECTRUM for CORRECTED_DATA and SIGMA_SPECTRUM for DATA). Default: False. Options: True/False                   

chanbin

Bin width for channel average in number of input channels. If a list is given, each bin applies to one of the selected SPWs. When chanbin is set to 1 all input channels are used considered for the average to produce a single output channel, this behaviour aims to be preserve backwards compatibility with the previous pre-averaging feature of clip mode. Default: 1    

timeavg

Pre-average data across time before analyzing visibilities for flagging. Partially flagged data is not be included in the average unless all data contributing to a given output channel is flagged. If present, WEIGHT_SPECTRUM/ SIGMA_SPECTRUM are used to compute a weighted average (WEIGHT_SPECTRUM for CORRECTED_DATA and SIGMA_SPECTRUM for DATA). Otherwise WEIGHT/ SIGMA are used to average together data from different integrations. Default: False. Options: True/False 

timebin

Bin width for time average in seconds. Default: '0s'

 

mode='quack' expandable parameters

Option to remove specified part of scan beginning/end.

quackinterval

Time in seconds from scan beginning or end to flag. Make time slightly smaller than the desired time. Default: 0.0. Type: int or float.

quackmode

Quack mode. Default: 'beg'. Options:

• 'beg'  ==> flag an interval at the beginning of scan
• 'endb' ==> flag an interval at the end of scan
• 'tail' ==> flag all but an interval at the beginning of scan
• 'end'  ==> flag all but an interval at end of scan

Visual representation of quack mode flagging one scan with 1s duration. The following diagram shows what is flagged for each quack mode when quackinterval is set to 0.25s. The flagged part is represented by crosses (+++++++++):

           scan with 1s duration
--------------------------------------------
beg
+++++++++++---------------------------------
                                 endb
---------------------------------+++++++++++
           tail
-----------+++++++++++++++++++++++++++++++++
end
+++++++++++++++++++++++++++++++++-----------

quackincrement

Increment quack flagging in time taking into account flagged data or not. Default: False. Type: bool

• False  ==> the quack interval is counted from the scan boundaries, as determined by the quackmode parameter, regardless if data has been flagged or not.
• True   ==> the quack interval is counted from the first unflagged data in the scan.

 

mode='shadow' expandable parameters

Option to flag data of shadowed antennas. This mode is not available for cal tables.

All antennas in the ANTENNA subtable of the MS (and the corresponding diameters) will be considered for shadow-flag calculations. For a given timestep, an antenna is flagged if any of its baselines (projected onto the uv-plane) is shorter than  radius$_{1}$ $+$ radius$_{2}$ $-$ tolerance. The value of 'w' is used to determine which antenna is behind the other. The phase-reference center is used for antenna-pointing direction.

tolerance

Amount of shadowing allowed (or tolerated), in meters. A positive number allows antennas to overlap in projection. A negative number forces antennas apart in projection. Zero implies a distance of radius$_{1}$ $+$ radius$_{2}$ between antenna centers. Default: 0.0

addantenna

It can be either a file name with additional antenna names, positions and diameters, or a Python dictionary with the same information. You can use the flaghelper functions to create the dictionary from a file. Default: ''. Type: string or {} (dictionary). To create a dictionary inside CASA:

Where antfile is a text file in disk that contains information such as:

name=VLA01
diameter=25.0
position=[-1601144.96146691, -5041998.01971858, 3554864.76811967]
name=VLA02
diameter=25.0
position=[-1601105.7664601889, -5042022.3917835914, 3554847.245159178]

 

mode='elevation' expandable parameters

Option to flag based on antenna elevation. This mode is not available for cal tables.

lowerlimit

Lower limiting elevation in degrees. Data coming from a baseline where one or both antennas were pointing at a strictly lower elevation (as function of time), will be flagged. Default: 0.0

upperlimit

Upper limiting elevation in degrees. Data coming from a baseline where one or both antennas were pointing at a strictly higher elevation (as function of time), will be flagged. Default: 90.0

 

mode='tfcrop', 'rflag', or 'extend' expandable parameters

ntime

Time range (in seconds or minutes) over which to buffer data before running the algorithm. Options: 'scan' or any other float value or string containing the units. Default: 'scan'. Examples: ntime='1.5min'; ntime=1.2 (taken in seconds). The dataset will be iterated through in time-chunks defined here.

combinescans

Accumulate data across scans depending on the value of ntime. Default: False. This parameter should be set to True only when ntime is specified as a time-interval (not 'scan'). When set to True, it will remove SCAN from the sorting columns, therefore it will only accumulate across scans if ntime is not set to 'scan'.

 

mode='tfcrop' expandable parameters

Flag using the TFCrop autoflag algorithm. For each field, spw, timerange (specified by ntime), and baseline:

1.  Average visibility amplitudes along time dimension to form an average spectrum
2. Calculate a robust piece-wise polynomial fit for the band-shape at the base of RFI spikes. Calculate 'stddev' of (data - fit).
3. Flag points deviating from the fit by more than N-stddev
4. Repeat (1-3) along the other dimension.

This algorithm is designed to operate on un-calibrated data (step (2)), as well as calibrated data. It is recommended to extend the flags after running this algorithm. See the sub-parameter extendflags below.

timecutoff

Flag threshold in time. Flag all data-points further than N-stddev from the fit. This threshold catches time-varying RFI spikes (narrow and broad-band), but will not catch RFI that is persistent in time. Default: 4.0.

Flagging is done in up to 5 iterations. The stddev calculation is adaptive and converges to a value that reflects only the data and no RFI. At each iteration, the same relative threshold is applied to detect flags. (Step (3) of the algorithm).

freqcutoff

Flag threshold in frequency. Flag all data-points further than N-stddev from the fit. Same as timecutoff, but along the frequency-dimension. This threshold catches narrow-band RFI that may or may not be persistent in time. Default: 3.0

timefit

Fitting function for the time direction. Default: 'line'. Options: 'line', 'poly'

A 'line' fit is a robust straight-line fit across the entire timerange (defined by ntime). A 'poly' fit is a robust piece-wise polynomial fit across the timerange. 

 

freqfit

Fitting function for the frequency direction. Same as for the timefit parameter. Default: 'poly'. Options: 'line', 'poly'. Choose 'line' only if you are operating on bandpass-corrected data, or residuals, and expect that the bandshape is linear. The 'poly' option works better on uncalibrated bandpasses with narrow-band RFI spikes.

maxnpieces

Maxinum number of pieces to allow in the piecewise-polynomial fits. Default: 7. Options: 1 - 9. This parameter is used only if timefit or freqfit are chosen as 'poly'. If there is significant broad-band RFI, reduce this number. Using too many pieces could result in the RFI being fitted in the clean bandpass. In later stages of the fit, a third-order polynomial is fit per piece, so for best results, please ensure that nchan/maxnpieces is at-least 10.

flagdimension

Choose the directions along which to perform flagging. Default: 'freqtime'; first flag along frequency, and then along time. Options: 'time', 'freq', 'timefreq', 'freqtime'. For most cases, 'freqtime' or 'timefreq' are appropriate, and differences between these choices are apparant only if RFI in one dimension is significantly stronger than the other. The goal is to flag the dominant RFI first. If there are very few (less than 5) channels of data, then choose 'time'. Similarly for 'freq'.

usewindowstats

Use sliding-window statistics to find additional flags. Default: 'none'. Options: 'none', 'sum', 'std', 'both'

The 'sum' option chooses to flag a point, if the mean-value in a window centered on that point deviates from the fit by more than N-stddev $/ 2.0$.

The 'std' option chooses to flag a point, if the 'local' stddev calculated in a window centered on that point is larger than N-stddev $/2.0$. This option is an attempt to catch noisy RFI that is not excluded in the polynomial fits, and which increases the global stddev, and results in fewer flags (based on the N-stddev threshold).

halfwin

Half width of sliding window to use with usewindowstats. Default: 1 (a 3-point window size). Options: 1,2,3

 

mode='tfcrop' or 'rflag' expandable parameters

extendflags

Extend flags along time, frequency and correlation. Default: True

 

mode='rflag' expandable parameters

Detect outliers based on the RFlag algorithm [1]. The polarization expression is given by the correlation parameter. Iterate through the data in chunks of time. For each chunk, calculate local statistics, and apply flags based on user supplied (or auto-calculated) thresholds.

• Time analysis (for each channel):
  • calculate local RMS of real and imaginary visibilities within a sliding time window
  • calculate the median RMS across time windows, deviations of local RMS from this median, and the median deviation
  • flag if local RMS is larger than timedevscale $x$ (medianRMS $+$ medianDev)
• Spectral analysis (for each time):
  • calculate avg of real and imaginary visibilities and their RMS across channels
  • calculate the deviation of each channel from this avg, and the median-deviation
  • flag if deviation is larger than freqdevscale $x$ medianDev

It is recommended to extend the flags after running this algorithm. See the sub-parameter extendflags below.

Notice that by default the flag implementation in CASA is able to calculate the thresholds and apply them on-the-fly (OTF). There is a significant performance gain with this approach, as the visibilities don't have to be read twice, and therefore is highly recommended (see example 1). Otherwise it is possible to reproduce the AIPS usage pattern by doing a first run with action='calculate' and a second run with action='apply'. The advantage of this approach is that the thresholds are calculated using the data from all scans, instead of calculating them for one scan only (see example 3).

Example usage :

1. Calculate thresholds automatically per scan, and use them to find flags. Specify scale-factor for time-analysis thresholds, use default for frequency.

   flagdata('my.ms', mode='rflag', spw='9', timedevscale=4.0)

2. Supply noise-estimates to be used with default scale-factors.

   flagdata(vis='my.ms', mode='rflag', spw='9', timedev=0.1, freqdev=0.5, action='calculate')

3. Two-passes. This replicates the usage pattern in AIPS.
   • The first pass saves commands in an output text files, with auto-calculated thresholds. Thresholds are returned from 'rflag' only when action='calculate'. The user can edit this file before doing the second pass, but the python-dictionary structure must be preserved. The parameters timedevscale and freqdevscale are not used in this first pass.
   • The second pass applies these commands (action='apply').

     flagdata(vis='my.ms', mode='rflag', spw='9,10', timedev='tdevfile.txt', freqdev='fdevfile.txt', action='calculate')

     flagdata(vis='my.ms', mode='rflag', spw='9,10', timedev='tdevfile.txt', freqdev='fdevfile.txt', action='apply')

With action='calculate', display='report' will produce diagnostic plots showing data-statistics and thresholds (the same thresholds as those written out to 'tdevfile.txt' and 'fdevfile.txt'). In this second pass, with action='apply', the parameters freqdevscale and timedevscale can be used to re-scale the thresholds calculated in the first pass.

winsize

Number of timesteps in the sliding time window (fparm(1) in AIPS). Default: 3

timedev

Time-series noise estimate (noise in AIPS). Default: [ ]. Examples: timedev = 0.5: Use this noise-estimate to calculate flags. Do not recalculate; timedev = [[1,9,0.2], [1,10,0.5]]: Use noise-estimate of 0.2 for field 1, spw 9, and noise-estimate of 0.5 for field 1, spw 10; timedev = [ ]: Auto-calculate noise estimates; timedev='timedevfile': Auto-calculate noise estimates and write them into a file with the name given (any string will be interpreted as a file name which will be checked for existence).

freqdev

Spectral noise estimate (scutoff in AIPS). This step depends on having a relatively-flat bandshape. Same parameter-options as timedev. Default: [ ]

timedevscale

For Step 1 (time analysis), flag a point if local RMS around it is larger than timedevscale $x$ timedev (fparm(0) in AIPS). This scale parameter is only applied when flagging (action='apply') and displaying reports (display option). It is not used when the thresholds are simply calculated and saved into files (action='calculate', as in the two-passes usage pattern of AIPS). Default: 5.0

freqdevscale

For Step 2 (spectral analysis), flag a point if local rms around it is larger than freqdevscale $x$ freqdev (fparm(10) in AIPS). Similarly as with timedevscale, freqdevscale is not used when the thresholds are simply calculated and saved into files (action='calculate', as in the two-passes usage pattern of AIPS). Default: 5.0

spectralmax

Flag whole spectrum if freqdev is greater than spectralmax (fparm(6) in AIPS). Default: 1E6

spectralmin

Flag whole spectrum if freqdev is less than spectralmin (fparm(5) in AIPS). Default: 0.0

 

mode='extend' expandable parameters

Extend and/or grow flags beyond what the basic algorithms detect. This mode will extend the accumulated flags available in the MS, regardless of which algorithm created them. It is recommended that any autoflag (tfcrop, rflag) algorithm be followed up by a flag extension. Extensions will apply only within the selected data, according to the settings of extendpols, growtime, growfreq, growaround, flagneartime, and flagnearfreq.

extendpols

Extend flags to all selected correlations. Default: True. Options: True/False. For example, to extend flags from RR to only RL and LR, a data-selection of correlation='RR,LR,RL' is required along with extendpols=True.

growtime

For any channel, flag the entire timerange in the current 2D chunk (set by ntime) if more than X% of the timerange is already flagged. Default: 50.0. Options: 0.0 - 100.0. This option catches the low-intensity parts of time-persistent RFI.

growfreq

For any timestep, flag all channels in the current 2D chunk (set by data-selection) if more than X% of the channels are already flagged. Default: 50.0. Options: 0.0 - 100.0. This option catches broad-band RFI that is partially identified by earlier steps.

growaround

Flag a point based on the number of flagged points around it. Default: False. Options: True/False. For every un-flagged point on the 2D time/freq plane, if more than four surrounding points are already flagged, flag that point. This option catches some wings of strong RFI spikes.

flagneartime

Flag points before and after every flagged one, in the time-direction. Default: False. Options: True/False. Note that this can result in excessive flagging.

flagnearfreq

Flag points before and after every flagged one, in the frequency-direction. Default: False. Options: True/False. This option allows flagging of wings in the spectral response of strong RFI. Note that this can result in excessive flagging.

 

mode='antint' expandable parameters

This mode flag all integrations in which a specified antenna is flagged. This mode operates for an spectral window. It flags any integration in which all baselines to a specified antenna are flagged, but only if this condition is satisfied in a fraction of channels within the spectral window of interest greater than a nominated fraction. For simplicity, it assumes that all polarization products must be unflagged for a baseline to be deemed unflagged. The antint mode implements the flagging approach introduced in 'antintflag' (https://doi.org/10.5281/zenodo.163546)                                                                                
                                                                                                                                       
The motivating application for introducing this mode is removal of data that will otherwise lead to changes in reference antenna during gain calibration, which will in turn lead to corrupted polarization calibration.

antint_ref_antenna

Check the baselines to this antenna. Note that this is not the same as the general 'antenna' parameter of flagdata. The parameter antint_ref_antenna is mandatory with the   'antint' mode and chooses the antenna for which the fraction of channels flagged will be checked.

minchanfrac

Minimum fraction of flagged channels required for a baseline  to be deemed as flagged. Takes values between 0-1 (float). In this mode flagdata does the following for every point in time. It checks the fraction of channels flagged for any of the polarization products and for every baseline to the antenna of interest. If the fraction is higher than this 'minchanfrac' threshold then the data are flagged for this pont in time (this includes all the rows selected with the flagdata command that have that timestamp). This parameter will be ignored if spw specifies a channel.

verbose

Print timestamps of flagged integrations to the log.

 

mode='unflag' expandable parameters

Unflag according to the data selection specified. See here.

mode='summary' expandable parameters

List the number of rows and flagged data points for the MS's meta-data. The resulting summary will be returned as a Python dictionary.

In 'summary' mode, the task returns a dictionary of flagging statistics.

Example1:

s will be a dictionary which contains:

• s['total']: total number of data
• s['flagged']: amount of flagged data

Example2: two summary commands in 'list' mode, intercalating a manual flagging command.

The dictionary returned in s will contain two dictionaries, one for each of the two summary modes.

• s['report0']['name']: 'InitFlags'
• s['report1']['name']: 'Autocorr'

minrel

Minimum number of flags (relative) to include in histogram. Default: 0.0

maxrel

Maximum number of flags (relative) to include in histogram. Default: 1.0

minabs

Minimum number of flags (absolute, inclusive) to include in histogram. Default: 0

maxabs

Maximum number of flags (absolute, inclusive) to include in histogram. To indicate infinity, use any negative number. Default: -1

spwchan

List the number of flags per spw and per channel. Default: False

spwcorr

Llist the number of flags per spw and per correlation. Default: False

basecnt

List the number of flags per baseline. Default: False

fieldcnt

Produce a separated breakdown per field. Default: False

name

Name for this summary, to be used as a key in the returned Python dictionary. It is possible to call the 'summary' mode multiple times in 'list' mode. When calling the 'summary' mode as a command in a list, one can give different names to each one of them so that they can be easily pulled out of the summary's dictionary. Default: 'Summary'

 

action

Action to perform in MS/cal table or in the input list of parameters. Options: 'none', 'apply', 'calculate'. Default: 'apply'

action='apply' or 'calculate' expandable parameters

action='apply' applies the flags to the MS. action='calculate' only calculates the flags but does not write them to the MS. This is useful if used together with the display to analyze the results before writing to the MS.

display

Display data and/or end-of-MS reports at run-time. It needs to read a datacolumn for the plotting. The default for an MS is DATA, but the task will use FLOAT_DATA for a Single-dish MS. Default: 'none'. Options: 'none', 'data', 'report', 'both'

• 'none' --> It will not display anything.
• 'data' --> display data and flags per-chunk at run-time, within an interactive GUI.
• This option opens a GUI to show the 2D time-freq planes of the data with old and new flags, for all correlations per baseline.
• The GUI allows stepping through all baselines (prev/next) in the current chunk (set by ntime), and stepping to the next-chunk.
• The flagdata task can be quit from the GUI, in case it becomes obvious that the current set of parameters is just wrong.
• There is an option to stop the display but continue flagging.
• 'report' --> displays end-of-MS reports on the screen.
• 'both' --> displays data per chunk and end-of-MS reports on the screen

 

action='apply' expandable parameters

flagbackup

Automatically backup flags before running the tool. Flagversion names are chosen automatically, and are based on the mode being used. Default: True. Options: True/False

 

action='' or 'none' description

When set to empty or 'none', the underlying tool will not be executed and no flags will be produced. No data selection will be done either. This is useful when used together with the parameter savepars to only save the current parameters (or list of parameters) to the FLAG_CMD sub-table or to an external file.  

 

savepars

Save the current parameters to the FLAG_CMD table of the MS or to an output text file.

Note that when display is set to anything other than 'none', savepars will be disabled. This is done because in an interactive mode, the user may skip data which may invalidate the initial input parameters and there is no way to save the interactive commands. When the input is a calibration table it is only possible to save the parameters to a file.

Default: False. Options: True/False

savepars=True expandable parameters

cmdreason

A string containing a reason to save to the FLAG_CMD table or to an output text file given by the outfile sub-parameter. If the input contains any reason, they will be replaced with this one. At the moment it is not possible to add more than one reason. Default: ' ', no reason will be added to output. Examples: cmdreason='CLIP_ZEROS'

outfile

Name of output file to save the current parameters. Default: ' ', will save the parameters to the FLAG_CMD table of the MS. Examples: outfile='flags.txt' will save the parameters in a text file.

overwrite

Overwrite the existing file given in outfile. Options: True/False. Default: True, it will remove the existing file given in outfile and save the current flag commands to a new file with the same name. When set to False, the task will exit with an error message if the file exist.

 

SYNTAX FOR COMMANDS GIVEN IN A FILE or LIST OF STRINGS

Basic Syntax Rules

1. Commands are strings (which may contain internal "strings") consisting of KEY=VALUE pairs separated by one whitespace only.

1. Use only ONE white space to separate the parameters (no commas). Each key should only appear once on a given command line/string.
2. There is an implicit mode for each command, with the default being 'manual' if not given.
3. Comment lines can start with '#' and will be ignored. The parser used in flagdata will check each parameter name and type and exit with an error if the parameter is not a valid flagdata parameter or of a wrong type.

Example:

scan='1~3' mode='manual'
# this line will be ignored
spw='9' mode='tfcrop' correlation='ABS_XX,YY' ntime=51.0
mode='extend' extendpols=True
scan='1~3,10~12' mode='quack' quackinterval=1.0

 

Citation Number	1
Citation Text	Greisen, Eric, Dec 31, 2011. AIPS documentation: Section E.5 of the AIPS cookbook (Appendix E: Special Considerations for EVLA data calibration and imaging in AIPS, http://www.aips.nrao.edu/cook.html#CEE )