0.1.18 flagdata
Requires:
Synopsis All purpose flagging task based on selections
Description
The task will select a subset of data explicitly for flagging, quacking, clipping,
autocorrelation flagging, or flagging of shadowed antennas. Unflagging is also
available.
In a dual polarization data set, each polarization can be flagged separately.
However, at present the calibration and imaging will only use data with both
parallel hands unflagged.
There is a vector mode of operation to execute several flagging options in one
execution.
Arguments
|
|
|
|
| Inputs | |
|
| vis | | Name of file to flag
|
| | | allowed: | string |
| | | Default: | |
| mode | | Mode
(manualflag,shadow,quack,summary,autoflag,alma,rfi)
|
| | | allowed: | string |
| | | Default: | manualflag |
| spw | | spectral-window/frequency/channel
|
| | | allowed: | any |
| | | Default: | variant |
| field | | Field names or field index numbers: ”==¿all,
field=’0 2,3C286’
|
| | | allowed: | any |
| | | Default: | variant |
| selectdata | | More data selection parameters (antenna, timerange etc)
|
| | | allowed: | bool |
| | | Default: | True |
| antenna | | antenna/baselines: ”==¿all, antenna = ’3,VA04’
|
| | | allowed: | any |
| | | Default: | variant |
| uvrange | | uv range: ”==¿all; uvrange = ’0 100klambda’, default
units=meters |
| | | allowed: | any |
| | | Default: | variant |
| timerange | | time range: ”==¿all, timerange=’09:14:0 09:54:0’
|
| | | allowed: | any |
| | | Default: | variant |
| correlation | | Select data based on correlation
|
| | | allowed: | any |
| | | Default: | variant |
| scan | | scan numbers: ”==¿all
|
| | | allowed: | any |
| | | Default: | variant |
| feed | | multi-feed numbers: Not yet implemented
|
| | | allowed: | any |
| | | Default: | variant |
| array | | (sub)array numbers: ”==¿all
|
| | | allowed: | any |
| | | Default: | variant |
| clipexpr | | Expression to clip on
|
| | | allowed: | any |
| | | Default: | variant ABS RR |
| clipminmax | | Range to use for clipping
|
| | | allowed: | any |
| | | Default: | variant |
| clipcolumn | | Data column to use for clipping
|
| | | allowed: | any |
| | | Default: | variant DATA |
| clipoutside | | Clip outside the range, or within it
|
| | | allowed: | any |
| | | Default: | variant True |
| quackinterval | | Quack n seconds from scan beginning/end
|
| | | allowed: | any |
| | | Default: | variant 0.0 |
| quackmode | | Quack mode. ’beg’ ==¿ beginning of scan. ’endb’ ==¿
end of scan. ’end’ ==¿ all but end of scan. ’tail’ ==¿ all
but beginning of scan
|
| | | allowed: | any |
| | | Default: | variant beg |
| quackincrement | | Flag incrementally in time?
|
| | | allowed: | any |
| | | Default: | variant False |
| autocorr | | Flag autocorrelations
|
| | | allowed: | any |
| | | Default: | variant False |
| unflag | | Unflag the data specified
|
| | | allowed: | any |
| | | Default: | variant False |
| algorithm | | Autoflag algorithm (timemed,freqmed)
|
| | | allowed: | string |
| | | Default: | timemed |
| recipe | | Flagging algorithm (pdb_recipe,vla_recipe)
|
| | | allowed: | string |
| | | Default: | vla_recipe |
| column | | Data column to operate on.
|
| | | allowed: | string |
| | | Default: | DATA |
| expr | | Expression to flag on
|
| | | allowed: | string |
| | | Default: | ABS RR |
| thr | | Flagging threshold (n sigma)
|
| | | allowed: | double |
| | | Default: | 5.0 |
| window | | Sliding median filter width
|
| | | allowed: | int |
| | | Default: | 10 |
| diameter | | Effective diameter (m) to use. -1 ==¿antenna diameter
|
| | | allowed: | double |
| | | Default: | -1.0 |
| flux | | Flux calibrator field index number: flux=’0’
|
| | | allowed: | string |
| | | Default: | |
| bpass | | Bandpass calibrator field index number: bpass=’0’
|
| | | allowed: | string |
| | | Default: | |
| gain | | Gain calibrator field index number: gain=’0’
|
| | | allowed: | string |
| | | Default: | |
| time_amp_cutoff | | Flagging thresholds in units of sigma, where sigma is the
stdev of the ”fit”
|
| | | allowed: | double |
| | | Default: | 4.0 |
| freq_amp_cutoff | | Flagging thresholds in units of sigma, where sigma is the
stdev of the ”fit”
|
| | | allowed: | double |
| | | Default: | 3.0 |
| freqlinefit | | Fit the bandpass with a straight line (True), or fit the
bandpass with a piecewise polynomial (False)
|
| | | allowed: | bool |
| | | Default: | False |
| auto_cross | | flag on cross-correlations and auto-correlations (0 : only
autocorrelations)
|
| | | allowed: | int |
| | | Default: | 1 |
| num_time | | number of time-steps in each chunk
|
| | | allowed: | int |
| | | Default: | 400 |
| start_chan | | channel range start (1 based)
|
| | | allowed: | int |
| | | Default: | 1 |
| end_chan | | channel range end (1 based)
|
| | | allowed: | int |
| | | Default: | 2048 |
| bs_cutoff | | baseline tolerance
|
| | | allowed: | double |
| | | Default: | 0.0 |
| ant_cutoff | | total autocorrelation amplitude threshold for a
functional antenna
|
| | | allowed: | double |
| | | Default: | 0.0 |
| showplots | | Show plots of each time-frequency chunk
|
| | | allowed: | bool |
| | | Default: | False |
| flag_level | | 0: flag only what is found. 1: extend flags one timestep
before and after. 2: extend flags one timestep and
channel before/after
|
| | | allowed: | int |
| | | Default: | 1 |
| |
Returns
void
Example
The task will flag a subset of data explicitly with the modes:
manualflag = flagging based on specific selection parameter
plus clipping and flagging autcorrelations
summary = report the amount of flagged data
quack = remove/keep specific time range at scan beginning/end
shadow = remove antenna-shadowed data
alma = ALMA pipeline auto-flagging
rfi = Radio Frequency Interference auto-flagging
autoflag = experimental auto-flagging outliers
Unflagging is also available with options manualflag and quack.
Note: I a dual polarization data set, each polarization can be flagged
separately. However, at present the calibration and imaging will
only use only data with both parallel hands unflagged.
Keyword arguments:
vis -- Name of input visibility file
default: none example: vis=’ngc5921.ms’
mode -- Mode of operation.
options: ’manualflag’,’autoflag’,’summary’,’quack’,’shadow’,’alma’
default: ’manualflag’
manualflag -- option does data-selected flagging.
If autocorr = F; clipminmax = [], then
flagging or unflagging [unflag=F, unflag=T] is specified by
the data selection parameters.
Other flagging options, with the appropriate data selection
parameters, that are available are:
autocorr -- Flag autocorrelations
default: False
options: True,False
clipexpr -- Clip using the following:
default: ’ABS RR’; example: clipexpr=’RE XX’
Options: ’ABS’,’ARG’,’RE’,’IM’,’NORM’ + ’ ’ +
’I’,’XX’,’YY’,’RR’,’LL’
clipminmax -- Range of data (Jy) that will NOT be flagged
default: [] means do not use clip option
example: [0.0,1.5]
clipcolumn -- Column to use for clipping.
default: ’DATA’
options: ’DATA’,’CORRECTED’,’MODEL’
clipoutside -- Clip OUTSIDE the range ?
default: True
example: False -> flag data WITHIN the range.
unflag -- Unflag the specified data
default: False (i.e. flag); example: unflag=True
quack: option to remove specified part of scan beginning
quackinterval -- Time in seconds from scan beginning/end to flag
Make time slightly smaller than the desired time
quackmode -- Quack mode
default: ’beg’
’beg’ ==> beginning of scan
’endb’ ==> end of scan.
’tail’ ==> all but beginning of scan
’end’ ==> all but end of scan.
quackincrement -- Quack incrementally in time?
default: False
True ==> the quack interval is counted from the
beginning of the scan
False ==> the quack interval is counted from the
first unflagged data in the scan
unflag -- Unflag the data
default => False (quack as indicated)
True (unquack)
autocorr -- Flag autocorrelations (independent option)
default => False
shadow: option to flag data of shadowed antennas
diameter -- effective diameter to use when determining if an antenna
is shadowed at a given time stamp. The given value is used
for all antennas. If this is set to a negative number
(default), the actual antenna diameters are used.
summary: lists number of rows and data points flagged
No subparameters
autoflag: experimental automatic flagging of outliers.
It is still under development
algorithm -- autoflag algorithm name
default: ’timemed’
options: ’timemed’,’freqmed’
column -- the column on which to operate (DATA, CORRECTED, MODEL)
expr -- expression to use for flagging option
default: ’ABS RR’; example: expr=’RE XX’
Options: ’ABS’,’ARG’,’RE’,’IM’,’NORM’ + ’ ’ +
’I’,’XX’,’YY’,’RR’,’LL’
thr -- flagging threshold as a multiple of standard-deviation ( n sigma )
window -- half width for sliding window median filters. The size should be
about the number of integration in a scan, or the number of
spectral channels.
rfi: RFI automatic flagging of outliers.
For more information, see ???
clipcolumn -- the column on which to operate (DATA, CORRECTED, MODEL)
clipexpr -- expression to use for flagging option
default: ’ABS RR’; example: expr=’RE XX’
Options: ’ABS’,’ARG’,’RE’,’IM’,’NORM’ + ’ ’ +
’I’,’XX’,’YY’,’RR’,’LL’
alma: Automatic flagging algorithm from the ALMA heuristics pipeline.
For more information, see ???
Note: selection parameters are *not* used in this mode. The algorithm
always run on the full MS.
recipe -- which algorithm to use
default: ’vla_recipe’
options: ’vla_recipe’, ’pdb_recipe’
flux -- which field to use as flux calibrator
bpass -- which field to use as bandpass calibrator
gain -- which field to use as gain calibrator
selectdata -- Other data selection parameters
default: True
antenna -- Select data based on baseline
default: ’’ (all); example: antenna=’5&6’ baseline 5-6
antenna=’5&6;7&8’ #baseline 5-6 and 7-8
antenna=’5’ # all baselines with antenna 5
antenna=’5,6’ # all baselines with antennas 5 and 6
spw -- Select data based on spectral window and channels
default: ’’ (all); example: spw=’1’
spw=’<2’ #spectral windows less than 2
spw=’>1’ #spectral windows greater than 1
spw=’0:0~10’ # first 10 channels from spw 0
spw=’0:0~5;56~60’ # multiple separated channel chunks.
correlation -- Correlation types
default: ’’ (all);
example: correlation=’RR LL’
field -- Select data based on field id(s) or name(s)
default: ’’ (all); example: field=’1’
field=’0~2’ # field ids inclusive from 0 to 2
field=’3C*’ # all field names starting with 3C
uvrange -- Select data within uvrange (default units meters)
default: ’’ (all); example:
uvrange=’0~1000klambda’; uvrange from 0-1000 kilo-lamgda
uvrange=’>4klamda’;uvranges greater than 4 kilo-lambda
uvrange=’0~1000km’; uvrange in kilometers
timerange -- Select data based on time range:
default = ’’ (all); example,
timerange = ’YYYY/MM/DD/hh:mm:ss~YYYY/MM/DD/hh:mm:ss’
Note: YYYY/MM/DD can be dropped as needed:
timerange=’09:14:0~09:54:0’ # this time range
timerange=’09:44:00’ # data within one integration of time
timerange=’>10:24:00’ # data after this time
timerange=’09:44:00+00:13:00’ #data 13 minutes after time
scan -- Select data based on scan number
default: ’’ (all); example: scan=’>3’
feed -- Selection based on the feed - NOT IMPLEMENTED YET
array -- Selection based on the antenna array
-- Vector mode --
For reasons of performance (to reduce the number loops through the MS),
several flagdata task invocations can be combined into a single
flagdata() run by giving vectors of parameters.
This is possible for the modes ’manualflag’ and ’quack’, only.
For example, the following three flagdata runs:
vis = ’my.ms’
mode = ’manualflag’
selectdata = True
flagdata(vis=’my.ms’, mode=’manualflag’, field=’3’, autocorr=True)
flagdata(vis=’my.ms’, mode=’manualflag’, field=’3’, timerange = ’6:0:0~6:23:00’)
flagdata(vis=’my.ms’, mode=’manualflag’, field=’3’, scan=’0’, spw=’0:60;62;64’)
can be combined into a single run by:
vis = ’my.ms’
mode = ’manualflag’
selectdata = True
field = ’3’
spw = [ ’’ , ’’ , ’0:60;62:64’ ]
autocorr = [ True , False , False ]
timerange = [ ’’ , ’6:0:0~6:23:0’, ’’ ]
scan = [ ’’ , ’’ , ’0’ ]
flagdata()
Note that field=’3’ is equivalent to field=[’3’,’3’,’3’]
Please send any comments or questions about CASA or AIPS++
to aips2-requests@nrao.edu
Copyright © 2008 Associated Universities Inc.,
Washington, D.C.
This code is available under the terms of the GNU General Public Lincense
Home |
Contact Us |
Directories |
Site Map |
Help |
Privacy Policy |
Search
Updated daily during alpha development.