NRAO Home > CASA > CASA Task Reference Manual

0.1.112 sdreduceold

Requires:

Synopsis
ASAP SD task: do sdcal, sdaverage, and sdbaseline in one task

Description

Task sdreduceold performs data selection, calibration, spectral averaging and/or baseline fitting for single-dish spectra. This task internally calls the tasks, sdcal, sdaverage, and sdbaseline and it can be used to run all the three steps in one task execution. This task has better performance than invoking the three tasks separately because it runs all three steps without writing intermediate data to disk.

It is possible to skip arbitrary operations by setting calmode = ’none’ (for calibration), average=False (for time and polarization averaging), kernel = ’none’ (for smoothing), and/or blfunc=’none’ (for baseline fitting).

Please take a look at descriptions of tasks, sdcal, sdaverage, and sdbalseline, for more information.

Arguments





Inputs

infile

name of input SD dataset

allowed:

string

Default:

antenna

select an antenna name or ID, e.g. ’PM03’ (only effective for MS input)

allowed:

any

Default:

variant 0

fluxunit

units of the flux (”=current)

allowed:

string

Default:

telescopeparam

parameters of telescope for flux conversion (see description in help of sdcal)

allowed:

string

Default:

field

select data by field IDs and names, e.g. ’3C2*’ (”=all)

allowed:

string

Default:

spw

select data by IF IDs (spectral windows), e.g. ’3,5,7’ (”=all)

allowed:

string

Default:

restfreq

the rest frequency, e.g. ’1.41GHz’ (default unit: Hz) (see examples in help)

allowed:

any

Default:

variant

frame

frequency reference frame (”=current)

allowed:

string

Default:

doppler

doppler convention (”=current). Effective only when spw selection is in velocity unit.

allowed:

string

Default:

timerange

select data by time range, e.g. ’09:14:0~09:54:0’ (”=all) (see examples in help of sdcal)

allowed:

string

Default:

scan

select data by scan numbers, e.g. ’21~23’ (”=all)

allowed:

string

Default:

pol

select data by polarization IDs, e.g. ’0,1’ (”=all)

allowed:

string

Default:

calmode

SD calibration mode (’none’ = skip calibration)

allowed:

string

Default:

none

fraction

fraction of the OFF data to mark as OFF spectra, e.g., ’10%’

allowed:

any

Default:

variant 10%

noff

number of the OFF data to mark (-1 = use fraction instead of number)

allowed:

int

Default:

-1

width

width of the pixel for edge detection

allowed:

double

Default:

0.5

elongated

the observed area is elongated in one direction

allowed:

bool

Default:

False

markonly

do calibration (False) or just mark OFF (True)

allowed:

bool

Default:

False

plotpointings

plot pointing direction for ON and OFF

allowed:

bool

Default:

False

tau

the zenith atmospheric optical depth for correction (0. = no correction)

allowed:

double

Default:

0.0

average

data averaging [True, False]

allowed:

bool

Default:

False

timeaverage

average spectra over time [True, False] (see examples in help of sdaverage)

allowed:

bool

Default:

False

tweight

weighting for time averaging

allowed:

string

Default:

tintsys

scanaverage

average spectra within a scan number [True, False] (see examples in help of sdaverage)

allowed:

bool

Default:

False

averageall

set True only when averaging spectra with different spectral resolutions

allowed:

bool

Default:

False

polaverage

average spectra over polarizations [True, False]

allowed:

bool

Default:

False

pweight

weighting for polarization averaging

allowed:

string

Default:

tsys

kernel

type of spectral smoothing kernel (’none’=no smoothing)

allowed:

string

Default:

none

kwidth

width of smoothing kernel in channels

allowed:

int

Default:

5

chanwidth

width of regridded channels

allowed:

string

Default:

5

maskmode

mode of setting additional channel masks

allowed:

string

Default:

auto

thresh

S/N threshold for linefinder

allowed:

double

Default:

5.0

avg_limit

channel averaging for broad lines

allowed:

int

Default:

4

edge

channels to drop at beginning and end of spectrum

allowed:

intArray

Default:

0

blfunc

baseline model function (’none’ = skip baseline fit)

allowed:

string

Default:

none

order

order of baseline model function

allowed:

int

Default:

5

npiece

number of element polynomials for cubic spline curve

allowed:

int

Default:

2

applyfft

automatically set wave numbers of sinusoids [True, False]

allowed:

bool

Default:

True

fftmethod

method for automatically set wave numbers of sinusoids

allowed:

string

Default:

fft

fftthresh

threshold to select wave numbers of sinusoids

allowed:

any

Default:

3.0

addwn

additional wave numbers to use

allowed:

any

Default:

0

rejwn

wave numbers NOT to use

allowed:

any

Default:

clipthresh

clipping threshold for iterative fitting

allowed:

double

Default:

3.0

clipniter

maximum iteration number for iterative fitting

allowed:

int

Default:

0

verifycal

interactively verify the results of calibration [True, False] (see description in sdcal)

allowed:

bool

Default:

False

verifysm

interactively verify the results of smoothing for each spectrum. [not available for kernel=”regrid”]

allowed:

bool

Default:

False

verifybl

interactively verify the results of baseline fitting for each spectrum (only for blfunc=”poly”. see description in help)

allowed:

bool

Default:

False

verbosebl

output baseline fitting results to logger [True, False]

allowed:

bool

Default:

True

bloutput

output baseline fitting results to a text file [True, False]

allowed:

bool

Default:

True

blformat

format of the text file specified with bloutput

allowed:

string

Default:

showprogress

show progress status for large data [True, False]

allowed:

bool

Default:

True

minnrow

minimum number of input spectra to show progress status in baseline fitting

allowed:

int

Default:

1000

outfile

name of output file (See a WARNING in help)

allowed:

string

Default:

outform

output file format (See a WARNING in help)

allowed:

string

Default:

ASAP

overwrite

overwrite the output file if already exists [True, False]

allowed:

bool

Default:

False

plotlevel

plot and summarize results (0=none). See description in each task

allowed:

int

Default:

0

Returns
void

Example

 
Keyword arguments:  
infile -- name of input SD dataset  
antenna -- select an antenna name or ID  
        default: 0  
        example: ’PM03’  
        NOTE this parameter is effective only for MS input  
fluxunit -- units for line flux  
        options: ’K’,’Jy’,’’  
        default: ’’ (keep current fluxunit in data)  
        WARNING: For GBT data, see description below.  
    >>> fluxunit expandable parameter  
        telescopeparam -- parameters of telescope for flux conversion  
                options: (str) name or (list) list of gain info  
                default: ’’ (none set)  
                example: if telescopeparam=’’, it tries to get the telescope  
                         name from the data.  
                         Full antenna parameters (diameter,ap.eff.) known  
                         to ASAP are  
                         ’ATPKSMB’, ’ATPKSHOH’, ’ATMOPRA’, ’DSS-43’,  
                         ’CEDUNA’,’HOBART’. For GBT, it fixes default fluxunit  
                         to ’K’ first then convert to a new fluxunit.  
                         telescopeparam=[104.9,0.43] diameter(m), ap.eff.  
                         telescopeparam=[0.743] gain in Jy/K  
                         telescopeparam=’FIX’ to change default fluxunit  
                         see description below  
 
field -- select data by field IDs and names  
        default: ’’ (use all fields)  
        example: field=’3C2*’ (all names starting with 3C2)  
                 field=’0,4,5~7’ (field IDs 0,4,5,6,7)  
                 field=’0,3C273’ (field ID 0 or field named 3C273)  
        this selection is in addition to the other selections to data  
spw -- select data by IF IDs (spectral windows)/channels  
       NOTE channel range selections are interpreted as mask regions to  
       INCLUDE in BASELINE fit, and ignored in the other operations.  
       when maskmode is ’auto’ or ’interact’, the channel mask  
       will be applied first before fitting as base mask  
        default: ’’ (use all IFs and channels)  
        example: spw=’3,5,7’ (IF IDs 3,5,7; all channels)  
                 spw=’<2’ (IF IDs less than 2, i.e., 0,1; all channels)  
                 spw=’30~45GHz’ (IF IDs with the center frequencies in range 30-45GHz; all channels)  
                 spw=’0:5~61’ (IF ID 0; channels 5 to 61)  
                 spw=’3:10~20;50~60’ (select multiple channel ranges within IF ID 3)  
                 spw=’3:10~20,4:0~30’ (select different channel ranges for IF IDs 3 and 4)  
                 spw=’1~4;6:15~48’ (for channels 15 through 48 for IF IDs 1,2,3,4 and 6)  
        this selection is in addition to the other selections to data  
    >>> spw expandable parameters  
        restfreq -- the rest frequency  
                    available type includes float, int, string, list of float,  
                    list of int, list of string, and list of dictionary. the  
                    default unit of restfreq in case of float, int, or string  
                    without unit is Hz. string input can be a value only  
                    (treated as Hz) or a value followed by unit for which ’GHz’,  
                    ’MHz’,’kHz’,and ’Hz’ are available.  
                    a list can be used to set different rest frequencies for  
                    each IF. the length of list input must be number of IFs.  
                    dictionary input should be a pair of line name and  
                    frequency with keys of ’name’ and ’value’, respectively.  
                    values in the dictionary input follows the same manner as  
                    as for single float or string input.  
                example: 345.796  
                         ’1420MHz’  
                         [345.8, 347.0, 356.7]  
                         [’345.8MHz’, ’347.0MHz’, ’356.7MHz’]  
                         [{’name’:’CO’,’value’:345}]  
        frame -- frequency reference frame  
                options: ’LSRK’, ’TOPO’, ’LSRD’, ’BARY’, ’GALACTO’, ’LGROUP’, ’CMB’  
                default: ’’ (keep current frame in data)  
        doppler -- doppler convention (effective only when spw is in  
                   velocity unit)  
                options: ’RADIO’, ’OPTICAL’, ’Z’, ’BETA’, or ’GAMMA’  
                default: ’’ (keep current doppler setting in data)  
 
timerange -- select data by time range  
        default: ’’ (use all)  
        example: timerange = ’YYYY/MM/DD/hh:mm:ss~YYYY/MM/DD/hh:mm:ss’  
                 Note: YYYY/MM/DD can be dropped as needed:  
                 timerange=’09:14:00~09:54:00’ # this time range  
                 timerange=’09:44:00’ # data within one integration of time  
                 timerange=’>10:24:00’ # data after this time  
                 timerange=’09:44:00+00:13:00’ #data 13 minutes after time  
        this selection is in addition to the other selections to data  
scan -- select data by scan numbers  
        default: ’’ (use all scans)  
        example: scan=’21~23’ (scan IDs 21,22,23)  
        this selection is in addition to the other selections to data  
pol -- select data by polarization IDs  
        default: ’’ (use all polarizations)  
        example: pol=’0,1’ (polarization IDs 0,1)  
        this selection is in addition to the other selections to data  
 
calmode -- calibration mode  
        options: ’ps’,’nod’,’otf’,’otfraster’,  
                 ’fs’,’quotient’,’none’  
        default: ’none’  
        example: choose mode ’none’ if you have already calibrated  
                 and want to correct for atmospheric opacity defined  
                 by tau, subtract baseline or average/smooth spectra.  
    >>> calmode expandable parameter  
         fraction -- edge marker parameter of ’otf’ and ’otfraster’.  
                     Specify a number of OFF integrations (at each  
                     side of the raster rows in ’otfraster’ mode)  
                     as a fraction of total number of integrations.  
                     In ’otfraster’ mode, number of integrations  
                     to be marked as OFF, n_off, is determined by  
                     the following formula,  
 
                        n_off = floor(fraction * n),  
 
                     where n is number of integrations per raster  
                     row. Note that n_off from both sides will be  
                     marked as OFF so that twice of specified  
                     fraction will be marked at most. For example,  
                     if you specify fraction=’10%’, resultant  
                     fraction of OFF integrations will be 20% at  
                     most.  
                     In ’otf’ mode, n_off is given by,  
 
                        n_off = floor(fraction * n),  
 
                     where n is number of total integrations.  
                     n_off is used as criterion of iterative marking  
                     process. Therefore, resulting total number of  
                     OFFs will be larger than n_off. In practice,  
                     fraction is a geometrical fraction of edge  
                     region. Thus, if integrations are concentrated  
                     on edge region (e.g. some of Lissajous  
                     patterns), then resulting n_off may be  
                     unexpectedly large.  
                 default: ’10%’  
                 options: ’20%’ in string style or float value less  
                          than 1.0 (e.g. 0.15).  
                          ’auto’ is available only for ’otfraster’.  
         noff -- edge marking parameter for ’otfraster’.  
                 It is used to specify a number of OFF scans near  
                 edge directly. Value of noff comes before setting  
                 by fraction. Note that n_off from both sides will  
                 be marked as OFF so that twice of specified noff  
                 will be marked at most.  
                 default: -1 (use fraction)  
                 options: any positive integer  
         width -- edge marking parameter for ’otf’.  
                  Pixel width with respect to a median spatial  
                  separation between neighboring two data in time.  
                  Default will be fine in most cases.  
                 default: 0.5  
                 options: float value  
         elongated -- edge marking parameter for ’otf’.  
                      Set True only if observed area is elongeted  
                      in one direction.  
                 options: (bool) True, False  
                 default: False  
         markonly -- set True if you want to save data just after  
                     edge marking (i.e. uncalibrated data) to see  
                     how OFF scans are defined.  
                 options: (bool) True, False  
                 default: False  
         plotpointings -- load plotter and plot pointing directions of  
                          ON and OFF scans.  
                 options: (bool) True, False  
                 default: False  
 
tau -- the zenith atmospheric optical depth for correction  
        default: 0.0 (no correction)  
average -- averaging on spectral data  
        options: (bool) True,False  
        default: False  
 
    >>>average expandable parameter  
        timeaverage -- average spectra over time  
                options: (bool) True, False  
                default: False  
                example: if True, this happens after calibration  
        tweight -- weighting for time averaging (effective only when  
                   timeaverage=True)  
                options: ’var’   (1/var(spec) weighted)  
                         ’tsys’  (1/Tsys**2 weighted)  
                         ’tint’  (integration time weighted)  
                         ’tintsys’  (Tint/Tsys**2)  
                         ’median’  ( median averaging)  
                default: ’tintsys’  
        scanaverage -- average spectra within a scan number (effective  
                       only when timeaverage=True)  
                       when True, spectra are NOT averaged over  
                       different scan numbers.  
                options: (bool) True, False  
                default: False  
        averageall -- average multi-resolution spectra (effective only  
                      when timeaverage=True)  
                      spectra are averaged by referring their frequency  
                      coverage  
                 default: False  
        polaverage -- average spectra over polarizations  
                options: (bool) True, False  
                default: False  
        pweight -- weighting for polarization averaging (effective only  
                   when polaverage=True)  
                options: ’var’  (1/var(spec) weighted)  
                         ’tsys’ (1/Tsys**2 weighted)  
                default: ’tsys’  
 
kernel -- type of spectral smoothing kernel  
        options: ’none’, ’hanning’,’gaussian’,’boxcar’,’regrid’, ’’(=’none’)  
        default: ’none’ (no smoothing)  
 
    >>>kernel expandable parameter  
        kwidth -- width of spectral smoothing kernel  
                options: (int) in channels  
                default: 5  
        example: 5 or 10 seem to be popular for boxcar  
                 ignored for hanning (fixed at 5 chans)  
                         (0 will turn off gaussian or boxcar)  
        chanwidth -- channel width of regridded spectra  
         default: ’5’ (in channels)  
         example: ’500MHz’, ’0.2km/s’  
 
maskmode -- mode of setting additional channel masks  
        options: ’auto’, ’list’, or ’interact’  
        default: ’auto’  
        example: maskmode=’auto’ runs linefinder to detect line regions  
                 to be excluded from fitting. this mode requires three  
                 expandable parameters: thresh, avg_limit, and edge.  
                 USE WITH CARE! May need to tweak the expandable parameters.  
                 maskmode=’list’ uses the given masklist only: no additional  
                 masks applied.  
                 maskmode=’interact’ allows users to manually modify the  
                 mask regions by dragging mouse on the spectrum plotter GUI.  
                 use LEFT or RIGHT button to add or delete regions,  
                 respectively.  
 
    >>> maskmode expandable parameters  
        thresh -- S/N threshold for linefinder. a single channel S/N ratio  
                  above which the channel is considered to be a detection.  
                default: 5  
        avg_limit -- channel averaging for broad lines. a number of  
                     consecutive channels not greater than this parameter  
                     can be averaged to search for broad lines.  
                default: 4  
        edge -- channels to drop at beginning and end of spectrum  
                default: 0  
                example: edge=[1000] drops 1000 channels at beginning AND end.  
                         edge=[1000,500] drops 1000 from beginning and 500  
                         from end.  
        Note: For bad baselines threshold should be increased,  
        and avg_limit decreased (or even switched off completely by  
        setting this parameter to 1) to avoid detecting baseline  
        undulations instead of real lines.  
 
blfunc -- baseline model function  
        options: ’poly’, ’chebyshev’, ’cspline’, or ’sinusoid’  
        default: ’none’ (no baseline fit)  
        example: blfunc=’poly’ uses a single polynomial line of  
                 any order which should be given as an expandable  
                 parameter ’order’ to fit baseline.  
                 blfunc=’chebyshev’ uses Chebyshev polynomials.  
                 blfunc=’cspline’ uses a cubic spline function, a piecewise  
                 cubic polynomial having C2-continuity (i.e., the second  
                 derivative is continuous at the joining points).  
                 blfunc=’sinusoid’ uses a combination of sinusoidal curves.  
    >>> blfunc expandable parameters  
        order -- order of baseline model function  
                options: (int) (<0 turns off baseline fitting)  
                default: 5  
                example: typically in range 2-9 (higher values  
                         seem to be needed for GBT)  
        npiece -- number of the element polynomials of cubic spline curve  
                options: (int) (<0 turns off baseline fitting)  
                default: 2  
        applyfft -- automatically set wave numbers of sinusoidal functions  
                    for fitting by applying some method like FFT.  
                options: (bool) True, False  
                default: True  
        fftmethod -- method to be used when applyfft=True. Now only  
                     ’fft’ is available and it is the default.  
        fftthresh -- threshold to select wave numbers to be used for  
                     sinusoidal fitting. both (float) and (str) accepted.  
                     given a float value, the unit is set to sigma.  
                     for string values, allowed formats include:  
                     ’xsigma’ or ’x’ (= x-sigma level. e.g., ’3sigma’), or  
                     ’topx’ (= the x strongest ones, e.g. ’top5’).  
                default is 3.0 (unit: sigma).  
        addwn -- additional wave number(s) of sinusoids to be used  
                 for fitting.  
                 (list) and (int) are accepted to specify every  
                 wave numbers. also (str) can be used in case  
                 you need to specify wave numbers in a certain range.  
                 default: [0] (i.e., constant is subtracted at least)  
                 example: 0  
                          [0,1,2]  
                          ’a-b’ (= a, a+1, a+2, ..., b-1, b),  
                          ’<a’  (= 0,1,...,a-2,a-1),  
                          ’>=a’ (= a, a+1, ... up to the maximum wave  
                                   number corresponding to the Nyquist  
                                   frequency for the case of FFT).  
        rejwn -- wave number(s) of sinusoid NOT to be used for fitting.  
                 can be set just as addwn but has higher priority:  
                 wave numbers which are specified both in addwn  
                 and rejwn will NOT be used.  
                 default: []  
        clipthresh -- clipping threshold for iterative fitting  
                 default: 3  
        clipniter -- maximum iteration number for iterative fitting  
                 default: 0 (no iteration, i.e., no clipping)  
 
verifycal -- interactively verify the results of calibration  
          See description of verify parameter in the task, sdcal,  
          for details.  
        options: (bool) True,False  
        default: False  
verifysm -- interactively verify the results of smoothing for each  
            spectrum.  
          See description of verify parameter in the task, sdaverage,  
          for details.  
        options: (bool) True,False  
        default: False  
        Note: verification is not yet available for kernel=’regrid’  
verifybl -- interactively verify the results of baseline fitting for  
            each spectrum.  
          See description of verify parameter in the task, sdbaseline,  
          for details.  
        options: (bool) True,False  
        default: False  
        NOTE: Currently available only when blfunc=’poly’  
verbosebl -- output fitting results to logger. If False, the fitting results  
             including coefficients, residual rms, etc., are not output to  
             the CASA logger, while the processing speed gets faster.  
        options: (bool) True, False  
        default: True  
bloutput -- output fitting results to a text file. if False, the fitting  
            results including coefficients, residual rms, etc., are not  
            output to a text file (<outfile>_blparam.txt), while  
            the processing speed gets faster.  
        options: (bool) True, False  
        default: True  
blformat -- format of the logger output and text file specified with bloutput  
        options: ’’, ’csv’  
        default: ’’ (same as in the past, easy to read but huge)  
showprogress -- show progress status for large data  
        options: (bool) True, False  
        default: True  
    >>> showprogress expandable parameter  
        minnrow -- minimum number of input spectra to show progress status  
                 default: 1000  
outfile -- name of output file  
        default: ’’ (<infile>_cal)  
outform -- output file format  
        options: ’ASAP’,’MS2’, ’ASCII’,’SDFITS’  
        default: ’ASAP’  
        NOTE the ASAP format is easiest for further sd  
        processing; use MS2 for CASA imaging.  
        If ASCII, then will append some stuff to  
        the outfile name  
overwrite -- overwrite the output file if already exists  
        options: (bool) True,False  
        default: False  
        NOTE this parameter is ignored when outform=’ASCII’  
plotlevel -- control for plotting and summary of results  
        options: (int) 0, 1, 2, and their negative counterparts  
default: 0 (no plotting)  
example: plotlevel=1; show plot of spectra (see description of  
                              the parameter in sdcal, sdaverage, and sdbaseline)  
                 plotlevel=2; additionally list data before and after operation.  
                 plotlevel<0 as abs(plotlevel), e.g.  
                 -1 => hardcopy of final plot at each step, i.e.,  
                 <outfile>_calspec.eps, <outfile>_smspec.eps, and/or  
                 <outfile>_bsspec.eps  
 
-------  
WARNING  
-------  
For the GBT raw SDFITS format data as input:  
SDtasks are able to handle GBT raw SDFITS format data since the  
data filler is available. However, the functionality is not well  
tested yet, so that there may be unknown bugs.  
 


More information about CASA may be found at the CASA web page

Copyright 2016 Associated Universities Inc., Washington, D.C.

This code is available under the terms of the GNU General Public Lincense


Home | Contact Us | Directories | Site Map | Help | Privacy Policy | Search