casa  $Rev:20696$
 All Classes Namespaces Files Functions Variables
Variables
ngc4826_tutorial_regression Namespace Reference

Variables

 scriptmode
 benchmarking
string scriptprefix
string prefix
 Clear out previous run results.
string msfile
tuple startTime
 N4826 - BIMA SONG Data 16apr98 source=ngc4826 phasecal=1310+323 fluxcal=3c273, Flux = 23 Jy on 16apr98 passcal= none - data were observed with online bandpass correction.
tuple startProc
tuple _mydatapath
tuple import2time
 Import and concatenate sources.
string concatvis
string freqtol
tuple concat2time
string vis
 TUTORIAL NOTES:
tuple spwid
tuple clearcal2time
 16 APR Calibration
tuple list2time
 List contents of MS.
tuple plotxy2time
 Plotting and Flagging.
string spw
 Flag end channels.
string timerange
string comment
 Some example clean-up editing Slightly high almost-edge channel in field='1', spw='4' (channel 2) can be flagged interactively with plotxy.
tuple flag2time
tuple setjy2time
 CALIBRATION.
string field
 Gain calibration.
string refant
string solint
tuple gaincal2time
string fluxtable
 Transfer the flux density scale:
string reference
tuple fluxscale2time
tuple user_check
 Plot calibration.
string showgui
tuple plotcal2time
float gaincurve
string gaintable
list spwmap
tuple correct2time
string outputvis
 Check calibrated data print '--Plotxy--' default(plotxy)
string datacolumn
tuple split2time
int clnsize
 print '--Clearcal (split data)--'
string imagename
list cell
string stokes
string mode
 Manually correct for mosaic response pattern using .image/.flux images.
string interpolation
string psfmode
string scaletype
int cyclefactor
 As we moved to clean by default in flat sigma rather than flat snr it converges less well cyclefactor=1.5,niter=10000,threshold='45mJy',.
float minpb
tuple clean2time
 NOTE: mosaic data ...Sault weighting implies a noise unform image.
tuple srcstat
 Statistics on clean image cube.
tuple refpix
tuple refbox
int blcx
int blcy
int trcx
int trcy
tuple offbox
tuple offstat
tuple cenbox
tuple offlinestat
tuple modstat
 Statistics on clean model.
string expr
tuple pbcorstat
 Statistics on PBcor image cube.
tuple pbcoroffstat
tuple pbcorofflinestat
tuple psfstat
tuple math2time
tuple pbcormodstat
string momfile
 Do zeroth and first moments.
string momzeroimage
string momoneimage
list moments
string chans
string outfile
 Now print out results The following writes a logfile for posterity.
tuple moments2time
tuple momzerostat
 Statistics on moment images.
tuple momonestat
tuple csys
float vel0
float vel35
tuple momoneplane0
tuple momoneplane35
tuple vismean_cal
 Get MS stats.
tuple vismean_src
tuple endProc
 An alternative is to mask the pbcor image before calculating moments.
tuple endTime
string testdate
 Previous results to be used for regression.
string testvers
float clean_image_max
 

RR, 3/11/2011 - The rmses went up, but not to their historical maxima.


float clean_offsrc_rms
float clean_offline_rms
float clean_momentzero_max
float clean_momentzero_rms
float clean_momentone_median
float clean_momentone_planezero
float clean_momentone_planelast
float vis_mean_cal
float vis_mean_src
float model_sum
float model_pbcor_sum
dictionary canonical
dictionary canonical_results
dictionary regression
string regressfile
dictionary prev_results
tuple fr
tuple u
list srcmax
 Calculate regression values.
list offrms
list offlinerms
list momzero_max
list momzero_rms
list momone_median
list momone_plane0
list momone_plane35
list psfcenter
list modflux
list pbcormodflux
dictionary new_regression
tuple datestring
tuple myvers
tuple myuser
tuple myuname
list myhost
list myos
tuple mycwd
tuple mypath
string mydataset
dictionary results
string op
float tol
list resultlist
list res
 prev
list new_val
list prev_val
 new_diff
string new_status
float datasize_raw
float datasize_ms
dictionary total
int nstages
dictionary stages
string pickfile
 Save regression results as dictionary using Pickle.
tuple f
tuple p
tuple logfile
string final_status
 regstate

Variable Documentation

Definition at line 113 of file ngc4826_tutorial_regression.py.

Definition at line 67 of file ngc4826_tutorial_regression.py.

Definition at line 932 of file ngc4826_tutorial_regression.py.

Definition at line 933 of file ngc4826_tutorial_regression.py.

Definition at line 1463 of file ngc4826_tutorial_regression.py.

Definition at line 1473 of file ngc4826_tutorial_regression.py.

Definition at line 854 of file ngc4826_tutorial_regression.py.

Definition at line 952 of file ngc4826_tutorial_regression.py.

Definition at line 1060 of file ngc4826_tutorial_regression.py.

NOTE: mosaic data ...Sault weighting implies a noise unform image.

Using ftmachine='mosaic', can also use ftmachine='ft' for more traditional image plane mosaicing NOTE: that niter is set to large number so that stopping point is controlled by threshold. NOTE: with pbcor=False, the final image is not "flux correct", instead the image has constant noise despite roll off in power as you move out from the phase center(s). Though this format makes it "look nicest", for all flux density measurements, and to get an accurate integrated intensity image, one needs to divide the srcimage.image/srcimage.flux in order to correct for the mosaic response pattern. One could also achieve this by setting pbcor=True in clean.

Definition at line 890 of file ngc4826_tutorial_regression.py.

RR, 3/11/2011 - The rmses went up, but not to their historical maxima.

The

model_pbcor_sum went down, but not to its historical minimum. Nobody seems

to know why.

clean_offsrc_rms = 0.0535 clean_offline_rms = 0.0563 model_pbcor_sum = 62.5022

RR, 3/12/2011 - And now, mysteriously, they're back to their 1/19/2011 values.

RR, 3/25 - 4/3/2011, after clean was changed to used the center of output channel frequencies, instead of center of the first input channel in each output channel.

Definition at line 1418 of file ngc4826_tutorial_regression.py.

Definition at line 1426 of file ngc4826_tutorial_regression.py.

Definition at line 1428 of file ngc4826_tutorial_regression.py.

Definition at line 1427 of file ngc4826_tutorial_regression.py.

Definition at line 1421 of file ngc4826_tutorial_regression.py.

Definition at line 1422 of file ngc4826_tutorial_regression.py.

Definition at line 1420 of file ngc4826_tutorial_regression.py.

Definition at line 1419 of file ngc4826_tutorial_regression.py.

16 APR Calibration

Definition at line 284 of file ngc4826_tutorial_regression.py.

print '--Clearcal (split data)--'

You might image the calibrater data:

print '--Clean (1310+323)--' default('clean')

clean(vis='ngc4826.tutorial.16apr98.1310+323.split.ms', imagename='ngc4826.tutorial.16apr98.cal.clean', cell=[1.,1.],imsize=[256,256], field='0',spw='0~7',threshold=10., mode='mfs',psfmode='clark',niter=100,stokes='I')

IMAGING OF NGC4826 MOSAIC

     Mosaic field spacing looks like:

     F3 (field 3)         F2 (field 2)

F4 (field 4) F0 (field 0) F1 (field 1)

   F5 (field 5)         F6 (field 6)

4x64 channels = 256 channels

Primary Beam should be about 1.6' FWHM (7m dishes, 2.7mm wavelength) Resolution should be about 5-8"

Image the target source mosaic:

Definition at line 848 of file ngc4826_tutorial_regression.py.

Some example clean-up editing Slightly high almost-edge channel in field='1', spw='4' (channel 2) can be flagged interactively with plotxy.

plotxy(vis='ngc4826.tutorial.ms', xaxis='channel',yaxis='amp',field='1',spw='4', averagemode='vector',timebin='1e7',crossscans=True, selectplot=True,newplot=False,title='Field 1 SPW 4')

Use Flagmanager to save a copy of the flags so far

Definition at line 526 of file ngc4826_tutorial_regression.py.

Definition at line 188 of file ngc4826_tutorial_regression.py.

Definition at line 184 of file ngc4826_tutorial_regression.py.

Definition at line 667 of file ngc4826_tutorial_regression.py.

Definition at line 1141 of file ngc4826_tutorial_regression.py.

As we moved to clean by default in flat sigma rather than flat snr it converges less well cyclefactor=1.5,niter=10000,threshold='45mJy',.

Definition at line 863 of file ngc4826_tutorial_regression.py.

Definition at line 775 of file ngc4826_tutorial_regression.py.

Definition at line 1782 of file ngc4826_tutorial_regression.py.

Definition at line 1781 of file ngc4826_tutorial_regression.py.

Definition at line 1611 of file ngc4826_tutorial_regression.py.

An alternative is to mask the pbcor image before calculating moments.

The following block shows how to do this.

Definition at line 1251 of file ngc4826_tutorial_regression.py.

Definition at line 1252 of file ngc4826_tutorial_regression.py.

Definition at line 983 of file ngc4826_tutorial_regression.py.

Definition at line 1830 of file ngc4826_tutorial_regression.py.

Gain calibration.

Correct the calibrater/target source data: Use new parm spwmap to apply gain solutions derived from spwid1 to all other spwids...

Definition at line 583 of file ngc4826_tutorial_regression.py.

Definition at line 1923 of file ngc4826_tutorial_regression.py.

Definition at line 533 of file ngc4826_tutorial_regression.py.

Definition at line 609 of file ngc4826_tutorial_regression.py.

Transfer the flux density scale:

Definition at line 603 of file ngc4826_tutorial_regression.py.

Definition at line 1519 of file ngc4826_tutorial_regression.py.

Definition at line 185 of file ngc4826_tutorial_regression.py.

Definition at line 588 of file ngc4826_tutorial_regression.py.

Definition at line 662 of file ngc4826_tutorial_regression.py.

Definition at line 663 of file ngc4826_tutorial_regression.py.

Definition at line 852 of file ngc4826_tutorial_regression.py.

Import and concatenate sources.

USB spectral windows written separately by miriad for 16apr98 Assumes these are in sub-directory called "fitsfiles" of working directory

Definition at line 160 of file ngc4826_tutorial_regression.py.

Definition at line 857 of file ngc4826_tutorial_regression.py.

List contents of MS.

Definition at line 306 of file ngc4826_tutorial_regression.py.

Definition at line 1858 of file ngc4826_tutorial_regression.py.

Definition at line 1025 of file ngc4826_tutorial_regression.py.

Definition at line 864 of file ngc4826_tutorial_regression.py.

Manually correct for mosaic response pattern using .image/.flux images.

Definition at line 856 of file ngc4826_tutorial_regression.py.

Definition at line 1432 of file ngc4826_tutorial_regression.py.

Definition at line 1431 of file ngc4826_tutorial_regression.py.

Definition at line 1595 of file ngc4826_tutorial_regression.py.

Statistics on clean model.

Definition at line 970 of file ngc4826_tutorial_regression.py.

Definition at line 1059 of file ngc4826_tutorial_regression.py.

Definition at line 1114 of file ngc4826_tutorial_regression.py.

Do zeroth and first moments.

NGC4826 LSR velocity is 408 km/s; delta is 20 km/s

Definition at line 1049 of file ngc4826_tutorial_regression.py.

Definition at line 1569 of file ngc4826_tutorial_regression.py.

Definition at line 1577 of file ngc4826_tutorial_regression.py.

Definition at line 1582 of file ngc4826_tutorial_regression.py.

Definition at line 1051 of file ngc4826_tutorial_regression.py.

Definition at line 1146 of file ngc4826_tutorial_regression.py.

Definition at line 1153 of file ngc4826_tutorial_regression.py.

Definition at line 1131 of file ngc4826_tutorial_regression.py.

Definition at line 1559 of file ngc4826_tutorial_regression.py.

Definition at line 1564 of file ngc4826_tutorial_regression.py.

Definition at line 1050 of file ngc4826_tutorial_regression.py.

Statistics on moment images.

Definition at line 1123 of file ngc4826_tutorial_regression.py.

Definition at line 80 of file ngc4826_tutorial_regression.py.

Definition at line 1623 of file ngc4826_tutorial_regression.py.

Definition at line 1626 of file ngc4826_tutorial_regression.py.

Definition at line 1621 of file ngc4826_tutorial_regression.py.

Definition at line 1622 of file ngc4826_tutorial_regression.py.

Definition at line 1624 of file ngc4826_tutorial_regression.py.

Definition at line 1620 of file ngc4826_tutorial_regression.py.

Definition at line 1616 of file ngc4826_tutorial_regression.py.

Definition at line 1614 of file ngc4826_tutorial_regression.py.

Definition at line 1758 of file ngc4826_tutorial_regression.py.

Definition at line 1607 of file ngc4826_tutorial_regression.py.

Definition at line 1763 of file ngc4826_tutorial_regression.py.

Definition at line 1756 of file ngc4826_tutorial_regression.py.

Definition at line 1802 of file ngc4826_tutorial_regression.py.

Definition at line 936 of file ngc4826_tutorial_regression.py.

Definition at line 1554 of file ngc4826_tutorial_regression.py.

Definition at line 957 of file ngc4826_tutorial_regression.py.

Definition at line 1549 of file ngc4826_tutorial_regression.py.

Definition at line 939 of file ngc4826_tutorial_regression.py.

Definition at line 1647 of file ngc4826_tutorial_regression.py.

Now print out results The following writes a logfile for posterity.

outfile='n4826.'+datestring+'.log'

Definition at line 1061 of file ngc4826_tutorial_regression.py.

Check calibrated data print '--Plotxy--' default(plotxy)

Pause script if you are running in scriptmode user_check=raw_input('Return to continue script
')

SPLIT THE DATA INTO SINGLE-SOURCE MS AND THEN IMAGE THE CALIBRATOR

Split out calibrated target source and calibrater data:

Definition at line 761 of file ngc4826_tutorial_regression.py.

Definition at line 1831 of file ngc4826_tutorial_regression.py.

Definition at line 1600 of file ngc4826_tutorial_regression.py.

Definition at line 1032 of file ngc4826_tutorial_regression.py.

Definition at line 1008 of file ngc4826_tutorial_regression.py.

Definition at line 1003 of file ngc4826_tutorial_regression.py.

Statistics on PBcor image cube.

Definition at line 999 of file ngc4826_tutorial_regression.py.

Save regression results as dictionary using Pickle.

Definition at line 1829 of file ngc4826_tutorial_regression.py.

Definition at line 647 of file ngc4826_tutorial_regression.py.

Plotting and Flagging.

The plotxy task is the interactive x-y display and flagging GUI

print '--Plotxy--' default(plotxy)

Definition at line 465 of file ngc4826_tutorial_regression.py.

Clear out previous run results.

Definition at line 79 of file ngc4826_tutorial_regression.py.

Definition at line 1746 of file ngc4826_tutorial_regression.py.

Definition at line 1516 of file ngc4826_tutorial_regression.py.

Definition at line 1757 of file ngc4826_tutorial_regression.py.

Definition at line 1587 of file ngc4826_tutorial_regression.py.

Definition at line 858 of file ngc4826_tutorial_regression.py.

Definition at line 1019 of file ngc4826_tutorial_regression.py.

Definition at line 584 of file ngc4826_tutorial_regression.py.

Definition at line 925 of file ngc4826_tutorial_regression.py.

Definition at line 604 of file ngc4826_tutorial_regression.py.

Definition at line 924 of file ngc4826_tutorial_regression.py.

Definition at line 1515 of file ngc4826_tutorial_regression.py.

Definition at line 1514 of file ngc4826_tutorial_regression.py.

Definition at line 1932 of file ngc4826_tutorial_regression.py.

Definition at line 1745 of file ngc4826_tutorial_regression.py.

Definition at line 1739 of file ngc4826_tutorial_regression.py.

Definition at line 1645 of file ngc4826_tutorial_regression.py.

Definition at line 859 of file ngc4826_tutorial_regression.py.

Definition at line 64 of file ngc4826_tutorial_regression.py.

Definition at line 70 of file ngc4826_tutorial_regression.py.

CALIBRATION.

Bandpasses are very flat because of observing mode used (online bandpass correction) so bandpass calibration is unnecessary for these data.

Derive gain calibration solutions. We will use VLA-like G (per-scan) calibration:

Set the flux density of 3C273 to 23 Jy

Definition at line 565 of file ngc4826_tutorial_regression.py.

Definition at line 640 of file ngc4826_tutorial_regression.py.

Definition at line 585 of file ngc4826_tutorial_regression.py.

Definition at line 778 of file ngc4826_tutorial_regression.py.

Flag end channels.

Definition at line 481 of file ngc4826_tutorial_regression.py.

Definition at line 262 of file ngc4826_tutorial_regression.py.

Definition at line 664 of file ngc4826_tutorial_regression.py.

Calculate regression values.

Definition at line 1544 of file ngc4826_tutorial_regression.py.

Statistics on clean image cube.

Definition at line 916 of file ngc4826_tutorial_regression.py.

Definition at line 1805 of file ngc4826_tutorial_regression.py.

Definition at line 111 of file ngc4826_tutorial_regression.py.

N4826 - BIMA SONG Data 16apr98 source=ngc4826 phasecal=1310+323 fluxcal=3c273, Flux = 23 Jy on 16apr98 passcal= none - data were observed with online bandpass correction.

NOTE: This data has been filled into MIRIAD, line-length correction done, and then exported as separate files for each source. 3c273 was not line length corrected since it was observed for such a short amount of time that it did not need it.

From miriad: source Vlsr = 408; delta V is 20 km/s

Definition at line 110 of file ngc4826_tutorial_regression.py.

Definition at line 855 of file ngc4826_tutorial_regression.py.

Previous results to be used for regression.

New values STM 2009-12-02 Release 0 (prerelease version)

for 400x400 clean

new values for flat noise clean

testdate = '2009-12-02 (STM)' testvers = 'CASA Version 3.0.1 Rev 10130' clean_image_max = 1.465047 clean_offsrc_rms = 0.058497 clean_offline_rms = 0.055416 clean_momentzero_max = 163.726852 clean_momentzero_rms = 15.206372 clean_momentone_median = 428.326385 clean_momentone_planezero = 696.702393 clean_momentone_planelast = 127.786629 vis_mean_cal = 194.915085 vis_mean_src = 54.627020 model_sum = 71.171693 model_pbcor_sum = 61.853749 #New values RR 2010-03-30 3.0.1 prerelease

for 400x400 clean

new values after start channel change. I did not update passing values.

testdate = '2010-03-30 (RR)' testvers = 'CASA Version 3.0.1 (build #10841)' clean_image_max = 1.615747 # was 1.465047 Peak hits a channel better? clean_offsrc_rms = 0.058497 clean_offline_rms = 0.055416 clean_momentzero_max = 163.726852 clean_momentzero_rms = 15.206372 clean_momentone_median = 429.658844 # was 428.326385; change << 1 chanwidth. clean_momentone_planezero = 696.702393 clean_momentone_planelast = 127.786629 vis_mean_cal = 194.915085 vis_mean_src = 54.627020 model_sum = 71.171693 model_pbcor_sum = 66.882499 # was 61.853749 Peak hits a channel better? # slight change in regression values - reason not known yet.

Remy's changes to cleanhelper, or something from Sanjay or Kumar?

testdate = '2010-04-24 (RR)' testvers = 'CASA Version 3.0.2 (build #11181)' clean_image_max = 1.615747 clean_offsrc_rms = 0.058497 clean_offline_rms = 0.055416 clean_momentzero_max = 163.726852 clean_momentzero_rms = 15.206372 clean_momentone_median = 423.6954 # was 429.6588; change << 1 chanwidth. clean_momentone_planezero = 696.702393 clean_momentone_planelast = 127.786629 vis_mean_cal = 194.915085 vis_mean_src = 54.627020 model_sum = 71.171693 model_pbcor_sum = 75.92 # was 66.88 Peak hits a channel better?

Definition at line 1416 of file ngc4826_tutorial_regression.py.

Definition at line 1417 of file ngc4826_tutorial_regression.py.

Definition at line 492 of file ngc4826_tutorial_regression.py.

Definition at line 1648 of file ngc4826_tutorial_regression.py.

Definition at line 1794 of file ngc4826_tutorial_regression.py.

Definition at line 934 of file ngc4826_tutorial_regression.py.

Definition at line 935 of file ngc4826_tutorial_regression.py.

Definition at line 1524 of file ngc4826_tutorial_regression.py.

Plot calibration.

Do interactive viewing of clean image.

Definition at line 626 of file ngc4826_tutorial_regression.py.

Definition at line 1142 of file ngc4826_tutorial_regression.py.

Definition at line 1143 of file ngc4826_tutorial_regression.py.

TUTORIAL NOTES:

You can invoke tasks in two ways:

(1) As function calls with arguments as shown above for concat and used extensively in this script, e.g.

task( par1=val1, par2=val2, ... )

with parameters set as arguments in the call. Note that in this case, the global parameter values are NOT used or changed, and any task parameters that are not specified as arguments to the call will be defaulted to the task-specific default values (see the "help task" description).

(2) By setting the values of the global parameters and then using the "go" command (if taskname is set) or calling the task with no arguments. For example:

default task par1 = val1 par2 = val2 ... inp task()

In this case, the "default" command sets the parmeters to their task defaults, and sets the "taskname" paramter to the task to be run. The "inp" command displays the current values for the task parameters. Then the call with no arguments runs with the globals.

Warning: "go" does not work inside scripts. See Cookbook.

Using the concat call above as an example, we would do:

default('concat')

vis = ['ngc4826.tutorial.3c273.5.ms', 'ngc4826.tutorial.3c273.6.ms', 'ngc4826.tutorial.3c273.7.ms', 'ngc4826.tutorial.3c273.8.ms', 'ngc4826.tutorial.1310+323.ll.9.ms', 'ngc4826.tutorial.1310+323.ll.10.ms', 'ngc4826.tutorial.1310+323.ll.11.ms', 'ngc4826.tutorial.1310+323.ll.12.ms', 'ngc4826.tutorial.1310+323.ll.13.ms', 'ngc4826.tutorial.1310+323.ll.14.ms', 'ngc4826.tutorial.1310+323.ll.15.ms', 'ngc4826.tutorial.1310+323.ll.16.ms', 'ngc4826.tutorial.ngc4826.ll.5.ms', 'ngc4826.tutorial.ngc4826.ll.6.ms', 'ngc4826.tutorial.ngc4826.ll.7.ms', 'ngc4826.tutorial.ngc4826.ll.8.ms']

concatvis='ngc4826.tutorial.ms' freqtol = "" dirtol = "1arcsec" async=False

concat()

Fix up the MS (temporary, changes to importfits underway)

Definition at line 260 of file ngc4826_tutorial_regression.py.

Definition at line 1429 of file ngc4826_tutorial_regression.py.

Definition at line 1430 of file ngc4826_tutorial_regression.py.

Get MS stats.

Definition at line 1170 of file ngc4826_tutorial_regression.py.

Definition at line 1173 of file ngc4826_tutorial_regression.py.