Previous Up Next

Appendix B  Python and CASA

CASA uses Python, IPython and matplotlib within the package. IPython is an enhanced, interactive shell to Python which provides many features for efficient command line interaction, while matplotlib is a Python 2-D plotting library for publication quality figures in different hardcopy formats.

From "Python is an interpreted, interactive, object-oriented programming language". Python is used as the underlying command line interface/scripting language to CASA. Thus, CASA inherits the features and the annoyances of Python. For example, since Python is inherently 0-based in its indexing of arrays, vectors, etc, CASA is also 0-based; any Index inputs (e.g., start (for start channel), fieldIndex, antennaID, etc) will start with 0. Another example is that indenting of lines means something to Python, of which users will have to be aware.

Currently, CASA uses python 2.6 (2.5 for the Mac OS 10.5 version). Some key links to python are:

Each of the features of these components behave in the standard way within CASA. In the following sections, we outline the key elements for analysis interactions; see the Python references and the IPython page for the full suite of functionality.

B.1  Python Packages

The following python packages are included in CASA: ipython, nose, pyfits, pytz, dbus, numpy, scientific python, twisted, zope.interface, foolscap, matplotlib, scipy. For their version numbers, please check the CASA page

B.2  Automatic parentheses

Automatic parenthesis is enabled for calling functions with argument lists; this feature is intended to allow less typing for common situations. IPython will display the interpretation of the line, beneath the one typed, as indicated by the ’-------->’. Default behavior in CASA is to have automatic parenthesis enabled.

B.3  Indentation

Python pays attention to indentation of lines in scripts or when you enter them interactively. It uses indentation to determine the level of nesting in loops. Be careful when cutting and pasting, if you get the wrong indentation, then unpredictable things can happen (usually it just gives an error).

A blank line can be used to return the indentation to a previous level. For example, expanded parameters in tasks cause indentation in subsequent lines in the interface. For example, the following snippet of inputs from clean can be cut and pasted without error due to the blank line after the indented parameters:

mode                =  'channel'        #   Type of selection
     nchan          =         -1        #   Number of channels to select 
     start          =          0        #   Start channel
     step           =          1        #   Increment between channels/velocity
     width          =          1        #   Channel width

alg                 =    'clark'        #   Algorithm to use

If the blank line were not there, an error would result if you pasted this at the casa prompt.

B.4  Lists and Ranges

Sometimes, you need to give a task a list of indices. For example, some tasks and tools expect a comma-separated Python list, e.g.

     scanlist = [241, 242, 243, 244, 245, 246]

You can use the Python range function to generate a list of consecutive numbers, e.g.

     scanlist = range(241,247)

giving the same list as above, e.g.

CASA <1>: scanlist=range(241,247)
CASA <2>: print scanlist
[241, 242, 243, 244, 245, 246]

Note that range starts from the first limit and goes to one below the second limit (Python is 0-based, and range is designed to work in loop functions). If only a single limit is given, the first limit is treated as 0, and the one given is used as the second, e.g.

CASA <3>: iflist=range(4)
CASA <4>: print iflist
[0, 1, 2, 3]

You can also combine multiple ranges by summing lists

CASA <5>: scanlist=range(241,247) + range(251,255)
CASA <6>: print scanlist
[241, 242, 243, 244, 245, 246, 251, 252, 253, 254]

B.5  Dictionaries

Python dictionaries are data structures that contain key:value pairs, sort of like a hash array. These are useful to store mini-databases of things. In CASA, the parameter values are kept in a dictionary behind the scenes.

To initialize a dictionary, say we call it mydict, for use:

CASA <7>: mydict = {}

To add members:

CASA <8>: mydict['source'] = '0137+331'
CASA <9>: mydict['flux'] = 5.4

To see its contents:

CASA <10>: mydict
  Out[10]: {'flux': 5.4000000000000004, 'source': '0137+331'}
CASA <11>: print mydict 
{'source': '0137+331', 'flux': 5.4000000000000004}

To access a specific entry:

CASA <12>: print mydict['flux']

B.5.1  Saving and Reading Dictionaries

To save a simple dictionary to a file:

CASA <13>: dictfile = open('','w')
CASA <14>: print >>dictfile,"mydict = ",mydict
CASA <15>: dictfile.close()
CASA <16>: !cat
IPython system call: cat
mydict =  {'source': '0137+331', 'flux': 5.4000000000000004}

CASA <17>: mydict = {}
CASA <18>: run
CASA <19>: mydict
  Out[19]: {'flux': 5.4000000000000004, 'source': '0137+331'}

More complex dictionaries, like those produced by imstat that contain NumPy arrays, require a different approach to save. The pickle module lets you save general data structures from Python. For example:

CASA <20>: import pickle 
CASA <21>: xstat
{'blc': array([0, 0, 0, 0]),
 'blcf': '15:24:08.404, +, I, 1.41281e+09Hz',
 'flux': array([ 4.0795296]),
 'max': array([ 0.05235516]),
 'maxpos': array([134, 134,   0,  38]),
 'maxposf': '15:21:53.976, +, I, 1.41374e+09Hz',
 'mean': array([  1.60097857e-05]),
 'medabsdevmed': array([ 0.00127436]),
 'median': array([ -1.17422514e-05]),
 'min': array([-0.0104834]),
 'minpos': array([160,   1,   0,  30]),
 'minposf': '15:21:27.899, +, I, 1.41354e+09Hz',
 'npts': array([ 3014656.]),
 'quartile': array([ 0.00254881]),
 'rms': array([ 0.00202226]),
 'sigma': array([ 0.0020222]),
 'sum': array([ 48.26399646]),
 'sumsq': array([ 12.32857318]),
 'trc': array([255, 255,   0,  45]),
 'trcf': '15:19:52.390, +, I, 1.41391e+09Hz'}

CASA <22>: mydict
  Out[22]: {'flux': 5.4000000000000004, 'source': '0137+331'}

CASA <23>: pickfile = 'myxstat.pickle'
CASA <24>: f = open(pickfile,'w')
CASA <25>: p = pickle.Pickler(f)
CASA <26>: p.dump(xstat)
CASA <27>: p.dump(mydict)
CASA <28>: f.close()

The dictionaries are now saved in pickle file myxstat.pickle in the current directory.

To retrieve:

CASA <29>: xstat2 = {}
CASA <30>: mydict2 = {}
CASA <31>: f = open(pickfile)
CASA <32>: u = pickle.Unpickler(f)
CASA <33>: xstat2 = u.load()
CASA <34>: mydict2 = u.load()
CASA <35>: f.close()
CASA <36>: xstat2
{'blc': array([0, 0, 0, 0]),
 'blcf': '15:24:08.404, +, I, 1.41281e+09Hz',
 'flux': array([ 4.0795296]),
 'max': array([ 0.05235516]),
 'maxpos': array([134, 134,   0,  38]),
 'maxposf': '15:21:53.976, +, I, 1.41374e+09Hz',
 'mean': array([  1.60097857e-05]),
 'medabsdevmed': array([ 0.00127436]),
 'median': array([ -1.17422514e-05]),
 'min': array([-0.0104834]),
 'minpos': array([160,   1,   0,  30]),
 'minposf': '15:21:27.899, +, I, 1.41354e+09Hz',
 'npts': array([ 3014656.]),
 'quartile': array([ 0.00254881]),
 'rms': array([ 0.00202226]),
 'sigma': array([ 0.0020222]),
 'sum': array([ 48.26399646]),
 'sumsq': array([ 12.32857318]),
 'trc': array([255, 255,   0,  45]),
 'trcf': '15:19:52.390, +, I, 1.41391e+09Hz'}

CASA <37>: mydict2
  Out[37]: {'flux': 5.4000000000000004, 'source': '0137+331'}

Thus, you can make scripts that save information and use it later, like for regressions.

Note that these examples use Python file-handling and IO, as well as importing modules such as pickle. See your friendly Python reference for more on this kind of stuff. It’s fairly obvious how it works.

B.6  Control Flow: Conditionals, Loops, and Exceptions

There are a number of ways to control the flow of execution in Python, including conditionals (if), loops (for and while), and exceptions (try). We will discuss the first two below.

B.6.1  Conditionals

The standard if block handles conditional execution or branches in Python:

   if <expression>:
   elif <expression>:
   elif <expression>:

Insert a pass statement if you want no action to be taken for a particular clause. The <expression> should reduce down to True or False.

For example,

if ( importmode == 'vla' ):
    # Import the data from VLA Export to MS
    print "Use importvla to read VLA Export and make an MS"
    archivefiles = datafile
    vis = msfile
    bandname = exportband
    autocorr = False
    antnamescheme = 'new'
    project = exportproject
elif ( importmode == 'fits' ):
    # Import the data from VLA Export to MS
    print "Use importuvfits to read UVFITS and make an MS"
    fitsfile = datafile
    vis = msfile
    # Copy from msfile
    print "Copying "+datafile+" to "+msfile
    os.system('cp -r '+datafile+' '+msfile)
    vis = msfile

chooses branches based on the value of the importmode Python variable (set previously in script).

B.6.2  Loops

The for loop

   for iter in seq:

iterates over elements of a sequence seq, assigning each in turn to iter. The sequence is usually a list of values.

For example,

splitms = ''
srclist = ['0137+331','2136+006','2202+422','2253+161','0319+415','0359+509']
spwlist = ['0','1']

for src in srclist:
    for spwid in spwlist:

        imname = splitms + '.' + src + '.' + spwid + '.clean'
    # Done with  spw

# Done with sources

As usual, blocks are closed by blank lines of the previous indentation level.

You can use the range (§ B.4) Python function to generate a numerical loop:

vis = ''
for i in range(0,6):
   fld = str(i)

# Done with fields [0, 1, 2, 3, 4, 5]

There is also a while loop construct

   while <expression>:

which executes the statement block while the <expression> is True. The while loop can also take an else block.

For example,

# Do an explicit set of clean iterations down to a limit
prevrms = 1.e10
while rms > 0.001 :
    if rms > prevrms:
         break                # the rms has increased, stop
    prevrms = rms

# Clean until the off-source rms residual, reaches 0.001 Jy

Note that you can exit a loop using the break statement, as we have here when the rms increases.

B.7  System shell access

For scripts, the os.system methods are the preferred way to access system shell commands (see § B.7.1).

In interactive mode, any input line beginning with a ’!’ character is passed verbatim (minus the ’!’) to the underlying operating system. Several common commands (ls, pwd, less) may be executed with or without the ’!’. Note that the cd command must be executed without the ’!’, and the cp command must use ’!’ as there is a conflict with the cp tool in casa.

For example:

  CASA [1]: pwd
  CASA [2]: ls n*
  CASA [3]: !cp -r ../ .

B.7.1  Using the os.system methods

To use this, you need the os package. This should be loaded by default by casa, but if not you can use

   import os

in your script.

For example, in our scripts we use this to clean up any existing output files

   # The prefix to use for all output files

   # Clean up old files
   os.system('rm -rf '+prefix+'*')

Note that the os package has many useful methods. You can see these by using tab-completion:

  CASA <2>: os.<tab>
os.EX_CANTCREAT            os._Environ                os.fdatasync               os.remove
os.EX_CONFIG               os.__all__                 os.fdopen                  os.removedirs
os.EX_DATAERR              os.__builtins__            os.fork                    os.rename
os.EX_IOERR                os.__class__               os.forkpty                 os.renames
os.EX_NOHOST               os.__delattr__             os.fpathconf               os.rmdir
os.EX_NOINPUT              os.__dict__                os.fstat                   os.sep
os.EX_NOPERM               os.__doc__                 os.fstatvfs                os.setegid
os.EX_NOUSER               os.__file__                os.fsync                   os.seteuid
os.EX_OK                   os.__getattribute__        os.ftruncate               os.setgid
os.EX_OSERR                os.__hash__                os.getcwd                  os.setgroups
os.EX_OSFILE               os.__init__                os.getcwdu                 os.setpgid
os.EX_PROTOCOL             os.__name__                os.getegid                 os.setpgrp
os.EX_SOFTWARE             os.__new__                 os.getenv                  os.setregid
os.EX_TEMPFAIL             os.__reduce__              os.geteuid                 os.setreuid
os.EX_UNAVAILABLE          os.__reduce_ex__           os.getgid                  os.setsid
os.EX_USAGE                os.__repr__                os.getgroups               os.setuid
os.F_OK                    os.__setattr__             os.getloadavg              os.spawnl
os.NGROUPS_MAX             os.__str__                 os.getlogin                os.spawnle
os.O_APPEND                os._copy_reg               os.getpgid                 os.spawnlp
os.O_CREAT                 os._execvpe                os.getpgrp                 os.spawnlpe
os.O_DIRECT                os._exists                 os.getpid                  os.spawnv
os.O_DIRECTORY             os._exit                   os.getppid                 os.spawnve
os.O_DSYNC                 os._get_exports_list       os.getsid                  os.spawnvp
os.O_EXCL                  os._make_stat_result       os.getuid                  os.spawnvpe
os.O_LARGEFILE             os._make_statvfs_result    os.isatty                  os.stat
os.O_NDELAY                os._pickle_stat_result     os.kill                    os.stat_float_times
os.O_NOCTTY                os._pickle_statvfs_result  os.killpg                  os.stat_result
os.O_NOFOLLOW              os._spawnvef               os.lchown                  os.statvfs
os.O_NONBLOCK              os.abort                   os.linesep                 os.statvfs_result
os.O_RDONLY                os.access                            os.strerror
os.O_RDWR                  os.altsep                  os.listdir                 os.symlink
os.O_RSYNC                 os.chdir                   os.lseek                   os.sys
os.O_SYNC                  os.chmod                   os.lstat                   os.sysconf
os.O_TRUNC                 os.chown                   os.major                   os.sysconf_names
os.O_WRONLY                os.chroot                  os.makedev                 os.system
os.P_NOWAIT                os.close                   os.makedirs                os.tcgetpgrp
os.P_NOWAITO               os.confstr                 os.minor                   os.tcsetpgrp
os.P_WAIT                  os.confstr_names           os.mkdir                   os.tempnam
os.R_OK                    os.ctermid                 os.mkfifo                  os.times
os.SEEK_CUR                os.curdir                  os.mknod                   os.tmpfile
os.SEEK_END                os.defpath                           os.tmpnam
os.SEEK_SET                os.devnull                 os.nice                    os.ttyname
os.TMP_MAX                 os.dup                               os.umask
os.UserDict                os.dup2                    os.openpty                 os.uname
os.WCONTINUED              os.environ                 os.pardir                  os.unlink
os.WCOREDUMP               os.errno                   os.path                    os.unsetenv
os.WEXITSTATUS             os.error                   os.pathconf                os.urandom
os.WIFCONTINUED            os.execl                   os.pathconf_names          os.utime
os.WIFEXITED               os.execle                  os.pathsep                 os.wait
os.WIFSIGNALED             os.execlp                  os.pipe                    os.wait3
os.WIFSTOPPED              os.execlpe                 os.popen                   os.wait4
os.WNOHANG                 os.execv                   os.popen2                  os.waitpid
os.WSTOPSIG                os.execve                  os.popen3                  os.walk
os.WTERMSIG                os.execvp                  os.popen4                  os.write
os.WUNTRACED               os.execvpe                 os.putenv                  
os.W_OK                    os.extsep                            
os.X_OK                    os.fchdir                  os.readlink                

B.7.2  Directory Navigation

In addition, filesystem navigation is aided through the use of bookmarks to simplify access to frequently-used directories:

  CASA [4]: cd /home/ballista/jmcmulli/other_data
  CASA [4]: pwd
  CASA [5]: bookmark other_data
  CASA [6]: cd /export/home/corsair-vml/jmcmulli/data
  CASA [7]: pwd
  CASA [8]: cd -b other_data
  (bookmark:data) -> /home/ballista/jmcmulli/other_data

For python scripts, there is a special command to change a directory.

os.system('cd ~/directory')

will NOT work but the following will:


B.7.3  Shell Command and Capture

See also § B.9 for the use of the command history.

1. sx shell_command, !!shell_command - this captures the output to a list

  CASA [1]: sx pwd # stores output of 'pwd' in a list
    Out[1]: ['/home/basho3/jmcmulli/pretest']

  CASA [2]: !!pwd  # !! is a shortcut for 'sx'
    Out[2]: ['/home/basho3/jmcmulli/pretest']

  CASA [3]: sx ls v* # stores output of 'pwd' in a list

  CASA [4]: x=_ # remember '_' is a shortcut for the output from the last command

  CASA [5]: x
   'vla_plotcal_bpass.png', 'vla_plotcal_fcal.jpg',

  CASA [6]: y=Out[2] # or just refer to the enumerated output

  CASA [7]: y
    Out[7]: ['/home/basho3/jmcmulli/pretest']

2. sc - captures the output to a variable; options are ’-l’ and ’-v’

  CASA [1]: sc x=pwd # capture output from 'pwd' to the variable 'x'

  CASA [2]: x
    Out[2]: '/home/basho3/jmcmulli/pretest'

  CASA [3]: sc -l x=pwd # capture the output from 'pwd' to the variable 'x' but
                        # split newlines into a list (similar to sx command)

  CASA [4]: x
    Out[4]: ['/home/basho3/jmcmulli/pretest']

  CASA [5]: sc -v x=pwd # capture output from 'pwd' to a variable 'x' and
                        # show what you get (verbose mode)
  x ==
  CASA [6]: x
    Out[6]: '/home/basho3/jmcmulli/pretest'

B.8  Logging

There are two components to logging within CASA. Logging of all command line inputs is done via IPython.

Upon startup, CASA will log all commands to a file called ipython.log. This file can be changed via the use of the  /.casa/ipython/ipythonrc file. This log file can be edited and re-executed as appropriate using the execfile feature (§ B.12).

Logging can be turned on and off using the logon, logoff commands.

The second component is the output from applications which is directed to the file ./casapy-YYYYMMDD-HHMMSS.log. See § 1.5.2 for more on the casalogger.

B.9  History and Searching

Numbered input/output history is provided natively within IPython. Command history is also maintained on-line.

  CASA [11]: x=1

  CASA [12]: y=3*x

  CASA [13]: z=x**2+y**2

  CASA [14]: x
    Out[14]: 1

  CASA [15]: y
    Out[15]: 3

  CASA [16]: z
    Out[16]: 10

  CASA [17]: Out[14]   # Note: The 'Out' vector contains command output
    Out[17]: 1

  CASA [18]: _15       # Note: The return value can be accessed by _number
    Out[18]: 3

  CASA [19]: ___       # Note: The last three return values can be accessed as:
    Out[19]: 10        #       _, __, ___

Command history can be accessed via the ’hist’ command. The history is reset at the beginning of every CASA session, that is, typing ’hist’ when you first start CASA will not provide any commands from the previous session. However, all of the commands are still available at the command line and can be accessed through the up or down arrow keys, and through searching.

  CASA [22]: hist
  1 : __IP.system("vi")  # Note:shell commands are designated in this way
  2 : ipmagic("run -i") # Note:magic commands are designated in this way
  3 : ipmagic("hist ")
  4 : more
  5 : __IP.system("more")
  6 : quickhelp()                # Note: autoparenthesis are added in the history
  7 :'')
  8 : im.summary()
  9 : ipmagic("pdoc im.setdata")
  10: im.close()
  11: quickhelp()
  12: ipmagic("logstate ")
  13: x=1
  14: y=3*x
  15: z=x**2+y**2
  16: x
  17: y
  18: z
  19: Out[16]
  20: _17
  21: ___

The history can be saved as a script or used as a macro for further use:

  CASA [24]: save 13:16
    File `` exists. Overwrite (y/[N])? y
    The following commands were written to file ``:
  CASA [25]: !more

Note that the history commands will be saved up to, but not including the last value (i.e., history commands 13-16 saves commands 13, 14, and 15).

There are two mechanisms for searching command history:

  1. Previous/Next: use Ctrl-p (previous,up) and Ctrl-n (next,down) to search through only the history items that match what you have typed so far (min-match completion). If you use Ctrl-p or Ctrl-n at a blank prompt, they behave just like the normal arrow keys.
  2. Search: Ctrl-r opens a search prompt. Begin typing and the system searches your history for lines that contain what you’ve typed so far, completing what it can. For example:
      CASA [37]: <CTRL-r>
    Typing anything after the colon will provide you with the last command matching the characters, for example, typing ’op’ finds:
    Subsequent hitting of Ctrl-r will search for the next command matching the characters.

B.10  Macros

Macros can be made for easy re-execution of previous commands. For example to store the commands 13-15 to the macro ’example’:

  CASA [31]: macro example 13:16
    Macro `example` created. To execute, type its name (without quotes).
    Macro contents:

  CASA [32]: z
    Out[32]: 6

  CASA [33]: z=10

  CASA [34]: example
    Out[34]: Executing Macro...

  CASA [35]: z
    Out[35]: 6

  CASA [36]:

B.11  On-line editing

You can edit files on-line in two ways:

  1. Using the shell access via ’!vi’
  2. Using the ed function; this will edit the file but upon closing, it will try to execute the file; using the ’’ example above:
      CASA [13]: ed # this will bring up the file in your chosen editor
                              # when you are finished editing the file, 
                              # it will automatically
                              # execute it (as though you had done a 
                              # execfile ''
        Editing... done. Executing edited code...
      CASA [14]: x
        Out[14]: 1
      CASA [15]: y
        Out[15]: 3
      CASA [16]: z
        Out[16]: 6

B.12  Executing Python scripts

Python scripts are simple text files containing lists of commands as if typed at the keyboard. Note: the auto-parentheses feature of IPython cannot be used in scripts, that is, you should make sure all function calls have any opening and closing parentheses.

  # file is
  # My script to plot the observed visibilities
  plotxy('','uvdist') #yaxis defaults to amplitude

This can be done by using the execfile command to execute this script. execfile will execute the script as though you had typed the lines at the CASA prompt.

  CASA [5]: execfile ''
  --------> execfile('')

If you don’t want to launch CASA and execute your script from the command line, you can use the ’-c’ option:

unix$   casa -c ''

B.13  How do I exit from CASA?

You can exit CASA by using the quit command. This will bring up the query

   Do you really want to exit ([y]/n)?

to give you a chance in case you did not mean to exit. You can also quit using %exit or CTRL-D.

If you don’t want to see the question "Do you really want to exit [y]/n?", then just type Exit or exit followed by return, and CASA will stop right then and there.

Previous Up Next