Getting Started Documentation Glish Learn More Programming Contact Us
Version 1.9 Build 1488
News FAQ
Search Home

next up previous contents
Next: 5. Reducing JCMT data in DISH Up: Volume 3 - Telescope Specific Processing Previous: 3. ATCA reduction

Subsections


4. Parkes Multibeam Reduction

Stacy Mader

4.1 Introduction

This chapter describes how to use AIPS++ in reducing Multibeam data from the ATNF's Parkes Radiotelescope. Data reduction is controlled via the graphical user interfaces (GUIs) Livedata, Gridzilla and Cubecat.

4.2 Configuring AIPS++

The source code for AIPS++ is available (at the ATNF) in the directory /nfs/aips++/stable. To setup paths for glish libraries, etc., perform the following: 

% source /nfs/aips++/stable/aipsinit.csh (for csh/tcsh shells)
% . /nfs/aips++/stable/aipsinit.sh       (for korn/bash shells)

4.3 Livedata

Livedata is the on-line and off-line processing software for data taken with the Parkes multibeam receiver, though has the capability of processing data from other telescopes. The principal task of Livedata is to remove the bandpass which is the dominant component of every raw spectrum. Livedata also calibrates the spectra, applies doppler tracking, smoothes and baselines the spectra, and has an excellent capability for visualising calibrated spectra. To startup Livedata, type: 

% livedata   ..or..
% glish -l livedata.g

Figure 4.1: The Livedata GUI with all clients enabled.
\begin{figure} \begin{center} \epsfig{file=livedatagui.ps,width=4.3in}\par\end{center}\end{figure}

Livedata directs and regulates the data flow between the following seven clients: 

Scheduler
       -> Reader
              -> Bandpass calibrator
                                  -> Monitor
                                  -> Statistics
                                  -> Writer
                                             -> Gridder

The flow of data is from left to right. All clients other than the reader may be disabled or short-circuited, i.e. any line(s) of the above may be bypassed other than the first. When selected, each client has a control panel on the Livedata GUI. By default, only the reader client is enabled at startup. A detailed description of each client and input arguements (via the GUI) are made below.

4.3.1 Scheduler

Figure 4.2 shows the Scheduler, which provides for realtime (live) and off-line data reduction. Realtime reduction differs from off-line reduction in that files newly created by the correlator may be discovered automatically (auto-queued) and that several attempts are made to read files which may be incomplete.

Figure 4.2: The Scheduler panel.
\begin{figure} \begin{center} \epsfig{file=livedata.scheduler.ps,width=5in}\end{center}\end{figure}

4.3.2 Multibeam Reader

This client reads data from sdfits (usually .sdfits) or rpfits (usually .rpf .mbf or .hpf) files or an AIPS++ Measurement Set (.ms2cal). The input data, which may be selected on a beam-by-beam basis, is packaged as a Glish record for the next client in the chain. The Reader client is enabled by default whereas the others (see below) are disabled.

Figure 4.3: The Reader panel.
\begin{figure} \begin{center} \epsfig{file=livedata.reader.ps,width=2.5in}\end{center}\end{figure}

Figure 4.4: The Reader calibration panel.
\begin{figure} \begin{center} \epsfig{file=livedata.reader.cal.ps,width=1.5in}\end{center}\end{figure}

4.3.3 Bandpass Calibration

This client does most of the work in calibrating Multibeam data. It operates in several modes depending on the method of observation:

Each of these modes uses a separate bandpass calibration strategy. These are based on robust statistical estimators, particularly median estimators, which allow rejection of RFI without human intervention.

Regarding the number of pre- and postcycles, the buffer must be large enough to contain the current scan, the previous and following scans, plus the first integration of the scan after the following scan. For example, if scan 2 is being processed, the buffer must contain all of scans 1, 2, and 3 plus the first integration of 4, the presence of which told us that scan 3 had finished. The reference spectrum for 2 is computed once when the first integration of scan 4 is read, and the first integration of scan 2 is then written out. This space is then available for the second integration of scan 4; each successive integration of scan 4 may overwrite the integration of scan 2 which has just been sent out. In cases where the buffer size (precycles + postcycles + 1) is insufficient (usually in the MX, or beam-switching mode), a warning is issued stating both pre- and postcycles should be increased by at least the present value of (precycles + postcycles + 1) plus an overrun value which is stated in the log message. Generally, the pre- and postcycle values can be calculated by: 3*nint, where nint is the number of integrations specified during observering. By not incrementing the values of the pre- and postcycle, the signal to noise ratio of the reference spectrum becomes degraded.

For the HIPASS, HVC and ZOA options, the pre- and postcycle values are set to 24, which implies a maximum of (24+24+1=49) scans are held in the buffer.

Figure 4.5: The Bandpass Calibration panel.
\begin{figure} \begin{center} \epsfig{file=livedata.bandpass.ps,width=4in}\end{center}\end{figure}

Figure 4.6: The Data Monitor panel.
\begin{figure} \begin{center} \epsfig{file=livedata.monitor.ps,width=3in}\end{center}\end{figure}

4.3.4 Data Monitor

The data monitor interfaces to MultibeamView, a specially modified version of the Karma package ``kview''. In fact, it invokes MultibeamView twice, once for each polarization, to provide two panels displaying frequency versus time with various image enhancement options. This provides visual inspection for each pair of polarizations of each beam, one pair at a time as data arrives. An example ("waterfall") panel is shown in Figure 4.7. It is anticipated that MultibeamVIew will be replace by a native AIPS++ imaging tool in the near future.

Changing the MultibeamView displays:

Obtaining 1-D Profiles within MultibeamView:

To obtain a 1-dimensional spectral profile:

The profile changes according to the position of mouse cursor on the XZ (Frequency-Time) image. To Zoom a profile:

4.3.5 Statistics

The statistics client reads through an AIPS++ Measurement Set and computes basic statistical measures for each beam and polarization.

4.3.6 Writer

Data is written to an AIPS++ Measurement Set by the writer client. These measurementsets are converted to either SDFITS or MS2 format.

4.4 Gridder

Each Multibeam integration consists of a spectrum taken at a particular point on the sky. The gridder (gridzilla) takes the spectra from a collection of bandpass-calibrated measurementsets and writes a 3-plane cube with the total number of spectra or scans per pixel in the first plane, the number of these occurring at nighttime in the second plane, and the number during daytime the third plane.

Figure 4.7: A MultibeamView panel displaying frequency vs time for beam 9, Polarization A. A faint galaxy can be seen near 1415 MHz at 08h46m.
\begin{figure} \begin{center} \epsfig{file=MBView.ps,width=4.5in}\end{center}\end{figure}

Figure 4.8: The Statistics window displaying Tsys for 7 beams.
\begin{figure} \begin{center} \epsfig{file=livedata.stats.ps,width=4.5in}\end{center}\end{figure}

Figure 4.9: The default Gridzilla GUI.
\begin{figure} \begin{center} \epsfig{file=gridzilla.gui.ps,width=4.7in}\end{center}\end{figure}

Typically each point in the sky is sampled many times in separate observations and the gridder combines these using robust statistics to automatically eliminate RFI. Being the most computationally intensive client, the gridder is usually used separately for off-line data reduction, but by pressing the Gridder button on the Livedata GUI, SDFITS files are transferred straight to the gridder. In this section, we assume reduction is off-line. To start up gridzilla: 

% glish -l gridzillarc.g

Both the gridzilla (Figure 4.9) and logger windows appear. The GUI contains several panels and entry widgets, each which is described below.

4.5 Creating HIPASS cubes: an example.

Although each standard HIPASS, ZOA and HVC cube is created by respective team members, those who wish to create new cubes (using different livedata processing options) can do so by reading on.

During the data taking process at Parkes, each scan is archived onto CDROMs as a raw (HPF) and processed (SDFITS) file. For each CDROM, there is a size file which lists files archived onto it. In order to identify which scans are located on which CDROM(s), a 'coverage' script is available which allows you to pass parameters enabling you to list the required CDROMs to extract scans from. The HIPASS script is 'coverage.pl' and for the HVC and ZOA surveys, they are 'coverage-hvc.pl' and 'coverage-zoa.pl' respectively. For ATNF sites, these scripts are located in /nfs/atapplic/multibeam/bin and each requires the $MB_CATALOG_PATH environment variable to be set where the size files are located: 

For csh/tcsh shells:
% setenv MB_CATALOG_PATH '/nfs/atapplic/multibeam/archive'

For sh/bash shells:
% set MB_CATALOG_PATH '/nfs/atapplic/multibeam/archive'

Say you wanted to create the standard HIPASS cube H035. To identify which scans are on which CDROMs, the coverage script has the following parameters: 

# t = file type: "SDF" or "HPF"
# c = cube number (overrides d,s,e)
# d = declination band
# s = start sequence
# e = end sequence
# m = scan letters required (eg "abcde")
# a = auto recurse in read dir eg. for CDs.
# r = read dir
# w = write dir
# x = read ext
# y = write ext
# q = quiet (0,1,2)
# b = (anything) means include bad scans - USE WITH CAUTION!

The following command will list all SDF (processed) CDROMs you need to make cube H035: 

coverage.pl -t SDF -c 035

The script goes through all size files and lists the required CDROMs, which in this case, are 68, 71, 156, 157, 160, 161, 166, 196, 280, 296, 360, 408 and 416. You can then configure the coverage script to automatically copy the required files from each loaded CDROM to disk by creating a shell script containing the following: 

#wait for CDROM to mount:
sleep 10
#now identify scans in cube H035 and copy to disk:
coverage.pl -t SDF \
            -c 035 \
            -r /cdrom/cdrom0 \
            -w /path/to/diskspace \
            -x sdfits \
            -y sdfits \
            -a 0 \
#eject and load another CDROM if required
eject cdrom

Once all scans have been copied from CDROM to disk and assuming you have entered the correct path(s) to the SDFITS files in the Search path entry box of the gridzilla GUI, selecting the HIPASS option at the Parameter set widget brings up the list of standard cubes. Each shows central coordinates and cube name as in Figure 4.10. The central position of cube H035 is shown as 1211-66 (12h 11m 00s, -66d 00m 00s J2000).

Gridzilla invokes the coverage script again and goes through all size files. After the script finishes, the log window lists all scans required to make up the cube and also if each scan was found in the search path(s). In this example we see that out of all the 'abcde' scans in the cube, only the 'a' scans where found. For this reason, the output FITS file name becomes 'H035_a.fits' instead of H035.fits (which indicates all SDFITS files were found). At this stage you can create the partial cube or load more scans with the coverage shell script.

Once you select all scans and press the Go button, the gridzilla log window lists parameters of the output cube such as velocity/frequency range and central pixel coordinates. Depending on the amount of available memory, gridzilla may combine all scans into a single cube, but if not, several "cubelets" will be made. You can concatenate them with Cubecat (see below).

Figure 4.10: Using gridzilla to construct the HIPASS cube H035.
\begin{figure} \begin{center} \epsfig{file=gridzilla.example.ps,width=4in}\end{center}\end{figure}

Figure 4.11: The default cubecat GUI.
\begin{figure} \begin{center} \epsfig{file=cubecat.gui.ps,width=4in}\end{center}\end{figure}

4.6 Cubecat

Due to memory allocation, gridzilla may not create a single cube. If gridzilla outputs may cubelets, you can concatentate them together with cubecat. Cubecat also allows for post-processing options such as continuum source ripple removal. To start up cubecat: 

% glish -l cubecat.g

The default GUI is displayed in Figure 4.11. Cubecat is self-documenting. For information on what a particular widget does, simply place the mouse cursor on the widget for an explaination. A more detailed description is available by pressing the right-most button on the mouse whilst over the widget. The type and amount of detail available can be altered via the "?" widget located at the top RH corner of the main GUI.

next up previous contents
Next: 5. Reducing JCMT data in DISH Up: Volume 3 - Telescope Specific Processing Previous: 3. ATCA reduction   Contents