ms - Tool

Postscript file available

Operations on measurement sets
include "ms.g"

Constructors

fitstoms	Create an ms tool from a UVFITS file
ms	Create an ms tool attached to a measurement set
sdfitstoms	Create an ms tool from a SDFITS file

Functions

clearbuffer	Clear the internal buffer.
clipbuffer	Clip the internal buffer with specified limits.
close	Detach the ms `tool` from the measurement set table
command	Construct a ms `tool` using a table query command
concatenate	Concatenate two measurement sets
continuumsub	Continuum fitting and subtraction in uv plane
createflaghistory	Create and initialize the FLAG_HISTORY column.
diffbuffer	Differentiate or difference the internal buffer.
done	Delete the ms `tool`
fillbuffer	Fill the internal buffer with data and flags.
flaglevel	return the current flagging level
getbuffer	Return the internal buffer as a Record for access from Glish.
getdata	Read values from the measurement set.
iswritable	Returns True is the underlying Table is writable
iterend	End the iteration and reset the selected table
iterinit	Initialize for iteration over a MS
iternext	Advance the iterator to the next lot of data
iterorigin	Set the iterator to the start of the data.
lister	List the measurement set's data in a concise form
listhistory	List history of the measurement set
name	Name of the measurement set table the tool is attached to.
nrow	Returns the number of rows in the measurement set
open	Attach the ms `tool` to a measurement set table
ptsrc	Reports global average of visibilities per field
putdata	Write new values into the measurement set
range	Get the range of values in the measurement set
restoreflags	Restore the specified flagging level from the FLAG_HISTORY column.
saveflags	Save the current flagging state to the FLAG_HISTORY column.
select	Select a subset of the measurement set.
selectchannel	Select and average frequency channels
selectinit	Initialize the selection of a MS
selectpolarization	Select and convert of polarizations
selecttaql	Select a subset of the measurement set.
setbufferflags	Set the flags in the buffer
split	make a new ms from a subset of actual ms
summary	Summarize the measurement set
tofits	Convert a measurement set to a UVFITS file
tosdfits	Convert a single dish AIPS++ measurement set to a SDFITS file
uvlsf	Continuum fitting and subtraction in uv plane
writebufferflags	Write the flags in the internal buffer back to the table.
writehistory	Add a row of arbitrary information to the measurement set history table

Description

The ms tool is provides functions to manipulate the contents of measurement set tables.

It is easy to get an ms tool and a measurement set table confused. The table is the disk files that contain your data. The tool is a process on your computer that provides functions that allow you access the data in the table. One tool can access only one table at any time. If you want to access two measurement sets simultaneously you need to create two ms tools.

The ms tool is frequently used as a server of data for higher-level tools such as visplot.

The ms tool functions can be categorised as shown below.

Creating a ms tool

A ms tool is usually associated with a measurement set table. The simplest and most common way to make a ms tool is to use the ms constructor which requires you to specify the name of the measurement set table.

There are two other constructors which convert FITS files to a measurement set table prior to creating the ms tool (which is attached to the newly created measurement set -- not the FITS file). These are the fitstoms constructor, which reads a UVFITS file, and the sdfitstoms constructor, which reads a single dish FITS file. The conversion step may take some time if the FITS file is large. However it only needs to be done once. The measurement set table is not deleted when you delete the ms tool, using the done function, or exit AIPS++. And once the measurement set table is created it is much faster to create an ms tool using the ms constructor.

Getting summary information

The summary function will display, in the logger, an overview of the measurement set. This will include listings of the fields, spectral windows & polarization setups used in the measurement set.

The range function will provide more quantitative information on the minimum, maximum or used values of specified parameters. When using this function you may need to do an initial selection, as described below, depending on whether the parameters you ask for change their shape. A list of parameters accepted by the range function is given in table 1.355 and this table also indicates when an initial selection is necessary.

The lister function provides a concise listing of the data in the measurement set.

The listhistory function lists the contents of the measurement set history table. The history table contains a record of changes made to the measurement set by autoflag, calibrater, imager and other tools.

Selecting data

As described in the ms module documentation a measurement set can contain data with a variety of different shapes. Some of the functions in this tool require the data to be in a fixed shape. Before you can use these functions you need to select a subset of the data in the measurement set where all the data has a fixed shape. There are two functions which can be used to do this. These are the selectinit and command functions.

The select function can be used to further refine which subset of the data will be used by the data access functions. This function allows you to select specific rows in a measurement set using a wide range of criteria.

The select function can only select whole rows in a measurement set. To select specific channels within a row you use the selectchannel function. Similarly to select specific polarizations you should use the selectpolarization function.

Reading and writing data

The getdata function is used to read data from the measurement set into Glish variables. You can select which columns of the measurement set main table you are interested in and only the subset of data specified using the selection functions described above will be retreived. Any frequency averaging (see the selectfrequency function) and polarization conversion (see the selectpolarization function) will be done when you retreive the data. The full power of Glish and other AIPS++ tools, like pgplotter, can then be used for adhoc inspection and calculations involving the data.

If the measurement set was opened for writing then the putdata function can be used to write the data back into the measurement set. When writing data back into the measurement set you cannot change the data shape or the coordinates of the data, only the numerical values. This means that you cannot write data that has been averaged in frequency or converted to different polarizations.

When using the getdata function with a large measurement set you need to be careful to not request too much data. The measurement set is stored on disk but Glish variables are stored in memory. To allow you to access large amounts of data in an ordered way the ms tool provides functions that allow you iterate through the data is convienient way.

If you need to step through the data in an orderly fashion, you can use the iteration functions. These allow you to set up an iteration order (iterinit), obtain the first iteration (iterorigin), go to the next iteration (iternext) and end the iteration prematurely (iterend). The iterorigin and iternext function set the currently selected table (as used by getdata and others) to the current iteration. At the end of the iteration, the original selection is restored.

You can iterate through a measurement set you have previously selected using select, but if you use select while iterating, you cannot get back the unselected iteration (without reiterating through the table until the current point).

The writehistory function allows messages to be appended to the measurement set history table should the you wish to do so. The listhistory function lists your messages and those created by autoflag, calibrater, imager and other tools.

Conversions to FITS

Just as the fitstoms constructor will convert a UVFITS file to a measurement set the tofits function will convert a measurement set to a UVFITS file. Similarly a single dish measurement set ie., one with a FLOAT_DATA column rather than a DATA column, can be converted to a single dish FITS file using the tosdfits function.

You cannot read a UVFITS file into a measurement set and write it out as a single dish FITS file or vice-versa.

Concatenation

The concatenate function can be used to append the data from one measurement set to the end of another. As all the data is copied this function may take some time if the measurement set to be copied is large. The measurement set needs to be opened for writing for this to work.

Splitting The split function allow you to make a new ms from a subset of the actual ms

Flagging data

The flag and buffer functions all belong together. The idea is to fill a buffer of data (fillbuffer), optionally retrieve it as a record (getbuffer) and display it in some data display tool, do operations like differencing (diffbuffer), clipping (clipbuffer) and manual edits of the displayed data (setbufferflags), and then write the flags back to the MS (writebufferflags), into the appropriate flag level, so you can choose to apply or undo them. The data is untouched by these functions.

Table: Items recognized by the range, select, getdata and putdata functions. Items marked with a are only available in interferometric measurement sets. Items marked with a are only available in interferometric measurement sets that have been processed with calibrator or imager. Items marked with a * do not require all the data in the selected measurement set is the same shape.
range select getdata putdata comment

Continued from previous page

range select getdata putdata comment

amplitude $\dag$ + - + - amplitude of observed data

corrected_amplitude $\ddag$ + - + - amplitude of corrected data

model_amplitude $\ddag$ + - + - amplitude of model data

ratio_amplitude $\ddag$ - - + - amplitude of corrected/model

residual_amplitude $\ddag$ - - + - amplitude of residual data

obs_residual_amplitude $\ddag$ - - + - amplitude of obs residual data

antenna1 * + + - 1st antenna id

antenna2 * + + - 2nd antenna id

antennas * - - - list of antenna names

array_id * + - -

axis_info - - + - description of data axes

chan_freq + - - - channel frequencies

corr_names + - - - list of polarization strings

corr_types + - - - list of polarization enum values

data $\dag$ - - + + complex observed data

corrected_data $\ddag$ - - + + complex corrected data

model_data $\ddag$ - - + + complex model data

ratio_data $\ddag$ - - + - complex corrected/model

residual_data $\ddag$ - - + + complex residual data

obs_residual_data $\ddag$ - - + + complex observed residual data

feed1 * + + - 1st feed in correlation

feed2 * + + - 2nd feed in correlation

field_id * + + - field number

fields * - - - list of field names

flag - - + + data flags

flag_row - - + + MS row flags

flag_sum - - + - flag summary

ha - - + - add hour angle to axis_info

ifr_number * + + - 1000*antenna1+antenna2

imaginary $\dag$ + - + - imag part of observed data

corrected_imaginary $\ddag$ + - + - imag part of corrected data

model_imaginary $\ddag$ + - + - imag part of model data

ratio_imaginary $\ddag$ - - + - imag part of corrected/model

residual_imaginary $\ddag$ - - + - imag part of residual data

obs_residual_imaginary $\ddag$ - - + - imag part of obs residual data

imaging_weight $\ddag$ + - + + weights used for imaging

last - - + - add LAST to axis_info

num_corr + - - - number of polarizations

num_chan + - - - number of freq channels

phase $\dag$ + - + - phase of observed data

corrected_phase $\ddag$ + - + - phase of corrected data

model_phase $\ddag$ + - + - phase of model data

ratio_phase $\ddag$ - - + - phase of corrected/model

residual_phase $\ddag$ - - + - phase of residual data

obs_residual_phase $\ddag$ - - + - phase of observed residual data

phase_dir + - - - list of phase centers & epoch

real $\dag$ + - + - real part of observed data

corrected_real $\ddag$ + - + - real part of corrected data

model_real $\ddag$ + - + - real part of model data

ratio_real $\ddag$ - - + - real part of corrected/model

residual_real $\ddag$ - - + - real part of residual data

obs_residual_real $\ddag$ - - + - real part of observed res. data

ref_frequency + - - - reference frequency

rows * + - - row numbers in original table

scan_number * + + -

sigma * - + + sigma of the data

data_desc_id * + + -

time * + + - MJD time range in seconds

times * + - - list of MJD timeslots

ut - - + - add UT to axis_info

uvw - - + - uvw vector

u * + + - u coordinate

v * + + - v coordinate

w * + + - w coordinate

uvdist * + + - uv distance

weight * - + + weight of the data

**Table:** Items recognized by the range, select, getdata and putdata functions. Items marked with a are only available in interferometric measurement sets. Items marked with a are only available in interferometric measurement sets that have been processed with calibrator or imager. Items marked with a `*` do not require all the data in the selected measurement set is the same shape.
	range	select	getdata	putdata	comment
Continued from previous page
	range	select	getdata	putdata	comment
amplitude $\dag$	+	-	+	-	amplitude of observed data
corrected_amplitude $\ddag$	+	-	+	-	amplitude of corrected data
model_amplitude $\ddag$	+	-	+	-	amplitude of model data
ratio_amplitude $\ddag$	-	-	+	-	amplitude of corrected/model
residual_amplitude $\ddag$	-	-	+	-	amplitude of residual data
obs_residual_amplitude $\ddag$	-	-	+	-	amplitude of obs residual data
antenna1	`*`	+	+	-	1st antenna id
antenna2	`*`	+	+	-	2nd antenna id
antennas	`*`	-	-	-	list of antenna names
array_id	`*`	+	-	-
axis_info	-	-	+	-	description of data axes
chan_freq	+	-	-	-	channel frequencies
corr_names	+	-	-	-	list of polarization strings
corr_types	+	-	-	-	list of polarization enum values
data $\dag$	-	-	+	+	complex observed data
corrected_data $\ddag$	-	-	+	+	complex corrected data
model_data $\ddag$	-	-	+	+	complex model data
ratio_data $\ddag$	-	-	+	-	complex corrected/model
residual_data $\ddag$	-	-	+	+	complex residual data
obs_residual_data $\ddag$	-	-	+	+	complex observed residual data
feed1	`*`	+	+	-	1st feed in correlation
feed2	`*`	+	+	-	2nd feed in correlation
field_id	`*`	+	+	-	field number
fields	`*`	-	-	-	list of field names
flag	-	-	+	+	data flags
flag_row	-	-	+	+	MS row flags
flag_sum	-	-	+	-	flag summary
ha	-	-	+	-	add hour angle to axis_info
ifr_number	`*`	+	+	-	1000*antenna1+antenna2
imaginary $\dag$	+	-	+	-	imag part of observed data
corrected_imaginary $\ddag$	+	-	+	-	imag part of corrected data
model_imaginary $\ddag$	+	-	+	-	imag part of model data
ratio_imaginary $\ddag$	-	-	+	-	imag part of corrected/model
residual_imaginary $\ddag$	-	-	+	-	imag part of residual data
obs_residual_imaginary $\ddag$	-	-	+	-	imag part of obs residual data
imaging_weight $\ddag$	+	-	+	+	weights used for imaging
last	-	-	+	-	add LAST to axis_info
num_corr	+	-	-	-	number of polarizations
num_chan	+	-	-	-	number of freq channels
phase $\dag$	+	-	+	-	phase of observed data
corrected_phase $\ddag$	+	-	+	-	phase of corrected data
model_phase $\ddag$	+	-	+	-	phase of model data
ratio_phase $\ddag$	-	-	+	-	phase of corrected/model
residual_phase $\ddag$	-	-	+	-	phase of residual data
obs_residual_phase $\ddag$	-	-	+	-	phase of observed residual data
phase_dir	+	-	-	-	list of phase centers & epoch
real $\dag$	+	-	+	-	real part of observed data
corrected_real $\ddag$	+	-	+	-	real part of corrected data
model_real $\ddag$	+	-	+	-	real part of model data
ratio_real $\ddag$	-	-	+	-	real part of corrected/model
residual_real $\ddag$	-	-	+	-	real part of residual data
obs_residual_real $\ddag$	-	-	+	-	real part of observed res. data
ref_frequency	+	-	-	-	reference frequency
rows	`*`	+	-	-	row numbers in original table
scan_number	`*`	+	+	-
sigma	`*`	-	+	+	sigma of the data
data_desc_id	`*`	+	+	-
time	`*`	+	+	-	MJD time range in seconds
times	`*`	+	-	-	list of MJD timeslots
ut	-	-	+	-	add UT to axis_info
uvw	-	-	+	-	uvw vector
u	`*`	+	+	-	u coordinate
v	`*`	+	+	-	v coordinate
w	`*`	+	+	-	w coordinate
uvdist	`*`	+	+	-	uv distance
weight	`*`	-	+	+	weight of the data

Example

include 'ms.g'
include 'statistics.g'
myms:=ms("3C273XC1.MS",readonly=F);
myms.selectinit(datadescid=2)
myms.select([antenna1=[1,3,5],uvdist=[1200,1900]]);
rec:=myms.getdata("weight corrected_data");
rec.weight*:=1.2;
rec.corrected_data-:=mean(rec.corrected_data);
myms.putdata(rec);
myms.close();

We open the MS for writing, select an array and spectral window and then select a few antennas and a uv range. We then get out the weight values and the corrected_data. We change these values in Glish and then write them back to the measurement set. Finally, we close the ms, causing the values to be written back to disk. Note: you cannot use the name ms as a Glish variable (e.g., ms:=ms("MyMS")), because that would overwrite the ms definition itself.

Next: ms - Constructor Up: ms - Module Previous: ms - Module Contents Index

	Package	general
	Module	ms