Module MeasurementSets
Changes made in the current development cycle can be
found in the changelog.
Handle storage and access of telescope data
Prerequisite
Review Status
- Reviewed By:
- Bob Garwood
- Date Reviewed:
- 1997/02/01
Etymology
The MeasurementSet is where all data are ultimately to be found
in AIPS++. Since, this is a collection of
measurements (either actual or simulated), the term MeasurementSet
seems appropriate. Often we use the abbreviation (and typedef) MS for
MeasurementSet.
Synopsis
The MeasurementSets module handles storage of telescope data and access
to it. The MeasurementSet is the class that gives access to the data.
A MeasurementSet (MS)
is a Table with subtables stored as keywords.
For each of these tables there is a separate class: the main table is
MeasurementSet, the subtables are MSAntenna,
MSArray, MSFeed, MSField, MSObsLog, MSObservation, MSSource,
MSSpectralWindow, MSSysCal, MSWeather.
Class hierarchy and Table layout
Each table has a number
of predefined columns and keywords, a subset of which is required to be
present. The column and keyword layout of each table is described in
AIPS++ Note 191
and in a separate class which contains two enum definitions.
The enum classes, e.g.,
MSMainEnums and
MSAntennaEnums, just contain a
PredefinedColumn (PDC) enum and a PredefinedKeyword (PDK) enum. These enum
definitions are used as template arguments for the generic class
MSTable<PDC,PDK> from which MeasurementSet
and all the subtable classes are derived.
Thus, e.g., the class MSAntenna is derived from
MSTable<MSAntennaEnums::PredefinedColumns, MSAntennaEnums::PredefinedKeywords>.
The MSTable class
provides a large number of common column and keyword helper functions.
They are useful when creating a Table following the MeasurementSet
conventions from scratch and assist in following the agreed upon
column and keyword naming conventions.
MSTable in turn is derived from Table, thus all the MS table classes are
Tables. Many operations on a MeasurementSet are Table operations. See the
Tables module for a list of
those operations.
The reason for this class hierarchy is to provide each of the table classes
with a separate namespace for its column and keyword enums, so that
e.g., they can all have a column named TIME, while sharing as much code
as possible. The MSTable class forwards any substantial work
to the MSTableImpl class which does the actual work, it is a purely
implementation class with static functions not of interest to the user.
Access to columns
To simplify the access of columns, and catch type errors in
the column declarations for column access objects (TableColumns) at
compile time, there is a helper class for each (sub)table (e.g.,
MSFieldColumns). The helper class for the MeasurementSet,
MSColumns gives access to the main table columns and the helper objects
for all subtables. A read-only version of these classes is also
provided (e.g., ROMSFieldColumns).
At present these classes are separate from the Table classes, mainly
to ensure that the member functions are only called on valid, completely
constructed MeasurementSet Tables. They also represent a large amount
of 'state' in the form of TableColumn objects of which only a couple
may actually be used.
Units and Measures
Columns in the MeasurementSet and its subtables can have several
keywords attached to describe the contents of the column.
The UNIT keyword contains an ASCII string describing the unit of
the values in the column. The unit member function (in MSTable) will
return this unit string, it can be used to construct a
Unit object for a particular column.
The MEASURE_TYPE keyword gives the AIPS++ Measure that applies to the
column (if any). See the
Measures module for a
list of available Measures and their use.
The MEASURE_REFERENCE keyword gives (part of) the reference frame needed
to interpret the values in a column. An example is J2000 for the POSITION
column. A number of static functions in MeasurementSet are available to
create a 'template' Measure for a column, which has the MEASURE_TYPE filled
in. Currently the following functions exist: directionMeasure,
positionMeasure, epochMeasure and frequencyMeasure. They return a
Measure which can then be filled with a value from a particular row from
the column to obtain, e.g., a proper MDirection Measure for the phase
center.
Reference Tables
Each of the MS classes has a member function
referenceCopy which takes the name of a new Table and a list (Block) of
column names to create a Table which references all columns in the
original table, except for those listed. The listed columns are
new columns which can be written to without affecting the original Table.
The main use of this is for the synthesis package where corrected and
model visibilities are stored as new DATA columns in an MS which
references the raw MS for the other columns. Except for these special
cases, the use of this function will be rare.
DATA and FLOAT_DATA columns
To accommodate both synthesis and single dish data efficiently, it was
decided that a MeasurementSet can have a Complex DATA column,
a float FLOAT_DATA column or both. If it has only a FLOAT_DATA column, the
corresponding DATA column can be created with the makeComplexData()
member function. In special cases, both columns could be present but
filled for different rows, with an extra index defined indicating in
which column to look (e.g., multi-feed single dish with cross correlations).
The details of this last scheme are yet to be worked out.
The table consistency checks (isValid()) do not require the presence
of either column.
Unset values in MeasurementSet Tables
For ID columns, the rule is that a value of -1 indicates that there is
no corresponding subtable in which to look up details. An example is
the PULSAR_ID column, which should be set to -1 if the optional
PULSAR subtable does not exist.
The rules for non integer unset values in MS tables have not
settled down yet.
For Floating point and Complex values the recommended practice is
to set the values to the FITS and IEEE value NaN,
with a bit pattern of all 1's.
In much of the present filler code unused values are filled with 0 instead.
Table destruction
Upon destruction, the table and all subtables are checked to see that the
MeasurementSet remains valid, i.e., all required columns are present.
An exception is thrown if not all required columns are present
Nevertheless, the table will be flushed to disk if it is writable -
preserving its state.
MS shorthand
While the class name, MeasurementSet, is descriptive, it is often
too long for many common uses. The typedef MS is provided as
a convenient shorthand for MeasurementSet. The example below uses this
typedef.
Example
This example illustrates creation and filling of the MeasurementSet.
// create the table descriptor
TableDesc simpleDesc = MS::requiredTableDesc()
// set up a new table
SetupNewTable newTab("simpleTab", simpleDesc, Table::New);
// create the MeasurementSet
MeasurementSet simpleMS(newTab);
// now we need to define all required subtables
// the following call does this for us if we don't need to
// specify details of Storage Managers or non-standard columns.
simpleMS.createDummySubtables(Table::New);
// fill MeasurementSet via its Table interface
// For example, construct one of the column access objects.
TableColumn feed(simpleMS, MS::columnName(MS::FEED1));
uInt rownr = 0;
// add a row
simpleMS.addRow();
// set the values in that row, e.g. the feed column
feed.putScalar(rownr,1);
// Access the position column in the ANTENNA subtable
ArrayColumn<Double> antpos(simpleMS.antenna(),
MSAntenna::columnName(MSAntenna::POSITION));
// Add a row to it and fill in the position
simpleMS.antenna().addRow();
Array<Double> position(3);
position(0)=1.; position(1)=2.; position(2)=3.;
antpos.put(0,position);
// .
// For standard columns the above can be done more easily using
// the MSColumns object.
// Create the MSColumns
MSColumns msc(simpleMS);
// and fill in the position
msc.antenna().position().put(0,position);
Example
This example illustrates read only access to an existing MeasurementSet
and creation of an MDirection Measure for the phase center.
// Create the MeasurementSet from an existing Table on disk
MeasurementSet ms("myMS");
// Create the RO column access objects for main table and subtables
ROMSColumns msc(ms);
// show data from row 5
cout << msc.data()(5) << endl;
// show phase center for row 3 in field table
Vector<double> phaseCtr=msc.field().phaseCenter()(3);
cout << phaseCtr<<endl;
// now create a Measure for the phaseCenter
MDirection phaseCenterMeasure =
MS::directionMeasure(msc.field().phaseCenter());
// put the value from row 3 in the Measure and print it
phaseCenterMeasure.set(MVPosition(phaseCtr));
cout <<"phase center:"<< phaseCenterMeasure<<endl;
Motivation
The attempt is to define a single, extensible, Table format that will
be able to cope with all, or at least most, radio telescope data.
Having a single MeasurementSet should make it easier to combine data
from different instruments. The format of the MeasurementSet,
table with subtables, was chosen to be able to cope with items
varying at different rates more efficiently than a 'flat' Table layout
would allow.
To Do
- Incorporate the MSColumn classes in the MeasurementSet classes?
- Variable (row to row) ReferenceFrame (e.g., J2000 mixed with
galactic, different Frequency reference frames mixed in the
same MS, etc.). This could be done with a column named
"column_name"_MEASURE_REFERENCE for each column with varying
Measure reference frames.
- ASDM2MSFiller -- (full description)
- AlmaTI2MS -- AlmaTI2MS: Convert ALMA-TI data to MS format (full description)
- CorrDataKeywords -- CorrDataKeywords: process CORRDATA-ALMATI FITS keywords (full description)
- DataParKeywords -- DataParKeywords: process DATAPAR-ALMATI FITS keywords (full description)
- MS1ToMS2Converter -- Class to convert a MeasurementSet v1 to v2. (full description)
- MSAntenna -- A Table intended to hold a MeasurementSet ANTENNA table. (full description)
- MSAntennaColumns -- A class to provide easy read-write access to MSAntenna columns (full description)
- MSAntennaEnums -- Enums for the MeasurementSet ANTENNA table (full description)
- MSAntennaGramFunctions -- Global functions for flex/bison scanner/parser for MSAntennaGram (full description)
- MSAntennaIndex -- Class to handle lookup or indexing into a MS ANTENNA subtable (full description)
- MSAntennaParse -- Class to hold values from antenna grammar parser (full description)
- MSColumns -- A class to provide easy read-write access to MeasurementSet columns (full description)
- MSConcat -- A class with functions for concatenating MeasurementSets (full description)
- MSContinuumSubtractor -- Fits and subtracts or models the continuum in spectra (full description)
- MSCorrGramFunctions -- Global functions for flex/bison scanner/parser for MSCorrGram (full description)
- MSCorrParse -- Class to hold values from UV dist grammar parser (full description)
- MSDataDescColumns -- A class to provide easy read-write access to MSDataDescription columns (full description)
- MSDataDescIndex -- Class to handle lookup or indexing into a MS DATA_DESC subtable (full description)
- MSDataDescription -- A Table intended to hold a MeasurementSet DATADESCRIPTION table. (full description)
- MSDataDescriptionEnums -- Enums for the MeasurementSet DATA_DESCRIPTION table (full description)
- MSDerivedValues -- MSDerivedValues calculates values derived from a MS (full description)
- MSDoppler -- A Table intended to hold a MeasurementSet DOPPLER table. (full description)
- MSDopplerColumns -- A class to provide easy read-write access to MSDoppler columns (full description)
- MSDopplerEnums -- Enums for the MeasurementSet DOPPLER table (full description)
- MSDopplerIndex -- (full description)
- MSDopplerUtil -- A utility class for MS Doppler tracking information (full description)
- MSFeed -- A Table intended to hold a MeasurementSet FEED table. (full description)
- MSFeedColumns -- A class to provide easy read-write access to MSFeed columns (full description)
- MSFeedEnums -- Enums for the MeasurementSet FEED table (full description)
- MSFeedIndex -- (full description)
- MSField -- A Table intended to hold a MeasurementSet FIELD table. (full description)
- MSFieldColumns -- A class to provide easy read-write access to MSField columns (full description)
- MSFieldEnums -- Enums for the MeasurementSet FIELD table (full description)
- MSFieldGramFunctions -- Global functions for flex/bison scanner/parser for MSFieldGram (full description)
- MSFieldIndex -- Class to handle lookup or indexing into a MS FIELD subtable (full description)
- MSFieldParse -- Class to hold values from field grammar parser (full description)
- MSFlagCmd -- A Table intended to hold a MeasurementSet FLAG_CMD table. (full description)
- MSFlagCmdColumns -- A class to provide easy read-write access to MSFlagCmd columns (full description)
- MSFlagCmdEnums -- Enums for the MeasurementSet FLAG_CMD table (full description)
- MSFlagger -- MSFlagger specifies selections on a MeasurementSet (full description)
- MSFreqOffIndex -- (full description)
- MSFreqOffset -- A Table intended to hold a MeasurementSet FREQ_OFFSET table. (full description)
- MSFreqOffsetColumns -- A class to provide easy read-write access to MSFreqOffset columns (full description)
- MSFreqOffsetEnums -- Enums for the MeasurementSet FREQ_OFFSET table (full description)
- MSHistory -- A Table intended to hold a MeasurementSet OBSERVATIONLOG table. (full description)
- MSHistoryColumns -- A class to provide easy read-write access to MSHistory columns (full description)
- MSHistoryEnums -- Enums for the MeasurementSet HISTORY table (full description)
- MSHistoryHandler -- A class to provide a simple interface to history writing (full description)
- MSInterval -- Small helper class to specify an 'interval' comparison (full description)
- MSIter -- An iterator class for MeasurementSets (full description)
- MSLister -- List visibility records from a Measurement Set (full description)
- MSMainColumns -- A class for easy read-write access to MeasurementSet main table columns (full description)
- MSMainEnums -- Enums for the MeasurementSet main table (full description)
- MSObservation -- A Table intended to hold a MeasurementSet OBSERVATION table. (full description)
- MSObservationColumns -- A class to provide easy read-write access to MSObservation columns (full description)
- MSObservationEnums -- Enums for the MeasurementSet OBSERVATION table (full description)
- MSObservationIndex -- Class to handle lookup or indexing into an MS OBSERVATION subtable (full description)
- MSParse -- Class to hold values from an ms grammar parser (full description)
- MSPointing -- A Table intended to hold a MeasurementSet POINTING table. (full description)
- MSPointingColumns -- A class to provide easy read-write access to MSPointing columns (full description)
- MSPointingEnums -- Enums for the MeasurementSet POINTING table (full description)
- MSPointingIndex -- (full description)
- MSPolarization -- A Table intended to hold a MeasurementSet POLARIZATION table. (full description)
- MSPolarizationColumns -- A class to provide easy read-write access to MSPolarization columns (full description)
- MSPolarizationEnums -- Enums for the MeasurementSet POLARIZATION table (full description)
- MSPolarizationIndex -- Class to handle lookup or indexing into an MS POLARIZATION subtable (full description)
- MSProcessor -- A Table intended to hold a MeasurementSet PROCESSOR table. (full description)
- MSProcessorColumns -- A class to provide easy read-write access to MSProcessor columns (full description)
- MSProcessorEnums -- Enums for the MeasurementSet PROCESSER table (full description)
- MSRange -- MSRange determines ranges of values in a MeasurementSet (full description)
- MSReader -- Read from an MS, coordinating all of the subtables in the process (full description)
- MSScanGramFunctions -- Global functions for flex/bison scanner/parser for MSScanGram (full description)
- MSScanParse -- Class to hold values from scan grammar parser (full description)
- MSSelUtil -- Helper class for MSFlagger with templated static function (full description)
- MSSelUtil2 -- Helper class for MSSelector/DOms with templated static functions (full description)
- MSSelection -- MSSelection: Class to represent a selection on an MS (full description)
- MSSelectionAntennaError -- Error thrown by MSAntennaXXXX classes. (full description)
- MSSelectionAntennaParseError -- (full description)
- MSSelectionError -- (full description)
- MSSelectionFieldError -- Error thrown by MSFieldXXXX classes. (full description)
- MSSelectionFieldParseError -- (full description)
- MSSelectionKeywords -- MSSelectionKeywords specifies selection keywords for the MeasurementSet (full description)
- MSSelectionSpwError -- Error thrown by MSSpwXXXX classes. (full description)
- MSSelectionSpwParseError -- (full description)
- MSSelectionSpwWarning -- (full description)
- MSSelectionTimeError -- Error thrown by MSTimeXXXX classes. (full description)
- MSSelectionTimeParseError -- (full description)
- MSSelectionUvDistError -- Error thrown by MSUvDistXXXX classes. (full description)
- MSSelectionUvDistParseError -- (full description)
- MSSelector -- MSSelector specifies selections on a MeasurementSet (full description)
- MSSimulator -- Create an empty MeasurementSet from observation and telescope descriptions. (full description)
- MSSource -- A Table intended to hold a MeasurementSet SOURCE table. (full description)
- MSSourceColumns -- A class to provide easy read-write access to MSSource columns (full description)
- MSSourceEnums -- Enums for the MeasurementSet SOURCE table (full description)
- MSSourceIndex -- (full description)
- MSSpWindowColumns -- A class to provide easy read-write access to MSSpectralWindow columns (full description)
- MSSpWindowIndex -- Class to handle lookup or indexing into a MS SPECTRAL_WINDOW subtable (full description)
- MSSpectralWindow -- A Table intended to hold a MeasurementSet SPECTRAL_WINDOW table. (full description)
- MSSpectralWindowEnums -- Enums for the MeasurementSet SPECTRAL_WINDOW table (full description)
- MSSpwGramFunctions -- Global functions for flex/bison scanner/parser for MSSpwGram (full description)
- MSSpwIndex -- Class to handle lookup or indexing into a MS Data_Desc_ID and SpectralWindow subtables (full description)
- MSSpwParse -- Class to hold values from field grammar parser (full description)
- MSState -- A Table intended to hold a MeasurementSet STATE table. (full description)
- MSStateColumns -- A class to provide easy read-write access to MSState columns (full description)
- MSStateEnums -- Enums for the MeasurementSet STATE table (full description)
- MSSummary -- Provides and lists information about the header of an image (full description)
- MSSysCal -- A Table intended to hold a MeasurementSet SYSCAL table. (full description)
- MSSysCalColumns -- A class to provide easy read-write access to MSSysCal columns (full description)
- MSSysCalEnums -- Enums for the MeasurementSet SYSCAL table (full description)
- MSSysCalIndex -- (full description)
- MSTable -- A Table intended to hold astronomical data (full description)
- MSTableImpl -- An implementation class for the MeasurementSet to share code. (full description)
- MSTableIndex -- (full description)
- MSTileLayout -- An helper class for deciding on tile shapes in MeasurementSets (full description)
- MSTimeGramFunctions -- Global functions for flex/bison scanner/parser for MSTimeGram (full description)
- MSTimeParse -- Class to hold values from time grammar parser (full description)
- MSUvDistGramFunctions -- Global functions for flex/bison scanner/parser for MSUvDistGram (full description)
- MSUvDistParse -- Class to hold values from UV dist grammar parser (full description)
- MSValidIds -- (full description)
- MSWeather -- A Table intended to hold a MeasurementSet WEATHER table. (full description)
- MSWeatherColumns -- A class to provide easy read-write access to MSWeather columns (full description)
- MSWeatherEnums -- Enums for the MeasurementSet WEATHER table (full description)
- MSWeatherIndex -- (full description)
- MeasurementSet -- A Table intended to hold astronomical data (a set of Measurements). (full description)
- NewMSSimulator -- Create an empty MeasurementSet from observation and telescope descriptions. (full description)
- ROMSAntennaColumns -- A class to provide easy read-only access to MSAntenna columns (full description)
- ROMSColumns -- A class to provide easy read-only access to MeasurementSet columns (full description)
- ROMSDataDescColumns -- A class to provide easy read-only access to MSDataDesc columns (full description)
- ROMSDopplerColumns -- A class to provide easy read-only access to MSDoppler columns (full description)
- ROMSFeedColumns -- A class to provide easy read-only access to MSFeed columns (full description)
- ROMSFieldColumns -- A class to provide easy access to MSField columns (full description)
- ROMSFlagCmdColumns -- A class to provide easy read-only access to MSFlagCmd columns (full description)
- ROMSFreqOffsetColumns -- A class to provide easy read-only access to MSFreqOffset columns (full description)
- ROMSHistoryColumns -- A class to provide easy read-only access to MSHistory columns (full description)
- ROMSMainColumns -- A class for easy read-only access to MeasurementSet main table columns (full description)
- ROMSObservationColumns -- A class to provide easy read-only access to MSObservation columns (full description)
- ROMSPointingColumns -- A class to provide easy read-only access to MSPointing columns (full description)
- ROMSPolarizationColumns -- A class to provide easy read-only access to MSPolarization columns (full description)
- ROMSProcessorColumns -- A class to provide easy read-only access to MSProcessor columns (full description)
- ROMSSourceColumns -- A class to provide easy read-only access to MSSource columns (full description)
- ROMSSpWindowColumns -- A class to provide easy read-only access to MSASpectralWindow columns (full description)
- ROMSStateColumns -- A class to provide easy read-only access to MSState columns (full description)
- ROMSSysCalColumns -- A class to provide easy read-only access to MSSysCal columns (full description)
- ROMSWeatherColumns -- A class to provide easy read-only access to MSWeather columns (full description)
- StokesConverter -- StokesConverter converts any set of polarizations into any other one (full description)
- TimeFields -- (full description)
- ddMgr -- (full description)
- timeMgr -- (full description)