MSFlagger.h

Classes

MSFlagger -- MSFlagger specifies selections on a MeasurementSet (full description)

class MSFlagger

Interface

Public Members
MSFlagger()
MSFlagger(MSSelector& msSel)
MSFlagger(const MSFlagger& other)
MSFlagger& operator=(const MSFlagger& other)
~MSFlagger()
void setMSSelector(MSSelector& msSel)
Bool fillDataBuffer(const String& item, Bool ifrAxis)
Record diffDataBuffer(const String& direction, Int window=1, Bool doMedian = False)
Record getDataBuffer()
Bool clipDataBuffer(Float pixelLevel, Float timeLevel, Float channelLevel)
Bool setDataBufferFlags(const Record& flags)
Bool writeDataBufferFlags()
Bool clearDataBuffer()
Bool createFlagHistory(Int nHis = 2)
Bool restoreFlags(Int level=-1)
Bool saveFlags(Bool newLevel)
Int flagLevel()
Protected Members
void fillFlagHist(Int nHis, Int numCorr, Int numChan, Table& tab)
Bool findHypercubeId(String& hyperCubeId, const String& column, const Table& tab)
void saveToFlagHist(Int level, Table& tab)
void applyFlagHist(Int level, Table& tab)
void getStats(Array<Float>& medTF, Array<Float>& adTF, Array<Float>& medT, Array<Float>& medFmedT, Array<Float>& adT, Array<Float>& medF, Array<Float>& medTmedF, Array<Float>& adF, const Array<Float>& diff, const Array<Bool>& flag, const Array<Bool>& flagRow)
void addStats(Record& buf, const Array<Bool>& flag, const Array<Bool> flagRow, const Array<Float>& data)
void reorderFlagRow(Array<Bool>& flagRow)
void diffMedian(Array<Float>& out, const Array<Float>& in, Int axis, const Array<Bool>& flag)
void applyRowFlags(Array<Bool>& flag, Array<Bool>& flagRow)
Bool check()

Description

Review Status

Date Reviewed:
yyyy/mm/dd

Prerequisite

Etymology

MSFlagger is a class that sets flags in an MS

Synopsis

This class is used to change the flag and flag_history columns in a MeasurementSet. It provides functions for automated flagging based on clipping the data that is too far from the median value. The ms DO uses this class to allow flagging from glish or a GUI.

    MSFlagger msFlagger(myMS);
    

Motivation

Flagging/editing of data is a central requirement in data processing, this class provides some simple flagging algorithms and the code that modifies & creates flag columns in the MS.

Thrown Exceptions

To Do

Member Description

MSFlagger()

MSFlagger(MSSelector& msSel)

construct from an MSSelector object

MSFlagger(const MSFlagger& other)

Copy constructor

MSFlagger& operator=(const MSFlagger& other)

Assignment

~MSFlagger()

void setMSSelector(MSSelector& msSel)

Change or Set the MS this MSFlagger refers to.

Bool fillDataBuffer(const String& item, Bool ifrAxis)

Fill an internal buffer with the data item requested, similar to getData except that the data is not returned, but kept around for further processing. Only a single DATA related quantity can be requested, the corresponding FLAG and FLAG_ROW columns are read automatically. Reorder the data to 4d with ifr and time axis if ifrAxis is True.

Record diffDataBuffer(const String& direction, Int window=1, Bool doMedian = False)

Difference the data, subtracting the average over a window of specified width and taking the absolute value. Complex quantities are turned into the corresponding amplitude after differencing. If doMedian==True the median difference is returned for window>2. For a window width of one, the previous sample is subtracted, giving a derivative like quantity. Note that the subtraction is done on row-by-row basis for TIME differencing, it is up to you to select a single baseline (if you didn't use ifrAxis=True in fillDataBuffer). Available directions are: TIME, CHANNEL Returns statistics over the buffer: median for times and channels, average absolute deviation over times, channels and all pixels.

Record getDataBuffer()

Return the contents of the internal data buffer, including the flags as a Record

Bool clipDataBuffer(Float pixelLevel, Float timeLevel, Float channelLevel)

Clip the data buffer at a specified level by setting the corresponding flags in the buffer. The cliplevel is specified as a multiple of the average absolute deviations returned by diffDataBuffer. A value of zero or less will skip the corresponding clip operation. Clipping will be done repeatedly, recalculating the deviations, until no more points are clipped.

Bool setDataBufferFlags(const Record& flags)

Replace the flags in the buffer with those in the supplied record. This allows interactive flagging from glish to be written back to the buffer for subsequent operations. The record should contain a flag and flag_row field.

Bool writeDataBufferFlags()

Write the flags in the buffer back to the table

Bool clearDataBuffer()

Clear the internal data buffer, reclaiming memory

Bool createFlagHistory(Int nHis = 2)

Create the FLAG_HISTORY column and initialize it from the FLAG_ROW and FLAG columns. Returns False if FLAG_HISTORY already exists. The first flagging bit is filled with the flags as found in the MS, subsequent bits can be used for user generated flags.

Bool restoreFlags(Int level=-1)

Apply the flags in the FLAG_HISTORY column to the FLAG and FLAG_ROW columns. Returns False if FLAG_HISTORY doesn't exist. The default argument will apply the currently active flag level (as specified by the FLAG_LEVEL column keyword). Sets the current level to the flag level restored.

Bool saveFlags(Bool newLevel)

Save the current flags to the FLAG_HISTORY. Save to the currently active level or (newLevel=True) the next highest level (if available). Will reset the current level to the level saved to.

Int flagLevel()

Return the current flaglevel (value of FLAG_LEVEL keyword)

void fillFlagHist(Int nHis, Int numCorr, Int numChan, Table& tab)

fill the FLAG_HISTORY column from the FLAG and FLAG_ROW column

Bool findHypercubeId(String& hyperCubeId, const String& column, const Table& tab)

find the HypercubeId column for a tiled column (if any)

void saveToFlagHist(Int level, Table& tab)

copy the flags to the flag history

void applyFlagHist(Int level, Table& tab)

copy the flag history back to the flags

void getStats(Array<Float>& medTF, Array<Float>& adTF, Array<Float>& medT, Array<Float>& medFmedT, Array<Float>& adT, Array<Float>& medF, Array<Float>& medTmedF, Array<Float>& adF, const Array<Float>& diff, const Array<Bool>& flag, const Array<Bool>& flagRow)

get buffer statistics - med=median, ad=average absolute deviation, T=Time, F=Frequency.

void addStats(Record& buf, const Array<Bool>& flag, const Array<Bool> flagRow, const Array<Float>& data)

add the statistics to a buffer

void reorderFlagRow(Array<Bool>& flagRow)

reorder from 2d to 1d (removing ifr axis)

void diffMedian(Array<Float>& out, const Array<Float>& in, Int axis, const Array<Bool>& flag)

collapse array "in" (with absolute differences) along specified axis by taking medians by profile taking into account the flags.

void applyRowFlags(Array<Bool>& flag, Array<Bool>& flagRow)

apply the row flags to the data flags and v.v.

Bool check()

check if we are attached to an MSSelector