VI2 Basic Operation

The VI2 (as did it ancestors) is a framework that walks the visibility data. A VI is created by providing a Measurement Set object to its constructor (or factory in the more complex scenarios). The user then walks the MS using a structure of the form:

VisBuffer2 * vb = vi2.getVisBuffer();

for (vi2.originChunks(); vi2.moreChunks(); vi2.nextChunk()){

    for (vi2.origin(); vi2.more(); vi2.next()){

        // access the current block of data using the VisBuffer2 object.

    }

}
  

A chunk is a set of rows in the MS that have the same selection criteria (usually the same array ID, data descriptor id, etc.) and falling within the same interval of time (e.g., ten seconds). The inner loop usually access a block of MS rows that have the same timestamp as well; I've been calling this a "subchunk". The rows in the subchunk are accessed via the VB2 that is attached to the MS (for some operations the data is copied into an unattached VB2 which does not update as the vi2 is incremented).

The usual mode of operation causes a VB2 to be filled with all the rows for a particular timestamp that have the same DDID, ArrayID, SourceID, etc. For some applications (e.g., imaging) that's pretty much it: they walk the MS from start to finish and use the data as they go; the same holds for applying the calibration corrections, though not for calculating the calibrations. Originally, applycal would apply the calibrations to the VB2 and then cause it to be written back out. Now we are working towards on-the-fly application so that there is never an on-disk set of applied calibrations; that's where the TransformingVI framework comes in.

The columns normally used to key the subchunk are ARRAY_ID, FIELD_ID, DATA_DESC_ID and TIME. The first three of these are simply tested for equality when selecting rows from the MS; for a chunk it allows a time + an interval (one of the MS constructor arguments) while for a subchunk, time is again checked for equality.

By normally, I mean the default values. You can play with the sort order and also use setRowBlocking to allow more timestamps into a subchunk (apparently of a use for single dish work where a subchunk would normally only contain a single row). Changing the sort order or the rowBlocking can have some odd effects and are not normally used only in special cases.

VisBuffer2

The data in each subchunk is accessed through the VisBuffer2 that is attached to the VI2. You can also create unattached VisBuffer2 objects if it is helpful to keep a copy of the subchunk data around. In that case you must be aware that the VisBuffer2 only loads the data on demand when the particular item is accessed; thus before you can copy the data to another VisBuffer2 you would need to "touch" the relevant data times. All the rows for the subchunk are accessed via the VisBuffer2 that is attached to the VI; this will have all the baselines and potentially the autocorrelations, too. The VB2 has an accessor method for each of the columns of the MS (more or less). The last dimension is the row number:

vb2.antenna1() --> Vector [nRows]

vb2.antenna2() --> Vector [nRows]

vb2.uvw() --> Matrix [3, nRows]

vb2.visCube() --> Cube [nCorrelations, nChannels, nRows]
				
    

The method names are usually close to the column names, though not always (the visibility cubes are one such case).

Modification

The data in the VisBuffer2 can be modified using various set methods. When all of the changes are made, then VisBuffer2::writeChangesBack can be called on the VI2 attached VB2. Not all fields can be changed: those that would change a row's key (e.g., timestamp, DDID, antenna1/2, etc.) cannot be changed on the attached VB2 since this would be incompatible with the way the data is read in.

Channel/Frequency Selection

You can specify a frequency range which will adjust the number of channels put into the cube; whether there is any I/O savings depends on how the data is tiled (under the usual tiling there probably won't be any I/O savings). It uses the setFrequencySelections method which in turn uses a FrequencySelections object. If the frequency selection was done through an MSSelection you can provide the MSSelection object to the FrequencySelections object and it will set them appropriately.

Multipass Use

For many applications the entire MS (or sequence of MSs) are accessed from start for finish. However, it is also easy to iterate over a chunk multiple times. For instance, an automatic flagging operation might read a chunk of data and compute the statistics for the data in that chunk, then make a second pass using the calculated stats to flag any points which appear to be outliers.

VisBuffer2 * vb = vi2.getVisBuffer();

    for (vi2.originChunks(); vi2.moreChunks(); vi2.nextChunk()){

        for (vi2.origin(); vi2.more(); vi2.next()){

            // gather stats during first pass

        }

        for (vi2.origin(); vi2.more(); vi2.next()){

            // use stats to flag the data during a second pass

        }
    }
    

VI2 Framework Structure

The class VisibilityIterator2 is the class that the user actually sees, however, like CasaCore's table class, it uses a delegation pattern: it delegates the actual implementation to an enclosed ViImplementation2 derived object which actually performs the operations. ViImplementation2 is an abstract class and has three main derived classes at this writing:

VisibilityIteratorImpl2

Reads the data from one or more Measurement Sets

TransformingVi2

Transforms data received from an input ViImplementation2 object

SimpleSimVi

Provides data that generates on the fly to simulate an MS

Transforming VI2

In this approach a hidden layer of Transforming Visibility Iterator implementations work to provide the outer, VisibilityIterator, with data that has gone through a number of transformations applied to the original data; it's similar to function composition, h(g(f(x))), where each each transforming VI is one function application. It's a bit more complex since the "x" in use is actually a fairly large data structure (usually the data contained in a VisBuffer2) rather than a scalar value; each step in the process modifies those components of the data that are appropriate to its transformation. For example, a simple transformation step might be the application of a Hann window to the visibility data which would modify the CORRECTED_DATA column but leave the others unchanged. At the other extreme is time averaging which modifies most of the columns in the MS; it's also unusual actually takes a number of VisBuffer2s as input and generates a single VB2 as its logical output.