The example uses the global Array function indgen to fill the data
and coordinate arrays with arbitrary values.
TiledDataStMan allows the user explicit control over the
definition and extension of hypercubes by means of the accessor
class TiledDataStManAccessor.
The user can determine which row should be put in which hypercube,
so it is possible to put row 0-9 in hypercube A, row 10-29 in B,
row 30-39 in A again, etc.. This makes it possible to use a tiled
storage manager for a data column containing data with
different shapes (e.g. line and continuum data). Actually,
this storage manager is developed for irregularly shaped
UV-data, but can be used for any purpose.
Each extensible hypercube uses a file of its own. This means that there
shouldn't be too many of them, otherwise the number of files may
get too high.
The TiledDataStMan has the following (extra) properties:
Note that the description of class ROTiledStManAccessor contains a discussion about the effect of setting the maximum cache size.
// Define the table description and the columns in it.
TableDesc td ("", "1", TableDesc::Scratch);
td.addColumn (ScalarColumnDesc<float> ("Time"));
td.addColumn (ScalarColumnDesc<float> ("Baseline"));
td.addColumn (ArrayColumnDesc<float> ("Pol", 1));
td.addColumn (ArrayColumnDesc<float> ("Freq", 1));
td.addColumn (ScalarColumnDesc<String> ("Id"));
td.addColumn (ArrayColumnDesc<float> ("Data", 2));
td.addColumn (ArrayColumnDesc<float> ("Weight", 2));
// Define the 4-dim hypercolumn with its data, coordinate and id columns.
td.defineHypercolumn ("TSMExample",
4,
stringToVector ("Data,Weight"),
stringToVector ("Pol,Freq,Baseline,Time"),
stringToVector ("Id"));
// Now create a new table from the description.
SetupNewTable newtab("tTiledDataStMan_tmp.data", td, Table::New);
// Create a TiledDataStMan storage manager for the hypercolumn
// and bind the columns to it.
TiledDataStMan sm1 ("TSMExample");
newtab.bindAll (sm1);
// Create the table with 42*30 rows.
Table table(newtab, 42*30);
// Create the accessor to be able to add a hypercube to this
// storage manager.
TiledDataStManAccessor accessor(table, "TSMExample");
// Define the values for the coordinates of the hypercube
// and put them into the record.
Vector<float> timeValues(42);
Vector<float> baselineValues(30);
Vector<float> freqValues(20);
Vector<float> polValues(12);
indgen (timeValues);
indgen (baselineValues, float(100));
indgen (freqValues, float(200));
indgen (polValues, float(300));
Record hyperDef;
hyperDef.define ("Time", timeValues);
hyperDef.define ("Baseline", baselineValues);
hyperDef.define ("Freq", freqValues);
hyperDef.define ("Pol", polValues);
// Define the id value as well.
hyperDef.define ("Id", "");
// Now add the hypercube with the given shape, tile shape,
// and coordinate and id values.
accessor.addHypercube (IPosition(4,12,20,30,42),
IPosition(4,4,5,6,7), hyperDef);
ArrayColumn<float> data (table, "Data");
ArrayColumn<float> weight (table, "Weight");
Matrix<float> array(IPosition(2,12,20));
uInt i;
indgen (array);
// Write some data into the data columns.
for (i=0; i<30*42; i++) {
data.put (i, array);
weight.put (i, array+float(100));
array += float(200);
}
// Prepare for reading the data back.
// Note that time and baseline are in fact scalar columns. They are
// superimposed dimensions on the hypercube.
ROScalarColumn<float> time (table, "Time");
ROScalarColumn<float> baseline (table, "Baseline");
ROArrayColumn<float> freq (table, "Freq");
ROArrayColumn<float> pol (table, "Pol");
ROScalarColumn<String> id (table, "Id");
float fValue;
String sValue;
for (i=0; i<table.nrow(); i++) {
data.get (i, array);
weight.get (i, array);
pol.get (i, polValues);
freq.get (i, freqValues);
baseline.get (i, fValue);
time.get (i, fValue);
id.get (i, sValue);
}
Note that in this example an id column was not necessary, because
there is only one hypercube.
The following example is more advanced. Two (extensible) hypercubes
are used for line and continuum data. Writing such a data set
could be done as shown. Reading it back is the same as above.
In this example the data columns contain line and continuum data.
So there are two types of data, each with their own shape and
stored in their own (extensible) hypercube. Note that the last
dimension of the hypercube shape is set to zero (to make extensible),
but the last tile shape dimension has been filled in,
because the exact tile shape must be known.
Before each put of the data the appropriate hypercube is extended.
Also the time has to be put, which is done (as an example) in
two different ways (using an explicit put and using the extendHypercube).
// Defining TableDesc and storage manager is same as in first example.
// Create the table.
Table table(newtab);
// Create the accessor to be able to add the hypercubes to this
// storage manager.
TiledDataStManAccessor accessor(table, "TSMExample");
// Fill the coordinate values.
// Note that the time axis of the hypercube will have length 0 to
// make it extensible. Therefore the time coordinate can only be
// filled in when the hypercube is extended.
Vector<float> baselineValues(30);
Vector<float> freqValuesCont(1);
Vector<float> freqValuesLine(20);
Vector<float> polValues(4);
indgen (baselineValues, float(100));
indgen (freqValuesLine, float(200));
indgen (freqValuesCont, float(200));
indgen (polValues, float(300));
Record hyperDefLine;
hyperDefLine.define ("Baseline", baselineValues);
hyperDefLine.define ("Pol", polValues);
// Make similar record for line data.
// Fill the correct id and frequency values for each type.
// Add the 2 hypercubes.
Record hyperDefCont (hyperDefLine);
hyperDefLine.define ("Id", "L");
hyperDefLine.define ("Freq", freqValuesLine);
hyperDefCont.define ("Id", "C");
hyperDefCont.define ("Freq", freqValuesCont);
// Add the hypercubes.
// Define their last dimension as zero to make them extensible.
accessor.addHypercube (IPosition(4,4,20,30,0),
IPosition(4,4,5,6,7), hyperDefLine);
accessor.addHypercube (IPosition(4,4,1,30,0),
IPosition(4,4,1,6,7), hyperDefCont);
ScalarColumn<float> time (table, "Time");
ScalarColumn<float> baseline (table, "Baseline");
ArrayColumn<float> freq (table, "Freq");
ArrayColumn<float> pol (table, "Pol");
ArrayColumn<float> data (table, "Data");
ArrayColumn<float> weight (table, "Weight");
Matrix<float> arrayLine(IPosition(2,4,20));
Matrix<float> arrayCont(IPosition(2,4,1));
indgen (arrayLine);
indgen (arrayCont);
// Write some data into the data columns.
// Alternately line and continuum is written.
// Each hypercube requires 30 rows to be added (i.e. nr of baselines).
// The last dimension of each hypercube is extended with 1.
uInt i, j;
uInt rownr = 0;
for (i=0; i<42; i++) {
if (i%2 == 0) {
table.addRow (30);
accessor.extendHypercube (1, hyperDefLine);
time.put (rownr, float(i));
for (j=0; j<30; j++) {
data.put (rownr, arrayLine);
weight.put (rownr, arrayLine);
rownr++;
}
}else{
table.addRow (30);
Vector<float> timeValue(1);
timeValue(0) = float(i);
hyperDefCont.define ("Time", timeValue);
accessor.extendHypercube (1, hyperDefCont);
time.put (rownr, float(i));
for (j=0; j<30; j++) {
data.put (rownr, arrayCont);
weight.put (rownr, arrayCont);
rownr++;
}
}
}
Note that in this example the time is defined in 2 different ways.
The first one by an explicit put, the second one as a record in
the extendHypercube call. The second way if the preferred one,
although it requires a bit more coding.
Clone this object. It does not clone TSMColumn objects possibly used.
Get the type name of the data manager (i.e. TiledDataStMan).
Make the object from the type name string. This function gets registered in the DataManager "constructor" map.
Forbid copy constructor.
Forbid assignment.
Add rows to the storage manager. This will only increase the number of rows. When a hypercube is added or extended, it will be checked whether the number of rows is sufficient.
Add a hypercube. The number of rows in the table must be large enough to accommodate this hypercube. The possible id values must be given in the record, while coordinate values are optional. The field names in the record should match the coordinate and id column names. The last dimension in the cube shape can be zero, indicating that the hypercube is extensible.
Extend the hypercube with the given number of elements in the last dimension. The record should contain the id values (to get the correct hypercube) and optionally coordinate values for the elements added.
Get the hypercube in which the given row is stored.
Get the hypercube in which the given row is stored. It also returns the position of the row in that hypercube.
Flush and optionally fsync the data. It returns a True status if it had to flush (i.e. if data have changed).
Let the storage manager create files as needed for a new table. This allows a column with an indirect array to create its file.
Read the header info.
Update the map of row numbers to cube number plus offset.
Check if the table is large enough to hold this hypercube extension.