Because the variables have to be generic and because of
limitations in the CFront compiler, several variables have to be
passed as void* and need to be casted in the set/get functions.
The mapping process has to be done by a (static) set and get function in the VirtualType class. When a RetypedArrayEngine object is constructed, it is possible to pass information in a TableRecord. This TableRecord is indirectly passed to the set/get functions. This is done by means of the function newCopyInfo, which can preprocess the information in the TableRecord and store it in another object. That object is passed to the set and get functions. At the end a function deleteCopyInfo is called to delete the object. Of course, it is not needed to allocate such an object; newCopyInfo can return a null pointer.
The virtual column data type class has to contain several functions. The example shows how they can be implemented.
void setElem (const StoredType* data, const IPosition& shape,
const void* extraArgument);
for each VirtualType element. This set function has to
fill the VirtualType object from the data. It can use the
shape and the extraArgument to know how it should do it.
Note that the given virtual element shape does not need to match the shape given to the constructor of the engine. It is possible that the user sets the shape of the stored array before putting the virtual array. In that case the system uses the relevant part of the stored array shape as the virtual element shape.
. Therefore it has to be declared as void* and the set function needs to cast it to Array<VirtualType>*.
E.g.: A StokesVector has 4 float elements.
// Construct the column object for the Stokes column.
ArrayColumn<StokesVector> stokesColumn (table, "StokesVirtualColumn");
// Put an array of StokesVector's with shape 512,512.
// This will implicitly set the shape of the underlying
// data column to 4,512,512.
// This put is very quick (it can copy all data in one go).
Array<StokesVector> stokesData (IPosition(2,512,512));
stokesColumn.put (rownr, stokesData);
// Get the column object for the Data column.
// Set its shape explicitly to 1,512,512,
ArrayColumn<float> dataColumn (table, "DataColumn");
dataColumn.setShape (rownr, IPosition(3,1,512,512));
// Now a put of the data results in calling the StokesVector::getElem
// function for each element with an IPosition(1,1); i.e. the
// data array needs only one value for each StokesVector.
stokesColumn.put (rownr, stokesData);
When reading a table back, the engine has to be registered. Otherwise it will be unknown to the table system. Similarly, the appropriate ArrayColumnDesc object has to be registered. This can be done as follows:
RetypedArrayEngine::registerClass();
ArrayColumnDesc tmp(ColumnDesc::registerMap);
When they are not registered, the open of the table will fail
telling which class could not be found.
The class had to be doubly templated. There were 2 reasons:
Originally it was the idea to have a mandatory nested CopyInfo class in the VirtualType class and use syntax like VirtualType::CopyInfo to access functions in it and to keep a pointer to such an object. Alas, the CFront compiler could not handle this.
Because the engine can serve only one column, it was possible to combine the engine and the column functionality in one class. This has been achieved using multiple inheritance. The advantage of this is that only one templated class is used, so fewer template instantiations are needed.
//# Forward Declarations
template<class T> class Array;
template<class T> class Vector;
class StokesVector
{
public:
StokesVector(): I_p(0), Q_p(0), U_p(0), V_p(0) {}
StokesVector(double i, double q, double u, double v)
: I_p(i), Q_p(q), U_p(u), V_p(v) {}
StokesVector(const StokesVector& that): I_p(that.I_p), Q_p(that.Q_p),
U_p(that.U_p), V_p(that.V_p) {}
StokesVector& operator= (const StokesVector& that)
{ I_p=that.I_p; Q_p=that.Q_p; U_p=that.U_p; V_p=that.V_p;
return *this; }
static String dataTypeId()
{ return "StokesVector"; }
// A StokesVector is 1-dim and contains 4 elements.
static IPosition shape()
{ return IPosition (1,4); }
// Preprocess possible information in the TableRecord.
static void* newCopyInfo (const TableRecord& record,
const IPosition& shape)
{ return new CopyInfo(record, shape); }
// Delete the object containing preprocessed information.
static void* deleteSetDet (void* copyInfo)
{ delete (CopyInfo*)copyInfo; }
// Convert a StoredType array to a VirtualType array.
// Do this in a CopyInfo function to use its preprocessed information.
static void set (void* copyInfo, void* out,
const Array<double>& in, const IPosition& shape)
{ ((CopyInfo*)copyInfo)->set (out, in, shape); }
// Convert a VirtualType array to a StoredType array.
// Do this in a CopyInfo function to use its preprocessed information.
static void get (void* copyInfo, Array<double>& out,
const void* in, const IPosition& shape)
{ ((CopyInfo*)copyInfo)->get (out, in, shape); }
// This nested class is used to hold preprocessed information. It
// holds a mask extracted from the TableRecord supplied to the engine.
// One can imagine that it could also extract a flag telling
// whether the stored data is stored as I,Q,U,V or as XX,YY,XY,YX
// (although such a conversion would probably be better handled
// by a separate virtual column engine).
class CopyInfo {
public:
// The constructor extracts the mask from the record.
void CopyInfo (const TableRecord& record)
{
RORecordFieldRef<Array<Bool> > field (record, 0);
mask_p = new Vector<Bool>;
*mask_p = *field;
}
// The set function fills the StokesVector.
// It uses the general functions for that purpose.
void set (void* vout, const Array<double>& in,
const IPosition& shape)
{
Array<StokesVector>& out = *(Array<StokesVector>*)vout;
if (shape.nelements() == 1 && shape(0) == 4) {
// All values available, copy in one go.
// This can be done because a StokesVector object
// only contains 4 double values (and no virtual
// function table).
retypedArrayEngineSet (out, in);
}else{
// Only some values available. Fill each
// StokesVector object using the shape and mask.
// The set function below is called for each object.
retypedArrayEngineSet (out, in, shape, (void*)mask_p);
}
}
// get is the opposite of set.
void get (Array<double>& out, const void* vin,
const IPosition& shape)
{
const Array<StokesVector>& in =
*(const Array<StokesVector>*)vin;
if (shape.nelements() == 1 && shape(0) == 4) {
retypedArrayEngineGet (out, in);
}else{
retypedArrayEngineGet (out, in, shape, (void*)mask_p);
}
private:
Vector<Bool>* mask_p;
};
// Set values of StokesVector using the mask.
// The shape is not used here.
void setElem (const double* data, const IPosition&, const void* maskPtr)
{
const Vector<Bool>& mask = *(const Vector<Bool>*)maskPtr;
I_p = Q_p = U_p = V_p = 0;
if (mask(0)) {
I_p = *data++;
}
if (mask(1)) {
Q_p = *data++;
}
if (mask(2)) {
U_p = *data++;
}
if (mask(3)) {
V_p = *data;
}
}
// Get values of StokesVector using the mask (opposite of setElem).
void getElem (double* data, const IPosition&, const void* maskPtr);
private:
double I_p, Q_p, U_p, V_p;
};
main() {
// First register the virtual column engine.
RetypedArrayEngine<StokesVector,double>::registerClass();
// Add ArrayColumnDesc<StokesVector> to column type map.
ArrayColumnDesc<StokesVector> tmp(ColumnDesc::registerMap);
// Build the table description.
TableDesc td("", "1", TableDesc::Scratch);
td.addColumn (ArrayColumnDesc<double> ("Data"));
td.addColumn (ArrayColumnDesc<StokesVector> ("Stokes"));
// Now create a new table from the description.
SetupNewTable newtab("tRetypedArrayEngine_tmp.data", td, Table::New);
// Create the virtual column engine with the stored columns Data.
RetypedArrayEngine<StokesVector,double> engine ("Stokes", "Data");
newtab.bindColumn ("Stokes", engine);
Table tab(newtab, 50);
// Fill the table via the virtual columns.
ArrayColumn<StokesVector> stokesColumn (tab, "Stokes");
Vector<StokesVector> vec(10);
uInt i;
for (i=0; i<tab.nrow(); i++) {
stokesColumn.put (i, vec);
}
}
Due to instantiation problems with the CFront-based ObjectCenter compiler (and probably other CFront-based compilers as well) the Array and Vector have to be forward declared. Array.h and Vector.h should NOT be included in this StokesVector.h, thus the implementations should not be inlined (they are too large anyway), but put in a separate .cc file where Array.h and Vector.h can be included.
Another compiler problem is that the variable mask_p is not automatically converted to a void*, so an explicit cast has to be done.
Construct an engine to map a virtual column containing arrays with an arbitrary data type to arrays in a stored column. StoredColumnName is the name of the column where the converted data will be put and must have data type StoredType. The virtual column using this engine must have data type VirtualType. The shape and record provided is handed to the newCopyInfo function in the VirtualType class. It can be used to determine how an element has to be handled when the stored data is incomplete.
Construct from a record specification as created by getmanagerSpec().
Destructor is mandatory.
Return the type name of the engine (i.e. its class name).
Get the name given to the engine (is the virtual column name).
Record a record containing data manager specifications.
Return the name of the class. This includes the names of the template arguments.
Register the class name and the static makeObject "constructor".
This will make the engine known to the table system.
The automatically invoked registration function in DataManReg.cc
contains RetypedArrayEngine
Assignment is not needed and therefore forbidden
(so it is made private and not implemented).
Clone the engine object.
Initialize the object for a new table.
It defines the keywords containing the engine parameters.
Preparing consists of setting the writable switch and
adding the initial number of rows in case of create.
Furthermore it reads the keywords containing the engine parameters
and allocates a CopyInfo object for the VirtualType.
Set the shape of the FixedShape arrays in the column.
This function only gets called if the column has FixedShape arrays.
The shape gets saved and used to set the shape of the arrays
in the stored in case the stored has non-FixedShape arrays.
Define the shape of the array in the given row.
When the shape of the (underlying) stored array has already been
defined, it checks whether its latter dimensions match the given
virtual shape. When matching, nothing will be done.
When mismatching or when the stored shape has not been defined
yet, the stored shape will be defined from the virtual shape and
the virtual element shape.
E.g. in case of a StokesVector a virtual shape of (512,512)
results in a stored shape of (4,512,512).
Get the dimensionality of the array in the given row.
Get the shape of the array in the given row.
This is done by stripping the first dimension(s) from the shape
of the underlying stored array.
Get an array in the given row.
Put an array in the given row.
Get a section of the array in the given row.
Put into a section of the array in the given row.
Get an entire column.
Put an entire column.
Get a section of all arrays in the column.
Put a section of all arrays in the column.
Check if the shapes of virtual and stored match.
Determine the shape of the virtual elements in the stored.
Copy the stored array to the virtual array.
It tries to optimize as much as possible.
Copy the virtual array to the stored array.
It tries to optimize as much as possible.
Determine the shape of a cell in the stored column from the
shape of the cell in the virtual column.
Convert the Slicer for a virtual to a Slicer for the stored.
RetypedArrayEngine (const RetypedArrayEngine<VirtualType,StoredType>&)
Copy constructor is only used by clone().
(so it is made private).
RetypedArrayEngine<VirtualType,StoredType>& operator= (const RetypedArrayEngine<VirtualType,StoredType>&)
DataManager* clone() const
void create (uInt initialNrrow)
void prepare()
void setShapeColumn (const IPosition& shape)
void setShape (uInt rownr, const IPosition& shape)
uInt ndim (uInt rownr)
IPosition shape (uInt rownr)
void getArray (uInt rownr, Array<VirtualType>& array)
void putArray (uInt rownr, const Array<VirtualType>& array)
void getSlice (uInt rownr, const Slicer& slicer, Array<VirtualType>& array)
void putSlice (uInt rownr, const Slicer& slicer, const Array<VirtualType>& array)
void getArrayColumn (Array<VirtualType>& array)
void putArrayColumn (const Array<VirtualType>& array)
void getColumnSlice (const Slicer& slicer, Array<VirtualType>& array)
void putColumnSlice (const Slicer& slicer, const Array<VirtualType>& array)
IPosition checkShape (const Array<VirtualType>& source, const Array<StoredType>& target)
void copyOnGet (Array<VirtualType>& array, const Array<StoredType>& stored)
void copyOnPut (const Array<VirtualType>& array, Array<StoredType>& stored)
IPosition storedShape (uInt rownr, const IPosition& virtualShape)
Slicer storedSlicer (const Slicer& virtualSlicer) const
static DataManager* makeObject (const String& dataManagerType, const Record& spec)
Define the "constructor" to construct this engine when a
table is read back.
This "constructor" has to be registered by the user of the engine.
If the engine is commonly used, its registration can be added
to the registerAllCtor function in DataManReg.cc.
That function gets automatically invoked by the table system.