casa  5.7.0-16
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
Public Member Functions | Static Public Member Functions | Protected Member Functions | Private Member Functions | Static Private Member Functions | Private Attributes | Static Private Attributes | Friends | List of all members
casacore::DataManager Class Referenceabstract

Abstract base class for a data manager. More...

#include <DataManager.h>

Inheritance diagram for casacore::DataManager:
casa::AsdmStMan casacore::Adios2StMan casacore::ISMBase casacore::MSMBase casacore::SSMBase casacore::StManAipsIO casacore::TiledStMan casacore::VirtualColumnEngine

Public Member Functions

 DataManager ()
 Default constructor. More...
 
virtual ~DataManager ()
 
virtual DataManagerclone () const =0
 Make a clone of the derived object. More...
 
virtual String dataManagerName () const
 Return the name of the data manager. More...
 
virtual String dataManagerType () const =0
 Return the type name of the data manager (in fact its class name). More...
 
void dataManagerInfo (Record &info) const
 Add SEQNR and SPEC (the DataManagerSpec subrecord) to the info. More...
 
virtual Record dataManagerSpec () const
 Return a record containing data manager specifications. More...
 
virtual Record getProperties () const
 Get data manager properties that can be modified. More...
 
virtual void setProperties (const Record &spec)
 Modify data manager properties given in record fields. More...
 
virtual Bool isStorageManager () const
 Is the data manager a storage manager? The default is yes. More...
 
virtual Bool canReallocateColumns () const
 Tell if the data manager wants to reallocate the data manager column objects. More...
 
virtual DataManagerColumnreallocateColumn (DataManagerColumn *column)
 Reallocate the column object if it is part of this data manager. More...
 
uInt sequenceNr () const
 Get the (unique) sequence nr of this data manager. More...
 
uInt ncolumn () const
 Get the nr of columns in this data manager (can be zero). More...
 
Bool asBigEndian () const
 Have the data to be stored in big or little endian canonical format? More...
 
const TSMOptiontsmOption () const
 Get the TSM option. More...
 
MultiFileBasemultiFile ()
 Get the MultiFile pointer (can be 0). More...
 
String keywordName (const String &keyword) const
 Compose a keyword name from the given keyword appended with the sequence number (e.g. More...
 
String fileName () const
 Compose a unique filename from the table name and sequence number. More...
 
ByteIO::OpenOption fileOption () const
 Get the AipsIO option of the underlying file. More...
 
virtual Bool isRegular () const
 Is this a regular storage manager? It is regular if it allows addition of rows and writing data in them. More...
 
Tabletable () const
 Get the table this object is associated with. More...
 
virtual void reopenRW ()
 Reopen the data manager for read/write access. More...
 
virtual Bool canAddRow () const
 Does the data manager allow to add rows? (default no) More...
 
virtual Bool canRemoveRow () const
 Does the data manager allow to delete rows? (default no) More...
 
virtual Bool canAddColumn () const
 Does the data manager allow to add columns? (default no) More...
 
virtual Bool canRemoveColumn () const
 Does the data manager allow to delete columns? (default no) More...
 
virtual Bool canRenameColumn () const
 Does the data manager allow to rename columns? (default yes) More...
 
virtual void setMaximumCacheSize (uInt nbytes)
 Set the maximum cache size (in bytes) to be used by a storage manager. More...
 
virtual void showCacheStatistics (std::ostream &) const
 Show the data manager's IO statistics. More...
 
DataManagerColumncreateScalarColumn (const String &columnName, int dataType, const String &dataTypeId)
 Create a column in the data manager on behalf of a table column. More...
 
DataManagerColumncreateDirArrColumn (const String &columnName, int dataType, const String &dataTypeId)
 Create a direct array column. More...
 
DataManagerColumncreateIndArrColumn (const String &columnName, int dataType, const String &dataTypeId)
 Create an indirect array column. More...
 
virtual void deleteManager ()=0
 The data manager will be deleted (because all its columns are requested to be deleted). More...
 
DataManagergetClone () const
 Has the object already been cloned? More...
 
void setClone (DataManager *clone) const
 Set the pointer to the clone. More...
 

Static Public Member Functions

static void registerCtor (const String &type, DataManagerCtor func)
 Register a mapping of a data manager type to its static construction function. More...
 
static DataManagerCtor getCtor (const String &dataManagerType)
 Get the "constructor" of a data manager (thread-safe). More...
 
static Bool isRegistered (const String &dataManagerType)
 Test if a data manager is registered (thread-safe). More...
 
static DataManagerunknownDataManager (const String &dataManagerType, const Record &spec)
 Serve as default function for theirRegisterMap, which catches all unknown data manager types. More...
 

Protected Member Functions

void decrementNcolumn ()
 Decrement number of columns (in case a column is deleted). More...
 
void setEndian (Bool bigEndian)
 Tell the data manager if big or little endian format is needed. More...
 
void setTsmOption (const TSMOption &tsmOption)
 Tell the data manager which TSM option to use. More...
 
void setMultiFile (MultiFileBase *mfile)
 Tell the data manager that MultiFile can be used. More...
 
virtual Bool hasMultiFileSupport () const
 Does the data manager support use of MultiFile? A derived class has to return True if it can use the MultiFile. More...
 
void throwDataTypeOther (const String &columnName, int dataType) const
 Throw an exception in case data type is TpOther, because the storage managers (and maybe other data managers) do not support such columns. More...
 

Private Member Functions

 DataManager (const DataManager &)
 The copy constructor cannot be used for this base class. More...
 
DataManageroperator= (const DataManager &)
 Assignment cannot be used for this base class. More...
 
virtual DataManagerColumnmakeScalarColumn (const String &columnName, int dataType, const String &dataTypeId)=0
 Create a column in the data manager on behalf of a table column. More...
 
virtual DataManagerColumnmakeDirArrColumn (const String &columnName, int dataType, const String &dataTypeId)=0
 Create a direct array column. More...
 
virtual DataManagerColumnmakeIndArrColumn (const String &columnName, int dataType, const String &dataTypeId)=0
 Create an indirect array column. More...
 
void checkDataType (const DataManagerColumn *colPtr, const String &columnName, int dataType, const String &dataTypeId) const
 Check if the data type of the created data manager column is correct. More...
 
virtual void addRow (uInt nrrow)
 Add rows to all columns. More...
 
virtual void removeRow (uInt rownr)
 Delete a row from all columns. More...
 
virtual void addColumn (DataManagerColumn *)
 Add a column. More...
 
virtual void removeColumn (DataManagerColumn *)
 Delete a column. More...
 
void setSeqnr (uInt nr)
 Set the sequence number of this data manager. More...
 
void linkToTable (Table &tab)
 Link the data manager to the Table object. More...
 
virtual Bool flush (AipsIO &ios, Bool fsync)=0
 Flush and optionally fsync the data. More...
 
virtual void create (uInt nrrow)=0
 Let the data manager initialize itself for a new table. More...
 
virtual void open (uInt nrrow, AipsIO &ios)=0
 Let the data manager initialize itself for an existing table. More...
 
virtual uInt open1 (uInt nrrow, AipsIO &ios)
 Open as above. More...
 
virtual void resync (uInt nrrow)=0
 Resync the data by rereading cached data from the file. More...
 
virtual uInt resync1 (uInt nrrow)
 Resync as above. More...
 
virtual void prepare ()
 Let the data manager initialize itself further. More...
 

Static Private Member Functions

static std::map< String,
DataManagerCtor > 
initRegisterMap ()
 Register the main data managers. More...
 

Private Attributes

uInt nrcol_p
 
uInt seqnr_p
 
Bool asBigEndian_p
 
TSMOption tsmOption_p
 
MultiFileBasemultiFile_p
 
Tabletable_p
 
DataManagerclone_p
 

Static Private Attributes

static std::map< String,
DataManagerCtor > 
theirRegisterMap
 Declare the mapping of the data manager type name to a static "makeObject" function. More...
 
static Mutex theirMutex
 

Friends

class SetupNewTable
 
class ColumnSet
 

Detailed Description

Abstract base class for a data manager.

Intended use:

Internal

Review Status

Reviewed By:
Gareth Hunt
Date Reviewed:
94Nov17

Synopsis

DataManager is the abstract base class for all kind of table data managers. There are currently 2 classes of data managers:

DataManager contains some common data and defines several virtual functions, which usually have to be implemented in the derived classes. It also contains some helper functions for the derived classes (like fileName().

The actual handling of a column by the data manager is defined in the abstract base class DataManagerColumn. Each data manager must have an associated class (derived from DataManagerColumn) to handle the columns.

There is a protocol defined how a data manager is created and initialized. For a new table it is:

For an existing table the procedure is slightly different:

Motivation

An abstract base class is needed to support data managers and virtual column engines in the table system in a transparant way.

To Do

Definition at line 224 of file DataManager.h.

Constructor & Destructor Documentation

casacore::DataManager::DataManager ( )

Default constructor.

virtual casacore::DataManager::~DataManager ( )
virtual
casacore::DataManager::DataManager ( const DataManager )
private

The copy constructor cannot be used for this base class.

The clone function should be used instead. The private declaration of this constructor makes it unusable.

Member Function Documentation

virtual void casacore::DataManager::addColumn ( DataManagerColumn )
privatevirtual

Add a column.

The default implementation throws a "not possible" exception.

Reimplemented in casacore::ForwardColumnEngine, casacore::StManAipsIO, casacore::SSMBase, casacore::ISMBase, casa::AsdmStMan, casacore::DerivedMSCal, and casacore::MSMBase.

virtual void casacore::DataManager::addRow ( uInt  nrrow)
privatevirtual
Bool casacore::DataManager::asBigEndian ( ) const
inline

Have the data to be stored in big or little endian canonical format?

Definition at line 293 of file DataManager.h.

References asBigEndian_p.

virtual Bool casacore::DataManager::canAddColumn ( ) const
virtual

Does the data manager allow to add columns? (default no)

Reimplemented in casacore::ForwardColumnEngine, casacore::StManAipsIO, casacore::SSMBase, casacore::ISMBase, casa::AsdmStMan, casacore::DerivedMSCal, and casacore::MSMBase.

virtual Bool casacore::DataManager::canAddRow ( ) const
virtual
virtual Bool casacore::DataManager::canReallocateColumns ( ) const
virtual

Tell if the data manager wants to reallocate the data manager column objects.

This is used by the tiling storage manager. By default it returns False.

Reimplemented in casacore::TiledStMan, and casacore::MSMBase.

virtual Bool casacore::DataManager::canRemoveColumn ( ) const
virtual

Does the data manager allow to delete columns? (default no)

Reimplemented in casacore::ForwardColumnEngine, casacore::StManAipsIO, casacore::SSMBase, casacore::ISMBase, casa::AsdmStMan, casacore::DerivedMSCal, and casacore::MSMBase.

virtual Bool casacore::DataManager::canRemoveRow ( ) const
virtual
virtual Bool casacore::DataManager::canRenameColumn ( ) const
virtual

Does the data manager allow to rename columns? (default yes)

void casacore::DataManager::checkDataType ( const DataManagerColumn colPtr,
const String columnName,
int  dataType,
const String dataTypeId 
) const
private

Check if the data type of the created data manager column is correct.

virtual DataManager* casacore::DataManager::clone ( ) const
pure virtual
virtual void casacore::DataManager::create ( uInt  nrrow)
privatepure virtual
DataManagerColumn* casacore::DataManager::createDirArrColumn ( const String columnName,
int  dataType,
const String dataTypeId 
)

Create a direct array column.

DataManagerColumn* casacore::DataManager::createIndArrColumn ( const String columnName,
int  dataType,
const String dataTypeId 
)

Create an indirect array column.

DataManagerColumn* casacore::DataManager::createScalarColumn ( const String columnName,
int  dataType,
const String dataTypeId 
)

Create a column in the data manager on behalf of a table column.

It calls makeXColumn and checks the data type.

Create a scalar column. The dataTypeId argument is gives the id (i.e. name) of the data type of the column. It is only used for virtual columns of a non-standard data type to be able to check if the correctness of the column data type.
Storage managers only handle standard data types and can readily ignore this argument.

void casacore::DataManager::dataManagerInfo ( Record info) const

Add SEQNR and SPEC (the DataManagerSpec subrecord) to the info.

virtual String casacore::DataManager::dataManagerName ( ) const
virtual
virtual Record casacore::DataManager::dataManagerSpec ( ) const
virtual
virtual String casacore::DataManager::dataManagerType ( ) const
pure virtual
void casacore::DataManager::decrementNcolumn ( )
inlineprotected

Decrement number of columns (in case a column is deleted).

Definition at line 383 of file DataManager.h.

References nrcol_p.

virtual void casacore::DataManager::deleteManager ( )
pure virtual

The data manager will be deleted (because all its columns are requested to be deleted).

So clean up the things needed (e.g. delete files).

Implemented in casacore::StManAipsIO, casacore::TiledStMan, casacore::SSMBase, casacore::ISMBase, casa::AsdmStMan, casacore::VirtualColumnEngine, casacore::MSMBase, and casacore::Adios2StMan.

String casacore::DataManager::fileName ( ) const

Compose a unique filename from the table name and sequence number.

Referenced by casacore::Adios2StMan::impl::fileName().

ByteIO::OpenOption casacore::DataManager::fileOption ( ) const

Get the AipsIO option of the underlying file.

virtual Bool casacore::DataManager::flush ( AipsIO ios,
Bool  fsync 
)
privatepure virtual

Flush and optionally fsync the data.

The AipsIO stream represents the main table file and can be used by virtual column engines to store SMALL amounts of data. It returns a True status if it had to flush (i.e. if data have changed).

Implemented in casacore::StManAipsIO, casacore::TiledDataStMan, casacore::SSMBase, casacore::TiledShapeStMan, casacore::ISMBase, casacore::TiledCellStMan, casacore::TiledColumnStMan, casa::AsdmStMan, casacore::VirtualColumnEngine, casacore::MSMBase, casacore::TiledFileHelper, and casacore::Adios2StMan.

DataManager* casacore::DataManager::getClone ( ) const
inline

Has the object already been cloned?

Definition at line 524 of file DataManager.h.

References clone_p.

static DataManagerCtor casacore::DataManager::getCtor ( const String dataManagerType)
static

Get the "constructor" of a data manager (thread-safe).

virtual Record casacore::DataManager::getProperties ( ) const
virtual

Get data manager properties that can be modified.

It is a subset of the data manager specification. The default implementation returns an empty record.

Reimplemented in casacore::BitFlagsEngine< StoredType >, casacore::SSMBase, casacore::TiledStMan, and casacore::ISMBase.

virtual Bool casacore::DataManager::hasMultiFileSupport ( ) const
protectedvirtual

Does the data manager support use of MultiFile? A derived class has to return True if it can use the MultiFile.

The default implementation returns False.

Reimplemented in casacore::SSMBase, casacore::ISMBase, and casacore::TiledStMan.

static std::map<String,DataManagerCtor> casacore::DataManager::initRegisterMap ( )
staticprivate

Register the main data managers.

static Bool casacore::DataManager::isRegistered ( const String dataManagerType)
static

Test if a data manager is registered (thread-safe).

virtual Bool casacore::DataManager::isRegular ( ) const
virtual

Is this a regular storage manager? It is regular if it allows addition of rows and writing data in them.


The default implementation returns True.

Reimplemented in casa::AsdmStMan.

virtual Bool casacore::DataManager::isStorageManager ( ) const
virtual

Is the data manager a storage manager? The default is yes.

Reimplemented in casacore::VirtualColumnEngine.

String casacore::DataManager::keywordName ( const String keyword) const

Compose a keyword name from the given keyword appended with the sequence number (e.g.

key_0). This makes the keyword name unique if multiple data managers are used with the same type.

void casacore::DataManager::linkToTable ( Table tab)
private

Link the data manager to the Table object.

virtual DataManagerColumn* casacore::DataManager::makeDirArrColumn ( const String columnName,
int  dataType,
const String dataTypeId 
)
privatepure virtual
virtual DataManagerColumn* casacore::DataManager::makeIndArrColumn ( const String columnName,
int  dataType,
const String dataTypeId 
)
privatepure virtual
virtual DataManagerColumn* casacore::DataManager::makeScalarColumn ( const String columnName,
int  dataType,
const String dataTypeId 
)
privatepure virtual
MultiFileBase* casacore::DataManager::multiFile ( )
inline

Get the MultiFile pointer (can be 0).

Definition at line 301 of file DataManager.h.

References multiFile_p.

uInt casacore::DataManager::ncolumn ( ) const
inline

Get the nr of columns in this data manager (can be zero).

Definition at line 289 of file DataManager.h.

References nrcol_p.

Referenced by casacore::Adios2StMan::impl::ncolumn().

virtual void casacore::DataManager::open ( uInt  nrrow,
AipsIO ios 
)
privatepure virtual

Let the data manager initialize itself for an existing table.

The AipsIO stream represents the main table file and must be used by virtual column engines to retrieve the data stored in the flush function.

Implemented in casacore::StManAipsIO, casacore::TiledStMan, casacore::SSMBase, casacore::ISMBase, casa::AsdmStMan, casacore::VirtualColumnEngine, casacore::MSMBase, and casacore::Adios2StMan.

virtual uInt casacore::DataManager::open1 ( uInt  nrrow,
AipsIO ios 
)
privatevirtual

Open as above.

The data manager can return the number of rows it thinks there are. This is particularly useful for data managers like LofarStMan whose data are written outside the table system, thus for which no rows have been added.
By default it calls open and returns nrrow.

DataManager& casacore::DataManager::operator= ( const DataManager )
private

Assignment cannot be used for this base class.

The private declaration of this operator makes it unusable.

virtual void casacore::DataManager::prepare ( )
privatevirtual
virtual DataManagerColumn* casacore::DataManager::reallocateColumn ( DataManagerColumn column)
virtual

Reallocate the column object if it is part of this data manager.

It returns a pointer to the new column object. This function is used by the tiling storage manager. By default it does nothing and returns the input pointer.

Reimplemented in casacore::TiledStMan, and casacore::MSMBase.

static void casacore::DataManager::registerCtor ( const String type,
DataManagerCtor  func 
)
static

Register a mapping of a data manager type to its static construction function.

It is fully thread-safe.

virtual void casacore::DataManager::removeColumn ( DataManagerColumn )
privatevirtual

Delete a column.

The default implementation throws a "not possible" exception.

Reimplemented in casacore::ForwardColumnEngine, casacore::StManAipsIO, casacore::SSMBase, casacore::ISMBase, casa::AsdmStMan, casacore::DerivedMSCal, and casacore::MSMBase.

virtual void casacore::DataManager::removeRow ( uInt  rownr)
privatevirtual

Delete a row from all columns.

The default implementation throws a "not possible" exception.

Reimplemented in casacore::ForwardColumnEngine, casacore::StManAipsIO, casacore::SSMBase, casacore::ISMBase, casa::AsdmStMan, casacore::MSMBase, and casacore::VirtualColumnEngine.

virtual void casacore::DataManager::reopenRW ( )
virtual

Reopen the data manager for read/write access.

By default it is assumed that a reopen for read/write does not have to do anything.

Reimplemented in casacore::ForwardColumnEngine, casacore::StManAipsIO, casacore::ForwardColumnIndexedRowEngine, casacore::TiledStMan, casacore::SSMBase, casacore::ISMBase, casa::AsdmStMan, casacore::CompressComplex, and casacore::CompressFloat.

virtual void casacore::DataManager::resync ( uInt  nrrow)
privatepure virtual

Resync the data by rereading cached data from the file.

This is called when a lock is acquired on the file and it appears that data in this data manager has been changed by another process.

Implemented in casacore::StManAipsIO, casacore::TiledStMan, casacore::SSMBase, casacore::ISMBase, casa::AsdmStMan, casacore::VirtualColumnEngine, casacore::MSMBase, and casacore::Adios2StMan.

virtual uInt casacore::DataManager::resync1 ( uInt  nrrow)
privatevirtual

Resync as above.

The data manager can return the number of rows it thinks there are. This is particularly useful for data managers like LofarStMan whose data are written outside the table system, thus for which no rows have been added.
By default it calls resync and returns nrrow.

uInt casacore::DataManager::sequenceNr ( ) const
inline

Get the (unique) sequence nr of this data manager.

Definition at line 285 of file DataManager.h.

References seqnr_p.

Referenced by casacore::RODataManAccessor::dataManagerSeqNr().

void casacore::DataManager::setClone ( DataManager clone) const
inline

Set the pointer to the clone.

Definition at line 528 of file DataManager.h.

References clone(), and clone_p.

void casacore::DataManager::setEndian ( Bool  bigEndian)
inlineprotected

Tell the data manager if big or little endian format is needed.

Definition at line 387 of file DataManager.h.

References asBigEndian_p.

virtual void casacore::DataManager::setMaximumCacheSize ( uInt  nbytes)
virtual

Set the maximum cache size (in bytes) to be used by a storage manager.

The default implementation does nothing.

Reimplemented in casacore::TiledStMan.

void casacore::DataManager::setMultiFile ( MultiFileBase mfile)
protected

Tell the data manager that MultiFile can be used.

Because MultiFile cannot be used with mmapped files, it sets the TSMOption accordingly.

virtual void casacore::DataManager::setProperties ( const Record spec)
virtual

Modify data manager properties given in record fields.

Only the properties as returned by getProperties are used, others are ignored. The default implementation does nothing.

Reimplemented in casacore::BitFlagsEngine< StoredType >, casacore::SSMBase, casacore::TiledStMan, and casacore::ISMBase.

void casacore::DataManager::setSeqnr ( uInt  nr)
inlineprivate

Set the sequence number of this data manager.

Definition at line 468 of file DataManager.h.

References seqnr_p.

void casacore::DataManager::setTsmOption ( const TSMOption tsmOption)
protected

Tell the data manager which TSM option to use.

virtual void casacore::DataManager::showCacheStatistics ( std::ostream &  ) const
virtual

Show the data manager's IO statistics.

By default it does nothing.

Referenced by casacore::RODataManAccessor::showCacheStatistics().

Table& casacore::DataManager::table ( ) const
inline

Get the table this object is associated with.

Definition at line 322 of file DataManager.h.

References table_p.

void casacore::DataManager::throwDataTypeOther ( const String columnName,
int  dataType 
) const
protected

Throw an exception in case data type is TpOther, because the storage managers (and maybe other data managers) do not support such columns.

const TSMOption& casacore::DataManager::tsmOption ( ) const
inline

Get the TSM option.

Definition at line 297 of file DataManager.h.

References tsmOption_p.

static DataManager* casacore::DataManager::unknownDataManager ( const String dataManagerType,
const Record spec 
)
static

Serve as default function for theirRegisterMap, which catches all unknown data manager types.

Thrown Exceptions

  • TableUnknownDataManager

Friends And Related Function Documentation

friend class ColumnSet
friend

Definition at line 227 of file DataManager.h.

friend class SetupNewTable
friend

Definition at line 226 of file DataManager.h.

Member Data Documentation

Bool casacore::DataManager::asBigEndian_p
private

Definition at line 412 of file DataManager.h.

Referenced by asBigEndian(), and setEndian().

DataManager* casacore::DataManager::clone_p
mutableprivate

Definition at line 416 of file DataManager.h.

Referenced by getClone(), and setClone().

MultiFileBase* casacore::DataManager::multiFile_p
private

Definition at line 414 of file DataManager.h.

Referenced by multiFile().

uInt casacore::DataManager::nrcol_p
private

Definition at line 410 of file DataManager.h.

Referenced by decrementNcolumn(), and ncolumn().

uInt casacore::DataManager::seqnr_p
private

Definition at line 411 of file DataManager.h.

Referenced by sequenceNr(), and setSeqnr().

Table* casacore::DataManager::table_p
private

Definition at line 415 of file DataManager.h.

Referenced by table().

Mutex casacore::DataManager::theirMutex
staticprivate

Definition at line 520 of file DataManager.h.

std::map<String,DataManagerCtor> casacore::DataManager::theirRegisterMap
staticprivate

Declare the mapping of the data manager type name to a static "makeObject" function.

Definition at line 519 of file DataManager.h.

TSMOption casacore::DataManager::tsmOption_p
private

Definition at line 413 of file DataManager.h.

Referenced by tsmOption().


The documentation for this class was generated from the following file: