casa::SSMBase Class Reference
[Tables_internal_classes]

#include <SSMBase.h>

Inheritance diagram for casa::SSMBase:

[legend]Collaboration diagram for casa::SSMBase:

[legend]List of all members.

---

Detailed Description

Base class of the Standard Storage Manager.

Intended use:

Internal

Review Status

Reviewed By:

UNKNOWN

Date Reviewed:

before2004/08/25

Test programs:

tStandardStMan

Prerequisite

• StandardStMan
• SSMColumn

Etymology

SSMBase is the base class of the Standard Storage Manager.

Synopsis

The global principles of this class are described in StandardStMan .

The Standard Storage Manager divides the data file in equally sized chunks called buckets. There are 3 types of buckets:

• Data buckets containing the fixed length data (scalars and direct arrays of data type Int, Float, Bool, etc.). For variable shaped data (strings and indirect arrays) they contain references to the actual data position in the string buckets or in an external file.
• String buckets containing strings and array of strings.
• Index buckets containing the index info for the data buckets.

Bucket access is handled by class BucketCache . It also keeps a list of free buckets. A bucket is freed when it is not needed anymore (e.g. all data from it are deleted).

Data buckets form the main part of the SSM. The data can be viewed as a few streams of buckets, where each stream contains the data of a given number of columns. Each stream has an SSMIndex object describing the number of rows stored in each data bucket of the stream. The SSM starts with a single bucket stream (holding all columns), but when columns are added, new bucket streams might be created.

For example, we have an SSM with a bucket size of 100 bytes. There are 5 Int columns (A,B,C,D,E) each taking 4 bytes per row. Column A, B, C, and D are stored in bucket stream 1, while column E is stored in bucket stream 2. So in stream 1 each bucket can hold 6 rows, while in stream 2 each bucket can hold 25 rows. For a 100 row table it will result in 17+4 data buckets.

A few classes collaborate to make it work:

• Each bucket stream has an SSMIndex object to map row number to bucket number. Note that in principle each bucket in a stream contains the same number of rows. However, when a row is deleted it is removed from its bucket shifting the remainder to the left. Data in the next buckets is not shifted, so that bucket has now one row less.
• For each column SSMBase knows to which bucket stream it belongs and at which offset the column starts in a bucket. Note that column data in a bucket are adjacent, which is done to make it easier to use the ColumnCache object in SSMColumn and to be able to efficiently store Bool values as bits.
• Each column has an SSMColumn object knowing how many bits each data cell takes in a bucket. The SSMColumn objects handle all access to data in the columns (using SSMBase and SSMIndex).

String buckets are used by class SSMStringHandler to store scalar strings and fixed and variable shaped arrays of strings. The bucketnr, offset, and length of such string (arrays) are stored in the data buckets.
Indirect arrays of other data types are also stored indirectly and their offset is stored in the data buckets. Such arrays are handled by class StIndArray which uses an extra file to store the arrays.

Index buckets are used by SSMBase to make the SSMIndex data persistent. It uses alternately 2 sets of index buckets. In that way there is always an index availanle in case the system crashes. If possible 2 halfs of a single bucket are used alternately, otherwise separate buckets are used.

Motivation

The public interface of SSMBase is quite large, because the other internal SSM classes need these functions. To have a class with a minimal interface for the normal user, class StandardStMan is derived from it.
StandardStMan needs an isA- instead of hasA-relation to be able to bind columns to it in class SetupNewTable .

To Do

• Remove AipsIO argument from open and close.
• When only 1 bucket in use addcolumn can check if there's enough room to fit the new column (so rearange the bucket) in the free row space.

Definition at line 158 of file SSMBase.h.

Public Member Functions
	SSMBase (Int aBucketSize=0, uInt aCacheSize=1)
	Create a Standard storage manager with default name SSM.
	SSMBase (const String &aDataManName, Int aBucketSize=0, uInt aCacheSize=1)
	Create a Standard storage manager with the given name.
	SSMBase (const String &aDataManName, const Record &spec)
	Create a Standard storage manager with the given name.
	~SSMBase ()
virtual DataManager *	clone () const
	Clone this object.
virtual String	dataManagerType () const
	Get the type name of the data manager (i.e.
virtual String	dataManagerName () const
	Get the name given to the storage manager (in the constructor).
virtual Record	dataManagerSpec () const
	Record a record containing data manager specifications.
uInt	getVersion () const
	Get the version of the class.
void	setCacheSize (uInt aCacheSize)
	Set the cache size (in buckets).
uInt	getCacheSize () const
	Get the current cache size (in buckets).
void	clearCache ()
	Clear the cache used by this storage manager.
void	showCacheStatistics (ostream &anOs) const
	Show the statistics of all caches used.
void	showIndexStatistics (ostream &anOs) const
	Show Statistics of all indices used.
void	showBaseStatistics (ostream &anOs) const
	Show Statistics of the Base offsets/index etc.
uInt	getBucketSize () const
	Get the bucket size.
uInt	getNRow () const
	Get the number of rows in this storage manager.
virtual Bool	canAddRow () const
	The storage manager can add rows.
virtual Bool	canRemoveRow () const
	The storage manager can delete rows.
virtual Bool	canAddColumn () const
	The storage manager can add columns.
virtual Bool	canRemoveColumn () const
	The storage manager can delete columns.
SSMColumn &	getColumn (uInt aColNr)
	Get access to the given column.
SSMIndex &	getIndex (uInt anIdxNr)
	Get access to the given Index.
void	setBucketDirty ()
	Make the current bucket in the cache dirty (i.e.
StManArrayFile *	openArrayFile (ByteIO::OpenOption anOpt)
	Open (if needed) the file for indirect arrays with the given mode.
char *	find (uInt aRowNr, uInt aColNr, uInt &aStartRow, uInt &anEndRow)
	Find the bucket containing the column and row and return the pointer to the beginning of the column data in that bucket.
uInt	getNewBucket ()
	Add a new bucket and get its bucket number.
char *	getBucket (uInt aBucketNr)
	Read the bucket (if needed) and return the pointer to it.
void	removeBucket (uInt aBucketNr)
	Remove a bucket from the bucket cache.
uInt	getRowsPerBucket (uInt aColumn) const
	Get rows per bucket for the given column.
SSMStringHandler *	getStringHandler ()
	Return a pointer to the (one and only) StringHandler object.
Static Public Member Functions
static DataManager *	makeObject (const String &aDataManType, const Record &spec)
	Make the object from the type name string.
static char *	readCallBack (void *anOwner, const char *aBucketStorage)
	Callbacks for BucketCache access.
static void	writeCallBack (void *anOwner, char *aBucketStorage, const char *aBucket)
static void	deleteCallBack (void *, char *aBucket)
static char *	initCallBack (void *anOwner)
Private Member Functions
	SSMBase (const SSMBase &that)
	Copy constructor (only meant for clone function).
SSMBase &	operator= (const SSMBase &that)
	Assignment cannot be used.
void	recreate ()
	(Re)create the index, file, and cache object.
virtual Bool	flush (AipsIO &, Bool doFsync)
	Flush and optionally fsync the data.
virtual void	create (uInt aNrRows)
	Let the storage manager create files as needed for a new table.
virtual void	open (uInt aRowNr, AipsIO &)
	Open the storage manager file for an existing table, read in the data, and let the SSMColumn objects read their data.
virtual void	resync (uInt aRowNr)
	Resync the storage manager with the new file contents.
virtual void	reopenRW ()
	Reopen the storage manager files for read/write.
virtual void	deleteManager ()
	The data manager will be deleted (because all its columns are requested to be deleted).
void	init ()
	Let the storage manager initialize itself (upon creation).
uInt	setBucketSize ()
	Determine and set the bucket size.
uInt	getNrIndices () const
	Get the number of indices in use.
virtual void	addRow (uInt aNrRows)
	Add rows to the storage manager.
virtual void	removeRow (uInt aRowNr)
	Delete a row from all columns.
virtual void	addColumn (DataManagerColumn *)
	Do the final addition of a column.
virtual void	removeColumn (DataManagerColumn *)
	Remove a column from the data file.
BucketCache &	getCache ()
	Get the cache object.
void	makeCache ()
	Construct the cache object (if not constructed yet).
void	readHeader ()
	Read the header.
void	readIndexBuckets ()
	Read the index from its buckets.
void	writeIndex ()
	Write the header and the indices.
virtual DataManagerColumn *	makeScalarColumn (const String &aName, int aDataType, const String &aDataTypeID)
	Create a scalar column.
virtual DataManagerColumn *	makeDirArrColumn (const String &aName, int aDataType, const String &aDataTypeID)
	Create a direct array column.
virtual DataManagerColumn *	makeIndArrColumn (const String &aName, int aDataType, const String &aDataTypeID)
	Create an indirect array column.
Private Attributes
String	itsDataManName
	Name of data manager.
StManArrayFile *	itsIosFile
	The file containing the indirect arrays.
uInt	itsNrRows
	The number of rows in the columns.
Block< uInt >	itsColumnOffset
	Column offset.
Block< uInt >	itsColIndexMap
	Row Index ID containing all the columns in a bucket.
PtrBlock< SSMIndex * >	itsPtrIndex
	Will contain all indices.
BucketCache *	itsCache
	The cache with the SSM buckets.
BucketFile *	itsFile
	The file containing all data.
SSMStringHandler *	itsStringHandler
	String handler class.
uInt	itsPersCacheSize
	The persistent cache size.
uInt	itsCacheSize
	The actual cache size.
uInt	itsNrBuckets
	The initial number of buckets in the cache.
uInt	itsNrIdxBuckets
	Nr of buckets needed for index.
Int	itsFirstIdxBucket
	Number of the first index bucket.
uInt	itsIdxBucketOffset
	Offset of index in first bucket.
Int	itsLastStringBucket
	Number of the first String Bucket.
uInt	itsIndexLength
	length of index memoryblock
uInt	itsFreeBucketsNr
	The nr of free buckets.
Int	itsFirstFreeBucket
	The first free bucket.
uInt	itsBucketSize
	The bucket size.
uInt	itsBucketRows
PtrBlock< SSMColumn * >	itsPtrColumn
	The assembly of all columns.
Bool	isDataChanged
	Has the data changed since the last flush?

---

Constructor & Destructor Documentation

casa::SSMBase::SSMBase	(	Int	aBucketSize = 0,
		uInt	aCacheSize = 1
	)			[explicit]

Create a Standard storage manager with default name SSM.

casa::SSMBase::SSMBase	(	const String &	aDataManName,
		Int	aBucketSize = 0,
		uInt	aCacheSize = 1
	)			[explicit]

Create a Standard storage manager with the given name.

casa::SSMBase::SSMBase	(	const String &	aDataManName,
		const Record &	spec
	)

Create a Standard storage manager with the given name.

The specifications are part of the record (as created by dataManagerSpec).

casa::SSMBase::~SSMBase

(

)

casa::SSMBase::SSMBase

(

const SSMBase &

that

)

[private]

Copy constructor (only meant for clone function).

---

Member Function Documentation

virtual DataManager* casa::SSMBase::clone

(

)

const [virtual]

Clone this object.

It does not clone SSMColumn objects possibly used. The caller has to delete the newly created object.

Implements casa::DataManager.

virtual String casa::SSMBase::dataManagerType

(

)

const [virtual]

Get the type name of the data manager (i.e.

StandardStMan).

Implements casa::DataManager.

virtual String casa::SSMBase::dataManagerName

(

)

const [virtual]

Get the name given to the storage manager (in the constructor).

Reimplemented from casa::DataManager.

virtual Record casa::SSMBase::dataManagerSpec

(

)

const [virtual]

Record a record containing data manager specifications.

Reimplemented from casa::DataManager.

uInt casa::SSMBase::getVersion

(

)

const

Get the version of the class.

void casa::SSMBase::setCacheSize

(

uInt

aCacheSize

)

Set the cache size (in buckets).

uInt casa::SSMBase::getCacheSize

(

)

const [inline]

Get the current cache size (in buckets).

Definition at line 452 of file SSMBase.h.

References itsCacheSize.

void casa::SSMBase::clearCache

(

)

Clear the cache used by this storage manager.

It will flush the cache as needed and remove all buckets from it.

void casa::SSMBase::showCacheStatistics

(

ostream &

anOs

)

const

Show the statistics of all caches used.

void casa::SSMBase::showIndexStatistics

(

ostream &

anOs

)

const

Show Statistics of all indices used.

void casa::SSMBase::showBaseStatistics

(

ostream &

anOs

)

const

Show Statistics of the Base offsets/index etc.

uInt casa::SSMBase::getBucketSize

(

)

const [inline]

Get the bucket size.

Definition at line 462 of file SSMBase.h.

References itsBucketSize.

uInt casa::SSMBase::getNRow

(

)

const [inline]

Get the number of rows in this storage manager.

Definition at line 457 of file SSMBase.h.

References itsNrRows.

virtual Bool casa::SSMBase::canAddRow

(

)

const [virtual]

The storage manager can add rows.

Reimplemented from casa::DataManager.

virtual Bool casa::SSMBase::canRemoveRow

(

)

const [virtual]

The storage manager can delete rows.

Reimplemented from casa::DataManager.

virtual Bool casa::SSMBase::canAddColumn

(

)

const [virtual]

The storage manager can add columns.

Reimplemented from casa::DataManager.

virtual Bool casa::SSMBase::canRemoveColumn

(

)

const [virtual]

The storage manager can delete columns.

Reimplemented from casa::DataManager.

static DataManager* casa::SSMBase::makeObject	(	const String &	aDataManType,
		const Record &	spec
	)			[static]

Make the object from the type name string.

This function gets registered in the DataManager "constructor" map. The caller has to delete the object.

SSMColumn & casa::SSMBase::getColumn

(

uInt

aColNr

)

[inline]

Get access to the given column.

Definition at line 475 of file SSMBase.h.

References itsPtrColumn.

SSMIndex & casa::SSMBase::getIndex

(

uInt

anIdxNr

)

[inline]

Get access to the given Index.

Definition at line 480 of file SSMBase.h.

References itsPtrIndex.

void casa::SSMBase::setBucketDirty

(

)

Make the current bucket in the cache dirty (i.e.

something has been changed in it and it needs to be written when removed from the cache). (used by SSMColumn::putValue).

StManArrayFile* casa::SSMBase::openArrayFile

(

ByteIO::OpenOption

anOpt

)

Open (if needed) the file for indirect arrays with the given mode.

Return a pointer to the object.

char* casa::SSMBase::find	(	uInt	aRowNr,
		uInt	aColNr,
		uInt &	aStartRow,
		uInt &	anEndRow
	)

Find the bucket containing the column and row and return the pointer to the beginning of the column data in that bucket.

It also fills in the start and end row for the column data.

uInt casa::SSMBase::getNewBucket

(

)

Add a new bucket and get its bucket number.

char* casa::SSMBase::getBucket

(

uInt

aBucketNr

)

Read the bucket (if needed) and return the pointer to it.

void casa::SSMBase::removeBucket

(

uInt

aBucketNr

)

Remove a bucket from the bucket cache.

uInt casa::SSMBase::getRowsPerBucket

(

uInt

aColumn

)

const

Get rows per bucket for the given column.

Referenced by casa::SSMColumn::getDataPtr().

SSMStringHandler * casa::SSMBase::getStringHandler

(

)

[inline]

Return a pointer to the (one and only) StringHandler object.

Definition at line 485 of file SSMBase.h.

References itsStringHandler.

static char* casa::SSMBase::readCallBack	(	void *	anOwner,
		const char *	aBucketStorage
	)			[static]

Callbacks for BucketCache access.

static void casa::SSMBase::writeCallBack	(	void *	anOwner,
		char *	aBucketStorage,
		const char *	aBucket
	)			[static]

static void casa::SSMBase::deleteCallBack	(	void *	,
		char *	aBucket
	)			[static]

static char* casa::SSMBase::initCallBack

(

void *

anOwner

)

[static]

SSMBase& casa::SSMBase::operator=

(

const SSMBase &

that

)

[private]

Assignment cannot be used.

void casa::SSMBase::recreate

(

)

[private]

(Re)create the index, file, and cache object.

It is used when all rows are deleted from the table.

virtual Bool casa::SSMBase::flush	(	AipsIO &	,
		Bool	doFsync
	)			[private, virtual]

Flush and optionally fsync the data.

It returns a True status if it had to flush (i.e. if data have changed).

Implements casa::DataManager.

virtual void casa::SSMBase::create

(

uInt

aNrRows

)

[private, virtual]

Let the storage manager create files as needed for a new table.

This allows a column with an indirect array to create its file.

Implements casa::DataManager.

virtual void casa::SSMBase::open	(	uInt	aRowNr,
		AipsIO &
	)			[private, virtual]

Open the storage manager file for an existing table, read in the data, and let the SSMColumn objects read their data.

Implements casa::DataManager.

virtual void casa::SSMBase::resync

(

uInt

aRowNr

)

[private, virtual]

Resync the storage manager with the new file contents.

This is done by clearing the cache.

Implements casa::DataManager.

virtual void casa::SSMBase::reopenRW

(

)

[private, virtual]

Reopen the storage manager files for read/write.

Reimplemented from casa::DataManager.

virtual void casa::SSMBase::deleteManager

(

)

[private, virtual]

The data manager will be deleted (because all its columns are requested to be deleted).

So clean up the things needed (e.g. delete files).

Implements casa::DataManager.

void casa::SSMBase::init

(

)

[private]

Let the storage manager initialize itself (upon creation).

It determines the bucket size and fills the index.

uInt casa::SSMBase::setBucketSize

(

)

[private]

Determine and set the bucket size.

It returns the number of rows per bucket.

uInt casa::SSMBase::getNrIndices

(

)

const [inline, private]

Get the number of indices in use.

Definition at line 447 of file SSMBase.h.

References itsPtrIndex.

virtual void casa::SSMBase::addRow

(

uInt

aNrRows

)

[private, virtual]

Add rows to the storage manager.

Per column it extends number of rows.

Reimplemented from casa::DataManager.

virtual void casa::SSMBase::removeRow

(

uInt

aRowNr

)

[private, virtual]

Delete a row from all columns.

Reimplemented from casa::DataManager.

virtual void casa::SSMBase::addColumn

(

DataManagerColumn *

)

[private, virtual]

Do the final addition of a column.

Reimplemented from casa::DataManager.

virtual void casa::SSMBase::removeColumn

(

DataManagerColumn *

)

[private, virtual]

Remove a column from the data file.

Reimplemented from casa::DataManager.

virtual DataManagerColumn* casa::SSMBase::makeScalarColumn	(	const String &	aName,
		int	aDataType,
		const String &	aDataTypeID
	)			[private, virtual]

Create a scalar column.

Implements casa::DataManager.

virtual DataManagerColumn* casa::SSMBase::makeDirArrColumn	(	const String &	aName,
		int	aDataType,
		const String &	aDataTypeID
	)			[private, virtual]

Create a direct array column.

Implements casa::DataManager.

virtual DataManagerColumn* casa::SSMBase::makeIndArrColumn	(	const String &	aName,
		int	aDataType,
		const String &	aDataTypeID
	)			[private, virtual]

Create an indirect array column.

Implements casa::DataManager.

BucketCache & casa::SSMBase::getCache

(

)

[inline, private]

Get the cache object.

This will construct the cache object if not present yet. The cache object will be deleted by the destructor.

Definition at line 467 of file SSMBase.h.

References itsCache, and makeCache().

void casa::SSMBase::makeCache

(

)

[private]

Construct the cache object (if not constructed yet).

Referenced by getCache().

void casa::SSMBase::readHeader

(

)

[private]

Read the header.

void casa::SSMBase::readIndexBuckets

(

)

[private]

Read the index from its buckets.

void casa::SSMBase::writeIndex

(

)

[private]

Write the header and the indices.

---

Member Data Documentation

String casa::SSMBase::itsDataManName [private]

Name of data manager.

Definition at line 378 of file SSMBase.h.

StManArrayFile* casa::SSMBase::itsIosFile [private]

The file containing the indirect arrays.

Definition at line 381 of file SSMBase.h.

uInt casa::SSMBase::itsNrRows [private]

The number of rows in the columns.

Definition at line 384 of file SSMBase.h.

Referenced by getNRow().

Block<uInt> casa::SSMBase::itsColumnOffset [private]

Column offset.

Definition at line 387 of file SSMBase.h.

Block<uInt> casa::SSMBase::itsColIndexMap [private]

Row Index ID containing all the columns in a bucket.

Definition at line 390 of file SSMBase.h.

PtrBlock<SSMIndex*> casa::SSMBase::itsPtrIndex [private]

Will contain all indices.

Definition at line 393 of file SSMBase.h.

Referenced by getIndex(), and getNrIndices().

BucketCache* casa::SSMBase::itsCache [private]

The cache with the SSM buckets.

Definition at line 396 of file SSMBase.h.

Referenced by getCache().

BucketFile* casa::SSMBase::itsFile [private]

The file containing all data.

Definition at line 399 of file SSMBase.h.

SSMStringHandler* casa::SSMBase::itsStringHandler [private]

String handler class.

Definition at line 402 of file SSMBase.h.

Referenced by getStringHandler().

uInt casa::SSMBase::itsPersCacheSize [private]

The persistent cache size.

Definition at line 405 of file SSMBase.h.

uInt casa::SSMBase::itsCacheSize [private]

The actual cache size.

Definition at line 408 of file SSMBase.h.

Referenced by getCacheSize().

uInt casa::SSMBase::itsNrBuckets [private]

The initial number of buckets in the cache.

Definition at line 411 of file SSMBase.h.

uInt casa::SSMBase::itsNrIdxBuckets [private]

Nr of buckets needed for index.

Definition at line 414 of file SSMBase.h.

Int casa::SSMBase::itsFirstIdxBucket [private]

Number of the first index bucket.

Definition at line 417 of file SSMBase.h.

uInt casa::SSMBase::itsIdxBucketOffset [private]

Offset of index in first bucket.

If >0, the index fits in a single bucket.

Definition at line 421 of file SSMBase.h.

Int casa::SSMBase::itsLastStringBucket [private]

Number of the first String Bucket.

Definition at line 424 of file SSMBase.h.

uInt casa::SSMBase::itsIndexLength [private]

length of index memoryblock

Definition at line 427 of file SSMBase.h.

uInt casa::SSMBase::itsFreeBucketsNr [private]

The nr of free buckets.

Definition at line 430 of file SSMBase.h.

Int casa::SSMBase::itsFirstFreeBucket [private]

The first free bucket.

Definition at line 433 of file SSMBase.h.

uInt casa::SSMBase::itsBucketSize [private]

The bucket size.

Definition at line 436 of file SSMBase.h.

Referenced by getBucketSize().

uInt casa::SSMBase::itsBucketRows [private]

Definition at line 437 of file SSMBase.h.

PtrBlock<SSMColumn*> casa::SSMBase::itsPtrColumn [private]

The assembly of all columns.

Definition at line 440 of file SSMBase.h.

Referenced by getColumn().

Bool casa::SSMBase::isDataChanged [private]

Has the data changed since the last flush?

Definition at line 443 of file SSMBase.h.

---

The documentation for this class was generated from the following file:

• tables/implement/Tables/SSMBase.h

---