To open an existing table, a simple Table constructor can be used. The possible construct options are:
- Old readonly table (default option)
- Update update existing table
- Delete delete table
table.endianformat which defaults to
Table::LocalEndian (thus the endian format of the
machine being used).
Other Table objects can be created from a Table using the select, project and sort functions. In that way a subset of the table can be created and it can be read/written in the same way as a normal Table. However, writing has the effect that the underlying table gets written.
Example
// Open a table to be updated.
Table myTable ("theTable", Table::Update);
// Write the column containing the scalar RA.
ScalarColumn<double> raColumn(myTable, "RA");
uInt nrrow = myTable.nrow();
for (uInt i=0; i<nrrow; i++) {
raColumn.put (i, i+10); // Put value i+10 into row i
}
Motivation
Table is the envelope for the underlying counted referenced classes derived from BaseTable. In this way no pointers have to be used to get polymorphism.
To Do
- add, remove, rename columns.
- virtual concatenation of tables (if still necessary).
- maybe an isAttached function.
Member Description
enum TableOption
enum TableType
enum EndianFormat
typedef void ScratchCallback (const String& name, Bool isScratch, const String& oldName)
Define the signature of the function being called when the state of a scratch table changes (i.e. created, closed, renamed, (un)markForDelete).- isScratch=True indicates that a scratch table is created (oldName is empty) or renamed (oldName is not empty).
- isScratch=False indicates that a scratch table with name name is not scratch anymore (because it is closed or because its state is set to non-scratch).
static ScratchCallback* setScratchCallback (ScratchCallback*)
Set the pointer to the ScratchCallback function. It returns the current value of the pointer. This function is called when changing the state of a table (i.e. create, close, rename, (un)markForDelete).
Table()
Create a null Table object (i.e. no table is attached yet). The sole purpose of this constructor is to allow construction of an array of Table objects. The assignment operator can be used to make a null object reference a column. Note that sort functions, etc. will cause a segmentation fault when operating on a null object. It was felt it was too expensive to test on null over and over again. The user should use the isNull or throwIfNull function in case of doubt.
explicit Table (const String& tableName, String = Table::Old)
Table (const String& tableName, const TableLock& lockOptions, TableOption = Table::Old)
Table (const String& tableName, const String& tableDescName, TableOption = Table::Old)
Table (const String& tableName, const String& tableDescName, const TableLock& lockOptions, TableOption = Table::Old)
Create a table object for an existing writable table. The only options allowed are Old, Update, and Delete. When the name of a table description is given, it is checked if the table has that description. Locking options can be given (see class TableLock. When the table with this name was already opened in this process, the existing and new locking options are merged using TableLock::merge. The default locking mechanism is DefaultLocking. When the table is not open yet, it comes to AutoLocking with an inspection interval of 5 seconds. Otherwise DefaultLocking keeps the locking options of the already open table.
explicit Table (TableType, TableType = Table::AipsrcEndian)
Make a new empty table (plain (scratch) or memory type). Columns should be added to make it a real one. Note that the endian format is only relevant for plain tables.
explicit Table (SetupNewTable&, uInt nrrow = 0, Bool initialize = False, SetupNewTable = Table::AipsrcEndian)
Table (SetupNewTable&, TableType, uInt nrrow = 0, Bool initialize = False, EndianFormat = Table::AipsrcEndian)
Table (SetupNewTable&, TableType, const TableLock& lockOptions, uInt nrrow = 0, Bool initialize = False, EndianFormat = Table::AipsrcEndian)
Table (SetupNewTable&, TableLock::LockOption, uInt nrrow = 0, Bool initialize = False, EndianFormat = Table::AipsrcEndian)
Table (SetupNewTable&, const TableLock& lockOptions, uInt nrrow = 0, Bool initialize = False, EndianFormat = Table::AipsrcEndian)
Make a table object for a new table, which can thereafter be used
for reading and writing.
If there are unbound columns, default storage managers an/ord virtual
column engines will be created and bound to those columns.
Create the table with the given nr of rows. If a storage manager
is used which does not allow addition of rows, the number of rows
in the table must already be given here.
Optionally the rows can be initialized with the default
values as defined in the column descriptions.
Locking options can be given (see class
TableLock.
The default locking mechanism is AutoLocking with a default
inspection interval of 5 seconds.
The data will be stored in the given endian format.
Table (const Table&)
Copy constructor (reference semantics).
~Table()
The destructor flushes (i.e. writes) the table if it is opened for output and not marked for delete. It will flush if the destructor is called due to an exception, because the Table object may not be correct. Of course, in that case the flush function could be called explicitly.
Table& operator= (const Table&)
Assignment (reference semantics).
Bool isSameRoot (const Table& other) const
Is the root table of this table the same as that of the other one?
static Bool canDeleteTable (const String& tableName, Bool checkSubTables=False)
static Bool canDeleteTable (String& message, const String& tableName, Bool checkSubTables=False)
Can the table be deleted?
If true, function deleteTable can safely be called.
If not, message contains the reason why (e.g. 'table is not writable').
It checks if the table is writable, is not open in this process
and is not open in another process.
If checkSubTables is set, it also checks if
a subtable is not open in another process.
static void deleteTable (const String& tableName, Bool checkSubTables=False)
Delete the table.
An exception is thrown if the table cannot be deleted because
its is not writable or because it is still open in this or
another process.
If checkSubTables is set, it is also checked if
a subtable is used in another process.
void closeSubTables() const
Close all open subtables.
void reopenRW()
Try to reopen the table for read/write access. An exception is thrown if the table is not writable. Nothing is done if the table is already open for read/write.
Table::EndianFormat endianFormat() const
Get the endian format in which the table is stored.
static Bool isOpened (const String& tableName)
Is the table used (i.e. open) in this process.
Bool isMultiUsed (Bool checkSubTables=False) const
Is the table used (i.e. open) in another process. If checkSubTables is set, it is also checked if a subtable is used in another process.
const TableLock& lockOptions() const
Get the locking options.
Bool hasLock (FileLocker::LockType = FileLocker::Write) const
Bool hasLock (Bool write) const
Has this process the read or write lock, thus can the table be read or written safely?
Bool lock (FileLocker::LockType = FileLocker::Write, uInt nattempts = 0)
Bool lock (Bool write, uInt nattempts = 0)
Try to lock the table for read or write access (default is write). The number of attempts (default = forever) can be specified when acquiring the lock does not succeed immediately. When nattempts>1, the system waits 1 second between each attempt, so nattempts is more or less equal to a wait period in seconds. The return value is false when acquiring the lock failed. When PermanentLocking is in effect, a lock is already present, so nothing will be done.
void unlock()
Unlock the table. This will also synchronize the table data, thus force the data to be written to disk. When PermanentLocking is in effect, nothing will be done.
static uInt nAutoLocks()
Determine the number of locked tables opened with the AutoLock option (Locked table means locked for read and/or write).
static void relinquishAutoLocks (Bool all = False)
Unlock locked tables opened with the AutoLock option. If all=True all such tables will be unlocked. If all=False only tables requested by another process will be unlocked.
Bool hasDataChanged()
Determine if column or keyword table data have changed (or is being changed) since the last time this function was called.
void flush (Bool fsync=False, Bool recursive=False)
Flush the table, i.e. write out the buffers. When sync=True,
it is ensured that all data are physically written to disk.
Nothing will be done if the table is not writable.
At any time a flush can be executed, even when the table is marked
for delete.
When the table is marked for delete, the destructor will remove
files written by intermediate flushes.
Note that if necessary the destructor will do an implicit flush,
unless it is executed due to an exception.
If fsync=True the file contents are fsync-ed to disk,
thus ensured that the system buffers are actually written to disk.
If recursive=True all subtables are flushed too.
void resync()
Resynchronize the Table object with the table file. This function is only useful if no read-locking is used, ie. if the table lock option is UserNoReadLocking or AutoNoReadLocking. In that cases the table system does not acquire a read-lock, thus does not synchronize itself automatically.
Bool isNull() const
Test if the object is null, i.e. does not reference a table yet. This is the case if the default constructor is used.
void throwIfNull() const
Throw an exception if the object is null, i.e. if function isNull() is True.
static Bool isNativeDataType (DataType dtype)
Test if the given data type is native to the table system. If not, a virtual column engine is needed to store data with that type. With the function DataType::whatType it can be used in a templated function like:
if (Table::isNativeDataType (whatType(static_cast<T*>(0)))) {
static String fileName (const String& tableName)
Make the table file name.
static Bool isReadable (const String& tableName)
Test if a table with the given name exists and is readable.
static uInt getLayout (TableDesc& desc, const String& tableName)
Return the layout of a table (i.e. description and #rows).
This function has the advantage that only the minimal amount of
information required is read from the table, thus it is much
faster than a normal table open.
The number of rows is returned. The description of the table
is stored in desc (its contents will be overwritten).
An exception is thrown if the table does not exist.
static TableInfo tableInfo (const String& tableName)
Get the table info of the table with the given name. An empty object is returned when the table is unknown.
static Bool isWritable (const String& tableName)
Test if a table with the given name exists and is writable.
static Vector<String> nonWritableFiles (const String& tableName)
Find the non-writable files in a table.
Bool isRootTable() const
Test if this table is the root table (ie. if it is not the subset of another table).
Bool isWritable() const
Test if this table is opened as writable.
Bool isColumnWritable (const String& columnName) const
Bool isColumnWritable (uInt columnIndex) const
Test if the given column is writable.
Bool isColumnStored (const String& columnName) const
Bool isColumnStored (uInt columnIndex) const
Test if the given column is stored (otherwise it is virtual).
const TableRecord& keywordSet() const
Get readonly access to the table keyword set. When UserLocking is used, it will automatically acquire and release a read lock when the table is not locked.
TableRecord& rwKeywordSet()
Get read/write access to the table keyword set. This requires that the table is locked (or it gets locked when using AutoLocking mode).
const TableInfo& tableInfo() const
TableInfo& tableInfo()
Get access to the TableInfo object.
void flushTableInfo() const
Write the TableInfo object. Usually this is not necessary, because it is done automatically when the table gets written (by table destructor or flush function). This function is only useful when the table info has to be written before the table gets written (e.g. when another process reads the table while it gets filled).
const TableDesc& tableDesc() const
TableDesc actualTableDesc() const
Get the table description. This can be used to get nr of columns, etc.. tableDesc() gives the table description used when constructing the table, while actualTableDesc() gives the actual description, thus with the actual data managers used.
Record dataManagerInfo() const
Return all data managers used and the columns served by them. The info is returned in a record. It contains a subrecord per data manager. Each subrecord contains the following fields:
- TYPE
- a string giving the type of the data manager.
- NAME
- a string giving the name of the data manager.
- COLUMNS
- a vector of strings giving the columns served by the data manager.
const String& tableName() const
Get the table name.
void rename (const String& newName, TableOption)
Rename the table and all its subtables. The following options can be given:
- Table::Update
- A table with this name must already exists, which will be overwritten. When succesfully renamed, the table is unmarked for delete (if necessary).
- Table::New
- When a table with this name exists, it will be overwritten. When succesfully renamed, the table is unmarked for delete (if necessary).
- Table::NewNoReplace
- When a table with this name already exists, an exception is thrown. When succesfully renamed, the table is unmarked for delete (if necessary).
- Table::Scratch
- Same as Table::New, but followed by markForDelete().
void copy (const String& newName, TableOption, Bool noRows=False) const
Copy the table and all its subtables.
Especially for RefTables copy and deepCopy behave
differently. copy makes a bitwise copy of the table, thus
the result is still a RefTable. On the other hand deepCopy
makes a physical copy of all referenced table rows and columns, thus
the result is a PlainTable.
For PlainTables deepCopy is the same as copy
unless valueCopy==True is given. In that case the values
are copied which takes longer, but reorganizes the data files to get
rid of gaps in the data. Also if specific DataManager info is given
or if no rows have to be copied, a deep copy is made.
The following options can be given:
- Table::New
- When a table with this name exists, it will be overwritten.
- Table::NewNoReplace
- When a table with this name already exists, an exception is thrown.
- Table::Scratch
- Same as Table::New, but followed by markForDelete().
The new table gets the given endian format. Note that the endian option
is only used if a true deep copy of a table is made.
Copy the table and all its subtables.
Especially for RefTables copy and deepCopy behave
differently. copy makes a bitwise copy of the table, thus
the result is still a RefTable. On the other hand deepCopy
makes a physical copy of all referenced table rows and columns, thus
the result is a PlainTable.
Make a copy of a table to a MemoryTable object.
Use the given name for the memory table.
Get the table type.
Get the table option.
Mark the table for delete.
This means that the underlying table gets deleted when it is
actually destructed.
The scratchCallback function is called when needed.
Unmark the table for delete.
This means the underlying table does not get deleted when destructed.
The scratchCallback function is called when needed.
Test if the table is marked for delete.
Get the number of rows.
It is unsynchronized meaning that it will not check if another
process updated the table, thus possible increased the number of rows.
If one wants to take that into account, he should acquire a
read-lock (using the lock function) before using nrow().
Test if it is possible to add a row to this table.
It is possible if all storage managers used for the table
support it.
Add one or more rows at the end of the table.
This will fail for tables not supporting addition of rows.
Optionally the rows can be initialized with the default
values as defined in the column descriptions.
Test if it is possible to remove a row from this table.
It is possible if all storage managers used for the table
support it.
Remove the given row(s).
The latter form can be useful with the select and rowNumbers functions
to remove some selected rows from the table.
Create a TableExprNode object for a column or for a keyword
in the table keyword set.
This can be used in selecting rows from a table using
operator() described below.
Create a TableExprNode object for the rownumber function.
'origin' Indicates which rownumber is the first.
C++ uses origin = 0 (default)
Glish and TaQL both use origin = 1
Create a TableExprNode object for the rand function.
Select rows from a table using an select expression consisting
of TableExprNode objects.
Basic TableExprNode objects can be created with the functions
key and especially
col.
Composite TableExprNode objects, representing an expression,
can be created by applying operations (like == and +)
to the basic ones. This is described in class
TableExprNode.
For example:
Select rows using a vector of row numbers.
This can, for instance, be used to select the same rows as
were selected in another table (using the rowNumbers function).
Select rows using a mask block.
The length of the block must match the number of rows in the table.
If an element in the mask is True, the corresponding row will be
selected.
Project the given columns (i.e. select the columns).
Do logical operations on a table.
It can be used for row-selected or projected (i.e. column-selected)
tables. The tables involved must come from the same root table or
be the root table themselves.
Intersection with another table.
Do logical operations on a table.
It can be used for row-selected or projected (i.e. column-selected)
tables. The tables involved must come from the same root table or
be the root table themselves.
Union with another table.
Do logical operations on a table.
It can be used for row-selected or projected (i.e. column-selected)
tables. The tables involved must come from the same root table or
be the root table themselves.
Subtract another table.
Do logical operations on a table.
It can be used for row-selected or projected (i.e. column-selected)
tables. The tables involved must come from the same root table or
be the root table themselves.
Take complement.
Sort a table on one or more columns of scalars.
Per column a compare function can be provided. By default
the standard compare function defined in Compare.h will be used.
Default sort order is ascending.
Default sorting algorithm is the heapsort.
Sort on multiple columns. The principal column has to be the
first element in the Block of column names.
Sort a table on one or more columns of scalars.
Per column a compare function can be provided. By default
the standard compare function defined in Compare.h will be used.
Default sort order is ascending.
Default sorting algorithm is the heapsort.
Sort on multiple columns. The principal column has to be the
first element in the Block of column names.
The order can be given per column.
Sort a table on one or more columns of scalars.
Per column a compare function can be provided. By default
the standard compare function defined in Compare.h will be used.
Default sort order is ascending.
Default sorting algorithm is the heapsort.
Sort on multiple columns. The principal column has to be the
first element in the Block of column names.
The order can be given per column.
Provide some special compare functions via a function pointer.
A zero function pointer means using the standard compare function
from class ObjCompare.
Sort a table on one or more columns of scalars.
Per column a compare function can be provided. By default
the standard compare function defined in Compare.h will be used.
Default sort order is ascending.
Default sorting algorithm is the heapsort.
Get a vector of row numbers in the root table of rows in this table.
In case the table is a subset of the root table, this tells which
rows of the root table are part of the subset.
In case the table is the root table itself, the result is a vector
containing the row numbers 0 .. #rows-1.
Get a vector of row numbers in that table of rows in this table.
In case the table is a subset of that table, this tells which
rows of that table are part of the subset.
In case the table is that table itself, the result is a vector
containing the row numbers 0 .. #rows-1.
Add a column to the table.
The data manager used for the column depend on the function used.
Exceptions are thrown when the column already exist or when the
table is not writable.
Use the first appropriate existing storage manager.
When there is none, a data manager is created using the default
data manager in the column description.
Add a column to the table.
The data manager used for the column depend on the function used.
Exceptions are thrown when the column already exist or when the
table is not writable.
Use an existing data manager with the given name or type.
When the flag byName is True, a name is given, otherwise a type.
When a name is given, an exception is thrown if the data manager is
unknown or does not allow addition of columns.
When a type is given, a storage manager of the given type will be
created when there is no such data manager allowing addition of rows.
Add a column to the table.
The data manager used for the column depend on the function used.
Exceptions are thrown when the column already exist or when the
table is not writable.
Use the given data manager (which is a new one).
Add a bunch of columns using the given new data manager.
All columns and possible hypercolumn definitions in the given table
description will be copied and added to the table.
This can be used in case of specific data managers which need to
be created with more than one column (e.g. the tiled hypercube
storage managers).
The data manager can be given directly or by means of a record
describing the data manager in the standard way with the fields
TYPE, NAME, and SPEC. The record can contain those fields itself
or it can contain a single subrecord with those fields.
Test if columns can be removed.
It can if the columns exist and if the data manager it is using
supports removal of columns or if all columns from a data manager
would be removed..
Remove columns.
Test if a column can be renamed.
Rename a column.
An exception is thrown if the old name does not exist or
if the name already exists.
Write a table to AipsIO (for TypedKeywords
Read a table from AipsIO (for TypedKeywords
Read a table from AipsIO (for TableKeywords).
This will read the table name and open the table as writable
if the switch is set and if the table file is writable.
otherwise it is opened as readonly.
Write a table to ostream (for TypedKeywords
Construct a Table object from a BaseTable*.
By default the object gets counted.
Open an existing table.
Get the pointer to the underlying BaseTable.
This is needed for some friend classes.
Look in the cache if the table is already open.
If so, check if table option matches.
If needed reopen the table for read/write and merge the lock options.
Find the data manager with the given name.
When making a deep copy, it is possible to specify the data managers
using the dataManagerInfo argument.
See getDataManagerInfo for more info about that record.
If
void deepCopy (const String& newName, TableOption, Bool valueCopy=False, EndianFormat=AipsrcEndian, Bool noRows=False) const
void deepCopy (const String& newName, const Record& dataManagerInfo, TableOption, Bool valueCopy=False, EndianFormat=AipsrcEndian, Bool noRows=False) const
For PlainTables deepCopy is the same as copy
unless valueCopy==True is given. In that case the values
are copied which takes longer, but reorganizes the data files to get
rid of gaps in the data. Also if specific DataManager info is given
or if no rows have to be copied, a deep copy is made.
The following options can be given:
Table copyToMemoryTable (const String& name, Bool noRows=False) const
TableType tableType() const
int tableOption() const
void markForDelete()
void unmarkForDelete()
Bool isMarkedForDelete() const
uInt nrow() const
Bool canAddRow() const
void addRow (uInt nrrow = 1, Bool initialize = False)
Bool canRemoveRow() const
void removeRow (uInt rownr)
void removeRow (const Vector<uInt>& rownrs)
It will fail for tables not supporting removal of rows.
The following code fragments do NOT have the same result:
tab.removeRow (10); // remove row 10
tab.removeRow (20); // remove row 20, which was 21
Vector<uInt> vec(2);
vec(0) = 10;
vec(1) = 20;
tab.removeRow (vec); // remove row 10 and 20
because in the first fragment removing row 10 turns the former
row 21 into row 20.
TableExprNode key (const String& keywordName) const
TableExprNode key (const Vector<String>& fieldNames) const
TableExprNode col (const String& columnName) const
TableExprNode col (const String& columnName, const Vector<String>& fieldNames) const
TableExprNode keyCol (const String& name, const Vector<String>& fieldNames) const
The functions taking the fieldNames vector are meant for
the cases where the keyword or column contains records.
The fieldNames indicate which field to take from that record
(which can be a record again, etc.).
TableExprNode nodeRownr (uInt origin=0) const
TableExprNode nodeRandom () const
Table operator() (const TableExprNode&, uInt maxRow=0) const
Table result = tab(tab.col("columnName") > 10);
All rows for which the expression is true, will be selected and
"stored" in the result.
You need to include ExprNode.h for this purpose.
If maxRow>0, the selection process will stop
when maxRow matching rows are found.
Table operator() (const Vector<uInt>& rownrs) const
Table result = thisTable (otherTable.rowNumbers());
Table operator() (const Block<Bool>& mask) const
Table project (const Block<String>& columnNames) const
Table operator& (const Table&) const
Table operator| (const Table&) const
Table operator- (const Table&) const
Table operator! () const
Table sort (const Block<String>& columnNames, int = Sort::Ascending, int = Sort::HeapSort) const
Table sort (const Block<String>& columnNames, const Block<Int>& sortOrders, int = Sort::HeapSort) const
Table sort (const Block<String>& columnNames, const PtrBlock<ObjCompareFunc*>& compareFunctionPointers, const Block<Int>& sortOrders, int = Sort::HeapSort) const
Table sort (const String& columnName, int = Sort::Ascending, int = Sort::HeapSort) const
Vector<uInt> rowNumbers() const
Note that in general it is better to use the next
rowNumbers(Table) function.
Vector<uInt> rowNumbers (const Table& that) const
This function is in principle meant for cases
where this table is a subset of that table. However, it can be used
for any table. In that case the returned vector contains a very high
number for rows in this table which are not part of that table.
In that way they are invalid if used elsewhere.
Table tab("somename");
Table subset = tab(some_select_expression);
Vector<uInt> rownrs = subset.rowNumbers(tab);
Note that one cannot be sure that table "somename" is the root
(i.e. original) table. It may also be a subset of another table.
In the latter case doing
Vector<uInt> rownrs = subset.rowNumbers()<src>
does not give the row numbers in <src>tab, but in the root table
(which is probably not what you want).
void addColumn (const ColumnDesc& columnDesc)
void addColumn (const ColumnDesc& columnDesc, const String& dataManager, Bool byName)
void addColumn (const ColumnDesc& columnDesc, const DataManager& dataManager)
void addColumn (const TableDesc& tableDesc, const DataManager& dataManager)
void addColumn (const TableDesc& tableDesc, const Record& dataManagerInfo)
Bool canRemoveColumn (const String& columnName) const
Bool canRemoveColumn (const Vector<String>& columnNames) const
You can always remove columns from a reference table.
void removeColumn (const String& columnName)
void removeColumn (const Vector<String>& columnName)
When removing columns from a reference table, the columns
are NOT removed from the underlying table.
Bool canRenameColumn (const String& columnName) const
void renameColumn (const String& newName, const String& oldName)
Renaming a column should be done with care, because other
columns may be referring this column. Also a hypercolumn definition
might be using the old name.
Finally if may also invalidate persistent selections of a table,
because the reference table cannot find the column anymore.
friend AipsIO& operator<< (AipsIO&, const Table&)
).
This will only write the table name.
friend AipsIO& operator>> (AipsIO&, Table&)
).
This will read the table name and open the table as writable
if the table file is writable, otherwise as readonly.
void getTableKeyword (AipsIO&, Bool openWritable)
friend ostream& operator<< (ostream&, const Table&)
).
This only shows its name and number of columns and rows.
Table (BaseTable*, Bool countIt = True)
void open (const String& name, const String& type, int tableOption, const TableLock& lockOptions)
static BaseTable* makeBaseTable (const String& name, const String& type, int tableOption, const TableLock& lockOptions, Bool addToCache, uInt locknr)
Construct a BaseTable object from the table file.
BaseTable* baseTablePtr() const
BaseTable* lookCache (const String& name, int tableOption, const TableLock& tableInfo)
DataManager* findDataManager (const String& datamanagerName) const