casacore::TSMCube Class Reference

Tiled hypercube in a table. More...

#include <TSMCube.h>

Inheritance diagram for casacore::TSMCube:
casacore::TSMCubeBuff casacore::TSMCubeMMap

List of all members.

Public Types

enum  AccessType {
  NoAccess,
  CellAccess,
  SliceAccess,
  ColumnAccess,
  ColumnSliceAccess
}
 

Define the possible access types for TSMDataColumn.

More...

Public Member Functions

 TSMCube (TiledStMan *stman, TSMFile *file, const IPosition &cubeShape, const IPosition &tileShape, const Record &values, Int64 fileOffset, Bool useDerived=False)
 Construct the hypercube using the given file with the given shape.
 TSMCube (TiledStMan *stman, AipsIO &ios, Bool useDerived=False)
 Reconstruct the hypercube by reading its data from the AipsIO stream.
virtual ~TSMCube ()
virtual void flushCache ()
 Flush the data in the cache.
void clearCache (Bool doFlush=True)
 Clear the cache, so data will be reread.
void emptyCache ()
 Empty the cache.
virtual void showCacheStatistics (ostream &os) const
 Show the cache statistics.
void putObject (AipsIO &ios)
 Put the data of the object into the AipsIO stream.
Int getObject (AipsIO &ios)
 Get the data of the object from the AipsIO stream.
virtual void resync (AipsIO &ios)
 Resync the object with the data file.
Bool isExtensible () const
 Is the hypercube extensible?
uInt bucketSize () const
 Get the bucket size (which is the length of a tile in external format).
uInt localTileLength () const
 Get the length of a tile in local format.
virtual void setShape (const IPosition &cubeShape, const IPosition &tileShape)
 Set the hypercube shape.
const IPositioncubeShape () const
 Get the shape of the hypercube.
const IPositiontileShape () const
 Get the shape of the tiles.
IPosition cellShape () const
 Get the shape of the data cells in the cube.
uInt coordinateSize (const String &coordinateName) const
 Get the size of a coordinate (i.e.
const RecordvalueRecord () const
 Get the record containing the id and coordinate values.
RecordrwValueRecord ()
Bool matches (const PtrBlock< TSMColumn * > &idColSet, const Record &idValues)
 Test if the id values match.
virtual void extend (uInt nr, const Record &coordValues, const TSMColumn *lastCoordColumn)
 Extend the last dimension of the cube with the given number.
void extendCoordinates (const Record &coordValues, const String &coordName, uInt length)
 Extend the coordinates vector for the given coordinate to the given length with the given coordValues.
virtual void accessSection (const IPosition &start, const IPosition &end, char *section, uInt colnr, uInt localPixelSize, uInt externalPixelSize, Bool writeFlag)
 Read or write a section in the cube.
virtual void accessStrided (const IPosition &start, const IPosition &end, const IPosition &stride, char *section, uInt colnr, uInt localPixelSize, uInt externalPixelSize, Bool writeFlag)
 Read or write a section in a strided way.
uInt cacheSize () const
 Get the current cache size (in buckets).
uInt calcCacheSize (const IPosition &sliceShape, const IPosition &windowStart, const IPosition &windowLength, const IPosition &axisPath) const
 Calculate the cache size (in buckets) for the given slice and access path.
virtual void setCacheSize (const IPosition &sliceShape, const IPosition &windowStart, const IPosition &windowLength, const IPosition &axisPath, Bool forceSmaller, Bool userSet)
 Set the cache size for the given slice and access path.
virtual void setCacheSize (uInt cacheSize, Bool forceSmaller, Bool userSet)
 Resize the cache object.
uInt validateCacheSize (uInt cacheSize) const
 Validate the cache size (in buckets).
Bool userSetCache () const
 Determine if the user set the cache size (using setCacheSize).
AccessType getLastColAccess () const
 Functions for TSMDataColumn to keep track of the last type of access to a hypercube.
const IPositiongetLastColSlice () const
void setLastColAccess (AccessType type)
void setLastColSlice (const IPosition &slice)

Static Public Member Functions

static uInt calcCacheSize (const IPosition &cubeShape, const IPosition &tileShape, Bool extensible, const IPosition &sliceShape, const IPosition &windowStart, const IPosition &windowLength, const IPosition &axisPath, uInt maxCacheSize, uInt bucketSize)
static uInt validateCacheSize (uInt cacheSize, uInt maxSize, uInt bucketSize)

Protected Member Functions

void setup ()
 Initialize the various variables.
void setupNrTiles ()
IPosition adjustTileShape (const IPosition &cubeShape, const IPosition &tileShape) const
 Adjust the tile shape to the hypercube shape.
void resizeTileSections ()
 Resize the IPosition member variables used in accessSection() if nrdim_p changes value.

Protected Attributes

char * cachedTile_p
TiledStManstmanPtr_p
 Pointer to the parent storage manager.
Bool useDerived_p
 Is the class used directly or only by a derived class only?
Record values_p
 The values of the possible id and coordinate columns.
Bool extensible_p
 Is the hypercube extensible?
uInt nrdim_p
 Dimensionality of the hypercube.
uInt nrTiles_p
 Number of tiles in the hypercube.
IPosition cubeShape_p
 The shape of the hypercube.
IPosition tileShape_p
 The shape of the tiles in the hypercube.
IPosition tilesPerDim_p
 The number of tiles in each hypercube dimension.
TSMShape expandedTileShape_p
 Precomputed tileShape information.
TSMShape expandedTilesPerDim_p
 Precomputed tilesPerDim information.
uInt nrTilesSubCube_p
 Number of tiles in all but last dimension (used when extending).
uInt tileSize_p
 The tilesize in pixels.
TSMFilefilePtr_p
 Pointer to the TSMFile object holding the data.
Int64 fileOffset_p
 Offset in the TSMFile object where the data of this hypercube starts.
Block< uIntexternalOffset_p
 Offset for each data column in a tile (in external format).
Block< uIntlocalOffset_p
 Offset for each data column in a tile (in local format).
uInt bucketSize_p
 The bucket size in bytes (is equal to tile size in bytes).
uInt localTileLength_p
 The tile size in bytes in local format.
BucketCachecache_p
 The bucket cache.
Bool userSetCache_p
 Did the user set the cache size?
AccessType lastColAccess_p
 Was the last column access to a cell, slice, or column?
IPosition lastColSlice_p
 The slice shape of the last column access to a slice.
IPosition nrTileSection_p
 IPosition variables used in accessSection(); declared here as member variables to avoid significant construction and desctruction overhead if they are local to accessSection() #tiles needed for the section.
IPosition startTile_p
 First tile needed.
IPosition endTile_p
 Last tile needed.
IPosition startPixelInFirstTile_p
 First pixel in first tile.
IPosition endPixelInFirstTile_p
 Last pixel in first tile.
IPosition endPixelInLastTile_p
 Last pixel in last tile.

Private Member Functions

 TSMCube (const TSMCube &)
 Forbid copy constructor.
TSMCubeoperator= (const TSMCube &)
 Forbid assignment.
BucketCachegetCache ()
 Get the cache object.
virtual void makeCache ()
 Construct the cache object (if not constructed yet).
virtual void resyncCache ()
 Resync the cache object.
virtual void deleteCache ()
 Delete the cache object.
void accessLine (char *section, uInt pixelOffset, uInt localPixelSize, Bool writeFlag, BucketCache *cachePtr, const IPosition &startTile, uInt endTile, const IPosition &startPixelInFirstTile, uInt endPixelInLastTile, uInt lineIndex)
 Access a line in a more optimized way.
char * readTile (const char *external)
 Define the functions doing the actual read and write of the data in the tile and converting it to/from local format.
void writeTile (char *external, const char *local)

Static Private Member Functions

static char * readCallBack (void *owner, const char *external)
 Define the callback functions for the BucketCache.
static void writeCallBack (void *owner, char *external, const char *local)
static char * initCallBack (void *owner)
static void deleteCallBack (void *owner, char *buffer)

Detailed Description

Tiled hypercube in a table.

Intended use:

Internal

Review Status

Reviewed By:
UNKNOWN
Date Reviewed:
before2004/08/25

Prerequisite

Etymology

TSMCube represents a hypercube in the Tiled Storage Manager.

Synopsis

TSMCube defines a tiled hypercube. The data is stored in a TSMFile object and accessed using a BucketCache object. The hypercube can be extensible in its last dimension to support tables with a size which is not known in advance.
Normally hypercubes share the same TSMFile object, but extensible hypercubes have their own TSMFile object (to be extensible). If the hypercolumn has multiple data columns, their cells share the same tiles. Per tile data column A appears first, thereafter B, etc.\.
The data in the cache is held in external format and is converted when accessed. The alternative would be to hold it in the cache in local format and convert it when read/written from the file. It was felt that the latter approach would generate more needless conversions.

The possible id and coordinate values are stored in a Record object. They are written in the main hypercube AipsIO file.

TSMCube uses the maximum cache size set for a Tiled Storage manager. The description of class ROTiledStManAccessor contains a discussion about the effect of setting the maximum cache size.

Motivation

TSMCube encapsulates all operations on a hypercube.

Definition at line 105 of file TSMCube.h.


Member Enumeration Documentation

Define the possible access types for TSMDataColumn.

Enumerator:
NoAccess 
CellAccess 
SliceAccess 
ColumnAccess 
ColumnSliceAccess 

Definition at line 109 of file TSMCube.h.


Constructor & Destructor Documentation

casacore::TSMCube::TSMCube ( TiledStMan stman,
TSMFile file,
const IPosition cubeShape,
const IPosition tileShape,
const Record values,
Int64  fileOffset,
Bool  useDerived = False 
)

Construct the hypercube using the given file with the given shape.

The record contains the id and possible coordinate values.
If the cubeshape is empty, the hypercube is still undefined and can be added later with setShape. That is only used by TiledCellStMan.
The fileOffset argument is meant for class TiledFileAccess.

casacore::TSMCube::TSMCube ( TiledStMan stman,
AipsIO ios,
Bool  useDerived = False 
)

Reconstruct the hypercube by reading its data from the AipsIO stream.

It will link itself to the correct TSMFile. The TSMFile objects must have been reconstructed in advance.

virtual casacore::TSMCube::~TSMCube (  )  [virtual]
casacore::TSMCube::TSMCube ( const TSMCube  )  [private]

Forbid copy constructor.


Member Function Documentation

void casacore::TSMCube::accessLine ( char *  section,
uInt  pixelOffset,
uInt  localPixelSize,
Bool  writeFlag,
BucketCache cachePtr,
const IPosition startTile,
uInt  endTile,
const IPosition startPixelInFirstTile,
uInt  endPixelInLastTile,
uInt  lineIndex 
) [private]

Access a line in a more optimized way.

virtual void casacore::TSMCube::accessSection ( const IPosition start,
const IPosition end,
char *  section,
uInt  colnr,
uInt  localPixelSize,
uInt  externalPixelSize,
Bool  writeFlag 
) [virtual]

Read or write a section in the cube.

It is assumed that the section buffer is long enough.

Reimplemented in casacore::TSMCubeBuff, and casacore::TSMCubeMMap.

virtual void casacore::TSMCube::accessStrided ( const IPosition start,
const IPosition end,
const IPosition stride,
char *  section,
uInt  colnr,
uInt  localPixelSize,
uInt  externalPixelSize,
Bool  writeFlag 
) [virtual]

Read or write a section in a strided way.

It is assumed that the section buffer is long enough.

Reimplemented in casacore::TSMCubeBuff, and casacore::TSMCubeMMap.

IPosition casacore::TSMCube::adjustTileShape ( const IPosition cubeShape,
const IPosition tileShape 
) const [protected]

Adjust the tile shape to the hypercube shape.

A size of 0 gets set to 1. A tile size > cube size gets set to the cube size.

uInt casacore::TSMCube::bucketSize (  )  const [inline]

Get the bucket size (which is the length of a tile in external format).

Definition at line 428 of file TSMCube.h.

References bucketSize_p.

uInt casacore::TSMCube::cacheSize (  )  const

Get the current cache size (in buckets).

Referenced by casacore::TiledFileAccess::cacheSize().

static uInt casacore::TSMCube::calcCacheSize ( const IPosition cubeShape,
const IPosition tileShape,
Bool  extensible,
const IPosition sliceShape,
const IPosition windowStart,
const IPosition windowLength,
const IPosition axisPath,
uInt  maxCacheSize,
uInt  bucketSize 
) [static]
uInt casacore::TSMCube::calcCacheSize ( const IPosition sliceShape,
const IPosition windowStart,
const IPosition windowLength,
const IPosition axisPath 
) const

Calculate the cache size (in buckets) for the given slice and access path.

IPosition casacore::TSMCube::cellShape (  )  const

Get the shape of the data cells in the cube.

void casacore::TSMCube::clearCache ( Bool  doFlush = True  ) 

Clear the cache, so data will be reread.

If wanted, the data is flushed before the cache is cleared.

uInt casacore::TSMCube::coordinateSize ( const String coordinateName  )  const

Get the size of a coordinate (i.e.

the number of values in it). If not defined, it returns zero.

const IPosition & casacore::TSMCube::cubeShape (  )  const [inline]

Get the shape of the hypercube.

Definition at line 436 of file TSMCube.h.

References cubeShape_p.

Referenced by casacore::TiledFileAccess::shape().

virtual void casacore::TSMCube::deleteCache (  )  [private, virtual]

Delete the cache object.

Reimplemented in casacore::TSMCubeBuff, and casacore::TSMCubeMMap.

static void casacore::TSMCube::deleteCallBack ( void *  owner,
char *  buffer 
) [static, private]
void casacore::TSMCube::emptyCache (  ) 

Empty the cache.

It will flush the cache as needed and remove all buckets from it resulting in a possibly large drop in memory used. It'll also clear the userSetCache_p flag.

Referenced by casacore::TiledFileAccess::clearCache().

virtual void casacore::TSMCube::extend ( uInt  nr,
const Record coordValues,
const TSMColumn lastCoordColumn 
) [virtual]

Extend the last dimension of the cube with the given number.

The record can contain the coordinates of the elements added.

Reimplemented in casacore::TSMCubeBuff, and casacore::TSMCubeMMap.

void casacore::TSMCube::extendCoordinates ( const Record coordValues,
const String coordName,
uInt  length 
)

Extend the coordinates vector for the given coordinate to the given length with the given coordValues.

It will be initialized to zero if no coordValues are given. If the coordinate vector does not exist yet, it will be created.

virtual void casacore::TSMCube::flushCache (  )  [virtual]

Flush the data in the cache.

Reimplemented in casacore::TSMCubeBuff, and casacore::TSMCubeMMap.

Referenced by casacore::TiledFileAccess::flush().

BucketCache * casacore::TSMCube::getCache (  )  [inline, private]

Get the cache object.

This will construct the cache object if not present yet.

Reimplemented in casacore::TSMCubeBuff, and casacore::TSMCubeMMap.

Definition at line 421 of file TSMCube.h.

References cache_p, and makeCache().

TSMCube::AccessType casacore::TSMCube::getLastColAccess (  )  const [inline]

Functions for TSMDataColumn to keep track of the last type of access to a hypercube.

It uses it to determine if the cache has to be reset.

Definition at line 456 of file TSMCube.h.

References lastColAccess_p.

const IPosition & casacore::TSMCube::getLastColSlice (  )  const [inline]

Definition at line 460 of file TSMCube.h.

References lastColSlice_p.

Int casacore::TSMCube::getObject ( AipsIO ios  ) 

Get the data of the object from the AipsIO stream.

It returns the data manager sequence number, which is -1 if no file is attached to the cube (for cells without a value).

static char* casacore::TSMCube::initCallBack ( void *  owner  )  [static, private]
Bool casacore::TSMCube::isExtensible (  )  const

Is the hypercube extensible?

uInt casacore::TSMCube::localTileLength (  )  const [inline]

Get the length of a tile in local format.

Definition at line 432 of file TSMCube.h.

References localTileLength_p.

virtual void casacore::TSMCube::makeCache (  )  [private, virtual]

Construct the cache object (if not constructed yet).

Reimplemented in casacore::TSMCubeBuff, and casacore::TSMCubeMMap.

Referenced by getCache().

Bool casacore::TSMCube::matches ( const PtrBlock< TSMColumn * > &  idColSet,
const Record idValues 
)

Test if the id values match.

TSMCube& casacore::TSMCube::operator= ( const TSMCube  )  [private]

Forbid assignment.

Reimplemented in casacore::TSMCubeBuff, and casacore::TSMCubeMMap.

void casacore::TSMCube::putObject ( AipsIO ios  ) 

Put the data of the object into the AipsIO stream.

static char* casacore::TSMCube::readCallBack ( void *  owner,
const char *  external 
) [static, private]

Define the callback functions for the BucketCache.

char* casacore::TSMCube::readTile ( const char *  external  )  [private]

Define the functions doing the actual read and write of the data in the tile and converting it to/from local format.

void casacore::TSMCube::resizeTileSections (  )  [protected]

Resize the IPosition member variables used in accessSection() if nrdim_p changes value.

virtual void casacore::TSMCube::resync ( AipsIO ios  )  [virtual]

Resync the object with the data file.

It reads the object, and adjusts the cache.

virtual void casacore::TSMCube::resyncCache (  )  [private, virtual]

Resync the cache object.

Reimplemented in casacore::TSMCubeBuff, and casacore::TSMCubeMMap.

Record & casacore::TSMCube::rwValueRecord (  )  [inline]

Definition at line 448 of file TSMCube.h.

References values_p.

virtual void casacore::TSMCube::setCacheSize ( uInt  cacheSize,
Bool  forceSmaller,
Bool  userSet 
) [virtual]

Resize the cache object.

If forceSmaller is False, the cache will only be resized when it grows. If the given size exceeds the maximum size with more than 10%, the maximum size will be used. The cacheSize has to be given in buckets.
The flag userSet inidicates if the cache size is set by the user (by an Accessor object) or automatically (by TSMDataColumn).

Reimplemented in casacore::TSMCubeBuff, and casacore::TSMCubeMMap.

virtual void casacore::TSMCube::setCacheSize ( const IPosition sliceShape,
const IPosition windowStart,
const IPosition windowLength,
const IPosition axisPath,
Bool  forceSmaller,
Bool  userSet 
) [virtual]

Set the cache size for the given slice and access path.

Reimplemented in casacore::TSMCubeBuff, and casacore::TSMCubeMMap.

Referenced by casacore::TiledFileAccess::setCacheSize().

void casacore::TSMCube::setLastColAccess ( TSMCube::AccessType  type  )  [inline]

Definition at line 464 of file TSMCube.h.

References lastColAccess_p.

void casacore::TSMCube::setLastColSlice ( const IPosition slice  )  [inline]
virtual void casacore::TSMCube::setShape ( const IPosition cubeShape,
const IPosition tileShape 
) [virtual]

Set the hypercube shape.

This is only possible if the shape was not defined yet.

Reimplemented in casacore::TSMCubeBuff, and casacore::TSMCubeMMap.

void casacore::TSMCube::setup (  )  [protected]

Initialize the various variables.

void casacore::TSMCube::setupNrTiles (  )  [protected]
virtual void casacore::TSMCube::showCacheStatistics ( ostream &  os  )  const [virtual]

Show the cache statistics.

Reimplemented in casacore::TSMCubeBuff, and casacore::TSMCubeMMap.

Referenced by casacore::TiledFileAccess::showCacheStatistics().

const IPosition & casacore::TSMCube::tileShape (  )  const [inline]

Get the shape of the tiles.

Definition at line 440 of file TSMCube.h.

References tileShape_p.

Referenced by casacore::TiledFileAccess::tileShape().

Bool casacore::TSMCube::userSetCache (  )  const [inline]

Determine if the user set the cache size (using setCacheSize).

Definition at line 452 of file TSMCube.h.

References userSetCache_p.

static uInt casacore::TSMCube::validateCacheSize ( uInt  cacheSize,
uInt  maxSize,
uInt  bucketSize 
) [static]
uInt casacore::TSMCube::validateCacheSize ( uInt  cacheSize  )  const

Validate the cache size (in buckets).

This means it will return the given cache size if smaller than the maximum cache size. Otherwise the maximum is returned.

const Record & casacore::TSMCube::valueRecord (  )  const [inline]

Get the record containing the id and coordinate values.

It is used by TSMIdColumn and TSMCoordColumn.

Definition at line 444 of file TSMCube.h.

References values_p.

static void casacore::TSMCube::writeCallBack ( void *  owner,
char *  external,
const char *  local 
) [static, private]
void casacore::TSMCube::writeTile ( char *  external,
const char *  local 
) [private]

Member Data Documentation

The bucket size in bytes (is equal to tile size in bytes).

Definition at line 390 of file TSMCube.h.

Referenced by bucketSize().

The bucket cache.

Reimplemented in casacore::TSMCubeBuff, and casacore::TSMCubeMMap.

Definition at line 394 of file TSMCube.h.

Referenced by getCache().

Definition at line 353 of file TSMCube.h.

The shape of the hypercube.

Definition at line 368 of file TSMCube.h.

Referenced by cubeShape().

Last pixel in first tile.

Definition at line 414 of file TSMCube.h.

Last pixel in last tile.

Definition at line 416 of file TSMCube.h.

Last tile needed.

Definition at line 410 of file TSMCube.h.

Precomputed tileShape information.

Definition at line 374 of file TSMCube.h.

Precomputed tilesPerDim information.

Definition at line 376 of file TSMCube.h.

Is the hypercube extensible?

Definition at line 362 of file TSMCube.h.

Offset for each data column in a tile (in external format).

Definition at line 386 of file TSMCube.h.

Offset in the TSMFile object where the data of this hypercube starts.

Definition at line 384 of file TSMCube.h.

Pointer to the TSMFile object holding the data.

Definition at line 382 of file TSMCube.h.

Was the last column access to a cell, slice, or column?

Definition at line 398 of file TSMCube.h.

Referenced by getLastColAccess(), and setLastColAccess().

The slice shape of the last column access to a slice.

Definition at line 400 of file TSMCube.h.

Referenced by getLastColSlice(), and setLastColSlice().

Offset for each data column in a tile (in local format).

Definition at line 388 of file TSMCube.h.

The tile size in bytes in local format.

Definition at line 392 of file TSMCube.h.

Referenced by localTileLength().

Dimensionality of the hypercube.

Definition at line 364 of file TSMCube.h.

Number of tiles in the hypercube.

Definition at line 366 of file TSMCube.h.

IPosition variables used in accessSection(); declared here as member variables to avoid significant construction and desctruction overhead if they are local to accessSection() #tiles needed for the section.

Definition at line 406 of file TSMCube.h.

Number of tiles in all but last dimension (used when extending).

Definition at line 378 of file TSMCube.h.

First pixel in first tile.

Definition at line 412 of file TSMCube.h.

First tile needed.

Definition at line 408 of file TSMCube.h.

Pointer to the parent storage manager.

Definition at line 356 of file TSMCube.h.

The shape of the tiles in the hypercube.

Definition at line 370 of file TSMCube.h.

Referenced by tileShape().

The tilesize in pixels.

Definition at line 380 of file TSMCube.h.

The number of tiles in each hypercube dimension.

Definition at line 372 of file TSMCube.h.

Is the class used directly or only by a derived class only?

Definition at line 358 of file TSMCube.h.

Did the user set the cache size?

Definition at line 396 of file TSMCube.h.

Referenced by userSetCache().

The values of the possible id and coordinate columns.

Definition at line 360 of file TSMCube.h.

Referenced by rwValueRecord(), and valueRecord().


The documentation for this class was generated from the following file:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines

Generated on 31 Aug 2016 for casa by  doxygen 1.6.1