casacore::StatisticsAlgorithm< AccumType, DataIterator, MaskIterator, WeightsIterator > Class Template Reference

Base class of statistics algorithm class hierarchy. More...

#include <StatisticsAlgorithm.h>

Inheritance diagram for casacore::StatisticsAlgorithm< AccumType, DataIterator, MaskIterator, WeightsIterator >:

Public Member Functions
virtual	~StatisticsAlgorithm ()
virtual void	addData (const DataIterator &first, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False)
	Add a dataset to an existing set of datasets on which statistics are to be calculated.
virtual void	addData (const DataIterator &first, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False)
virtual void	addData (const DataIterator &first, const MaskIterator &maskFirst, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1)
virtual void	addData (const DataIterator &first, const MaskIterator &maskFirst, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1)
virtual void	addData (const DataIterator &first, const WeightsIterator &weightFirst, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False)
virtual void	addData (const DataIterator &first, const WeightsIterator &weightFirst, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False)
virtual void	addData (const DataIterator &first, const WeightsIterator &weightFirst, const MaskIterator &maskFirst, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1)
virtual void	addData (const DataIterator &first, const WeightsIterator &weightFirst, const MaskIterator &maskFirst, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1)
virtual StatisticsData::ALGORITHM	algorithm () const =0
	get the algorithm that this object uses for computing stats
void	deleteSortedArray ()
	delete any (partially) sorted array
virtual AccumType	getMedian (CountedPtr< uInt64 > knownNpts=NULL, CountedPtr< AccumType > knownMin=NULL, CountedPtr< AccumType > knownMax=NULL, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt64 nBins=10000)=0
virtual AccumType	getMedianAndQuantiles (std::map< Double, AccumType > &quantileToValue, const std::set< Double > &quantiles, CountedPtr< uInt64 > knownNpts=NULL, CountedPtr< AccumType > knownMin=NULL, CountedPtr< AccumType > knownMax=NULL, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt64 nBins=10000)=0
	The return value is the median; the quantiles are returned in the `quantileToValue` map.
virtual AccumType	getMedianAbsDevMed (CountedPtr< uInt64 > knownNpts=NULL, CountedPtr< AccumType > knownMin=NULL, CountedPtr< AccumType > knownMax=NULL, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt64 nBins=10000)=0
	get the median of the absolute deviation about the median of the data.
AccumType	getQuantile (Double quantile, CountedPtr< uInt64 > knownNpts=NULL, CountedPtr< AccumType > knownMin=NULL, CountedPtr< AccumType > knownMax=NULL, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt64 nBins=10000)
virtual std::map< Double, AccumType >	getQuantiles (const std::set< Double > &quantiles, CountedPtr< uInt64 > npts=NULL, CountedPtr< AccumType > min=NULL, CountedPtr< AccumType > max=NULL, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt64 nBins=10000)=0
	get a map of quantiles to values.
virtual AccumType	getStatistic (StatisticsData::STATS stat)
	get the value of the specified statistic
virtual std::pair< Int64, Int64 >	getStatisticIndex (StatisticsData::STATS stat)=0
	certain statistics such as max and min have locations in the dataset associated with them.
virtual StatsData< AccumType >	getStatistics ()
virtual void	setData (const DataIterator &first, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False)
	setdata() clears any current datasets or data provider and then adds the specified data set as the first dataset in the (possibly new) set of data sets for which statistics are to be calculated.
virtual void	setData (const DataIterator &first, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False)
virtual void	setData (const DataIterator &first, const MaskIterator &maskFirst, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1)
virtual void	setData (const DataIterator &first, const MaskIterator &maskFirst, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1)
virtual void	setData (const DataIterator &first, const WeightsIterator &weightFirst, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False)
virtual void	setData (const DataIterator &first, const WeightsIterator &weightFirst, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False)
virtual void	setData (const DataIterator &first, const WeightsIterator &weightFirst, const MaskIterator &maskFirst, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1)
virtual void	setData (const DataIterator &first, const WeightsIterator &weightFirst, const MaskIterator &maskFirst, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1)
virtual void	setDataProvider (StatsDataProvider< CASA_STATP > *dataProvider)
	instead of settng and adding data "by hand", set the data provider that will provide all the data sets.
virtual void	setStatsToCalculate (std::set< StatisticsData::STATS > &stats)
	Provide guidance to algorithms by specifying a priori which statistics the caller would like calculated.
Protected Member Functions
	StatisticsAlgorithm ()
StatisticsAlgorithm< CASA_STATP > &	operator= (const StatisticsAlgorithm< CASA_STATP > &other)
	use copy semantics
virtual void	_addData ()
	Allows derived classes to do things after data is set or added.
virtual void	_clearData ()
const vector< Int64 > &	_getCounts () const
const vector< DataIterator > &	_getData () const
StatsDataProvider< CASA_STATP > *	_getDataProvider ()
const vector< uInt > &	_getDataStrides () const
const std::map< uInt, Bool > &	_getIsIncludeRanges () const
const std::map< uInt, MaskIterator >	_getMasks () const
const std::map< uInt, uInt > &	_getMaskStrides () const
const std::map< uInt, DataRanges > &	_getRanges () const
virtual AccumType	_getStatistic (StatisticsData::STATS stat)=0
virtual StatsData< AccumType >	_getStatistics ()=0
const std::set < StatisticsData::STATS >	_getStatsToCalculate () const
std::vector< AccumType > &	_getSortedArray ()
virtual const std::set < StatisticsData::STATS > &	_getUnsupportedStatistics () const
const std::map< uInt, WeightsIterator > &	_getWeights () const
void	_setSortedArray (const vector< AccumType > &v)
Static Protected Member Functions
static std::map< Double, uInt64 >	_indicesFromQuantiles (uInt64 npts, const std::set< Double > &quantiles)
*static std::map< uInt64, AccumType >	_valuesFromArray (vector< AccumType > &myArray, const std::set< uInt64 > &indices)
	The array can be changed by paritally sorting it up to the largest index.
Private Member Functions
void	_throwIfDataProviderDefined () const
Private Attributes
vector< DataIterator >	_data
std::map< uInt, WeightsIterator >	_weights
	maps data to weights
std::map< uInt, MaskIterator >	_masks
	maps data to masks
vector< Int64 >	_counts
vector< uInt >	_dataStrides
std::map< uInt, uInt >	_maskStrides
std::map< uInt, Bool >	_isIncludeRanges
std::map< uInt, DataRanges >	_dataRanges
vector< AccumType >	_sortedArray
std::set< StatisticsData::STATS >	_statsToCalculate
std::set< StatisticsData::STATS >	_unsupportedStats
StatsDataProvider< CASA_STATP > *	_dataProvider

Detailed Description

template<class AccumType, class DataIterator, class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
class casacore::StatisticsAlgorithm< AccumType, DataIterator, MaskIterator, WeightsIterator >

Base class of statistics algorithm class hierarchy.

The default implementation is such that statistics are only calculated when getStatistic() or getStatistics() is called. Until then, the iterators which point to the beginning of data sets, masks, etc. are held in memory. Thus, the caller must keep all data sets available for the statistics object until these methods are called, and of course, if the actual data values are changed between adding data and calculating statistics, the updated values are used when calculating statistics. Derived classes may override this behavior.

PRECISION CONSIDERATIONS Many statistics are computed via accumulators. This can lead to precision issues, especially for large datasets. For this reason, it is highly recommended that the data type one uses as the AccumType be of higher precision, if possible, than the data type pointed to by input iterator. So for example, if one has a data set of Float values (to which the InputIterator type points to), then one should use type Double for the AccumType. In this case, the Float data values will be converted to Doubles before they are accumulated.

METHODS OF PROVIDING DATA Data may be provided in one of two mutually exclusive ways. The first way is simpler, and that is to use the setData()/addData() methods. Calling setData() will clear any previous data that was added via these methods or via a data provider (see below). Calling addData() subsequently to setData() will add a data set to the set of data sets on which statistics will be calculated. In order for this to work correctly, the iterators which are passed into these methods must still be valid when statistics are calculated (although note that some derived classes allow certain statistics to be updated as data sets are added via these methods. See specific classes for details).

The second way to provide data is via a data provider. This takes the form of a derived class of StatsDataProvider, in which various methods are implemented for retrieving various information about the data sets. Such an interface is necessary for data which does not easily lend itself to be provided via the setData()/addData() methods. For example, in the case of iterating through a lattice, a lattice iterator will overwrite the memory location of the previous chunk of data with the current chunk of data. Therefore, if one does not wish to load data from the entire lattice into memory (which is why LatticeIterator was designed in this way), one must the LatticeStatsDataProvider class, which the statistics framework will use to iteratate through the lattice, only keeping one chunk of the data of the lattice in memory at one time.

QUANTILES A quantile is a value contained in a data set, such that, it has a zero-based index of ceil(q*n)-1 in the equivalent ordered dataset, where 0 < q < 1 specifies the fractional location within the ordered dataset and n is the total number of elements. Note that, for a dataset with an odd number of elements, the median is the same as the quantile value when q = 0.5. However, there is no such correspondance between the median in a dataset with an even number of elements, since the median in that case is given by the mean of the elements of zero-based indeces n/2-1 and n/2 in the equivalent ordered dataset. Thus, in the case of a dataset with an even number of values, the median may not even exist in the dataset, while a quantile value must exist in the dataset. Note when calculating quantile values, a dataset that does not fall in specified dataset ranges, is not included via a stride specification, is masked, or has a weight of zero is not considered a member of the dataset for the pruposes of quantile calculations.

Definition at line 105 of file StatisticsAlgorithm.h.