Class to calculate statistics in a "classical" sense, ie using accumulators with no special filtering beyond optional range filtering etc. More...
#include <ClassicalStatistics.h>
Public Member Functions | |
ClassicalStatistics () | |
ClassicalStatistics (const ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator > &cs) | |
copy semantics | |
virtual | ~ClassicalStatistics () |
ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator > & | operator= (const ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator > &other) |
copy semantics | |
virtual StatisticsData::ALGORITHM | algorithm () const |
get the algorithm that this object uses for computing stats | |
virtual AccumType | getMedian (CountedPtr< uInt64 > knownNpts=NULL, CountedPtr< AccumType > knownMin=NULL, CountedPtr< AccumType > knownMax=NULL, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt64 nBins=10000) |
In the following group of methods, if the size of the composite dataset is smaller than binningThreshholdSizeBytes , the composite dataset will be (perhaps partially) sorted and persisted in memory during the call. | |
virtual AccumType | getMedianAndQuantiles (std::map< Double, AccumType > &quantiles, const std::set< Double > &fractions, CountedPtr< uInt64 > knownNpts=NULL, CountedPtr< AccumType > knownMin=NULL, CountedPtr< AccumType > knownMax=NULL, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt64 nBins=10000) |
If one needs to compute both the median and quantile values, it is better to call getMedianAndQuantiles() rather than getMedian() and getQuantiles() separately, as the first will scan large data sets fewer times than calling the separate methods. | |
virtual AccumType | getMedianAbsDevMed (CountedPtr< uInt64 > knownNpts=NULL, CountedPtr< AccumType > knownMin=NULL, CountedPtr< AccumType > knownMax=NULL, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt64 nBins=10000) |
get the median of the absolute deviation about the median of the data. | |
virtual std::map< Double, AccumType > | getQuantiles (const std::set< Double > &fractions, CountedPtr< uInt64 > knownNpts=NULL, CountedPtr< AccumType > knownMin=NULL, CountedPtr< AccumType > knownMax=NULL, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt64 nBins=10000) |
Get the specified quantiles. | |
virtual void | getMinMax (AccumType &mymin, AccumType &mymax) |
scan the dataset(s) that have been added, and find the min and max. | |
virtual uInt64 | getNPts () |
scan the dataset(s) that have been added, and find the number of good points. | |
virtual std::pair< Int64, Int64 > | getStatisticIndex (StatisticsData::STATS stat) |
see base class description | |
Bool | hasData () const |
Has any data been added to this object? Will return False if the object has been reset and no data have been added afterward. | |
virtual void | reset () |
reset object to initial state. | |
virtual void | setCalculateAsAdded (Bool c) |
Should statistics be updated with calls to addData or should they only be calculated upon calls to getStatistics etc? Beware that calling this will automatically reinitialize the object, so that it will contain no references to data et al. | |
void | setDataProvider (StatsDataProvider< AccumType, DataIterator, MaskIterator, WeightsIterator > *dataProvider) |
An exception will be thrown if setCalculateAsAdded(True) has been called. | |
void | setStatsToCalculate (std::set< StatisticsData::STATS > &stats) |
Provide guidance to algorithms by specifying a priori which statistics the caller would like calculated. | |
Protected Member Functions | |
virtual void | _accumNpts (uInt64 &npts, const DataIterator &dataBegin, Int64 nr, uInt dataStride) const |
scan through the data set to determine the number of good (unmasked, weight > 0, within range) points. | |
virtual void | _accumNpts (uInt64 &npts, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const |
virtual void | _accumNpts (uInt64 &npts, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride) const |
virtual void | _accumNpts (uInt64 &npts, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const |
virtual void | _accumNpts (uInt64 &npts, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, Int64 nr, uInt dataStride) const |
virtual void | _accumNpts (uInt64 &npts, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, Int64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const |
virtual void | _accumNpts (uInt64 &npts, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const |
virtual void | _accumNpts (uInt64 &npts, const DataIterator &dataBegin, const WeightsIterator &weightBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride) const |
void | _accumulate (StatsData< AccumType > &stats, const AccumType &datum, const LocationType &location) |
void | _accumulate (StatsData< AccumType > &stats, const AccumType &datum, const AccumType &weight, const LocationType &location) |
void | _addData () |
Allows derived classes to do things after data is set or added. | |
void | _clearData () |
void | _clearStats () |
void | _doMinMax (AccumType &vmin, AccumType &vmax) |
scan dataset(s) to find min and max | |
virtual void | _findBins (vector< vector< uInt64 > > &binCounts, vector< CountedPtr< AccumType > > &sameVal, vector< Bool > &allSame, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const vector< typename StatisticsUtilities< AccumType >::BinDesc > &binDesc, const vector< AccumType > &maxLimit) const |
Get the counts of data within the specified histogram bins. | |
virtual void | _findBins (vector< vector< uInt64 > > &binCounts, vector< CountedPtr< AccumType > > &sameVal, vector< Bool > &allSame, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude, const vector< typename StatisticsUtilities< AccumType >::BinDesc > &binDesc, const vector< AccumType > &maxLimit) const |
virtual void | _findBins (vector< vector< uInt64 > > &binCounts, vector< CountedPtr< AccumType > > &sameVal, vector< Bool > &allSame, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const vector< typename StatisticsUtilities< AccumType >::BinDesc > &binDesc, const vector< AccumType > &maxLimit) const |
virtual void | _findBins (vector< vector< uInt64 > > &binCounts, vector< CountedPtr< AccumType > > &sameVal, vector< Bool > &allSame, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude, const vector< typename StatisticsUtilities< AccumType >::BinDesc > &binDesc, const vector< AccumType > &maxLimit) const |
virtual void | _findBins (vector< vector< uInt64 > > &binCounts, vector< CountedPtr< AccumType > > &sameVal, vector< Bool > &allSame, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, Int64 nr, uInt dataStride, const vector< typename StatisticsUtilities< AccumType >::BinDesc > &binDesc, const vector< AccumType > &maxLimit) const |
virtual void | _findBins (vector< vector< uInt64 > > &binCounts, vector< CountedPtr< AccumType > > &sameVal, vector< Bool > &allSame, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, Int64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude, const vector< typename StatisticsUtilities< AccumType >::BinDesc > &binDesc, const vector< AccumType > &maxLimit) const |
virtual void | _findBins (vector< vector< uInt64 > > &binCounts, vector< CountedPtr< AccumType > > &sameVal, vector< Bool > &allSame, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude, const vector< typename StatisticsUtilities< AccumType >::BinDesc > &binDesc, const vector< AccumType > &maxLimit) const |
virtual void | _findBins (vector< vector< uInt64 > > &binCounts, vector< CountedPtr< AccumType > > &sameVal, vector< Bool > &allSame, const DataIterator &dataBegin, const WeightsIterator &weightBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const vector< typename StatisticsUtilities< AccumType >::BinDesc > &binDesc, const vector< AccumType > &maxLimit) const |
Bool | _getDoMaxMin () const |
Bool | _getIDataset () const |
virtual StatsData< AccumType > | _getInitialStats () const |
AccumType | _getStatistic (StatisticsData::STATS stat) |
StatsData< AccumType > | _getStatistics () |
virtual StatsData< AccumType > & | _getStatsData () |
retreive stats structure. | |
virtual const StatsData < AccumType > & | _getStatsData () const |
virtual void | _minMax (CountedPtr< AccumType > &mymin, CountedPtr< AccumType > &mymax, const DataIterator &dataBegin, Int64 nr, uInt dataStride) const |
virtual void | _minMax (CountedPtr< AccumType > &mymin, CountedPtr< AccumType > &mymax, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const |
virtual void | _minMax (CountedPtr< AccumType > &mymin, CountedPtr< AccumType > &mymax, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride) const |
virtual void | _minMax (CountedPtr< AccumType > &mymin, CountedPtr< AccumType > &mymax, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const |
virtual void | _minMax (CountedPtr< AccumType > &mymin, CountedPtr< AccumType > &mymax, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, Int64 nr, uInt dataStride) const |
virtual void | _minMax (CountedPtr< AccumType > &mymin, CountedPtr< AccumType > &mymax, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, Int64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const |
virtual void | _minMax (CountedPtr< AccumType > &mymin, CountedPtr< AccumType > &mymax, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const |
virtual void | _minMax (CountedPtr< AccumType > &mymin, CountedPtr< AccumType > &mymax, const DataIterator &dataBegin, const WeightsIterator &weightBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride) const |
virtual void | _populateArray (vector< AccumType > &ary, const DataIterator &dataBegin, Int64 nr, uInt dataStride) const |
populate an unsorted array with valid data. | |
virtual void | _populateArray (vector< AccumType > &ary, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const |
ranges | |
virtual void | _populateArray (vector< AccumType > &ary, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride) const |
virtual void | _populateArray (vector< AccumType > &ary, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const |
mask and ranges | |
virtual void | _populateArray (vector< AccumType > &ary, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, Int64 nr, uInt dataStride) const |
weights | |
virtual void | _populateArray (vector< AccumType > &ary, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, Int64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const |
weights and ranges | |
virtual void | _populateArray (vector< AccumType > &ary, const DataIterator &dataBegin, const WeightsIterator &weightBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride) const |
weights and mask | |
virtual void | _populateArray (vector< AccumType > &ary, const DataIterator &dataBegin, const WeightsIterator &weightBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const |
weights, mask, ranges | |
virtual void | _populateArrays (vector< vector< AccumType > > &arys, uInt64 ¤tCount, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const vector< std::pair< AccumType, AccumType > > &includeLimits, uInt64 maxCount) const |
Create a vector of unsorted arrays, one array for each bin defined by includeLimits . | |
virtual void | _populateArrays (vector< vector< AccumType > > &arys, uInt64 ¤tCount, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude, const vector< std::pair< AccumType, AccumType > > &includeLimits, uInt64 maxCount) const |
ranges | |
virtual void | _populateArrays (vector< vector< AccumType > > &arys, uInt64 ¤tCount, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const vector< std::pair< AccumType, AccumType > > &includeLimits, uInt64 maxCount) const |
virtual void | _populateArrays (vector< vector< AccumType > > &arys, uInt64 ¤tCount, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude, const vector< std::pair< AccumType, AccumType > > &includeLimits, uInt64 maxCount) const |
mask and ranges | |
virtual void | _populateArrays (vector< vector< AccumType > > &arys, uInt64 ¤tCount, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, Int64 nr, uInt dataStride, const vector< std::pair< AccumType, AccumType > > &includeLimits, uInt64 maxCount) const |
weights | |
virtual void | _populateArrays (vector< vector< AccumType > > &arys, uInt64 ¤tCount, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, Int64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude, const vector< std::pair< AccumType, AccumType > > &includeLimits, uInt64 maxCount) const |
weights and ranges | |
virtual void | _populateArrays (vector< vector< AccumType > > &arys, uInt64 ¤tCount, const DataIterator &dataBegin, const WeightsIterator &weightBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const vector< std::pair< AccumType, AccumType > > &includeLimits, uInt64 maxCount) const |
weights and mask | |
virtual void | _populateArrays (vector< vector< AccumType > > &arys, uInt64 ¤tCount, const DataIterator &dataBegin, const WeightsIterator &weightBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude, const vector< std::pair< AccumType, AccumType > > &includeLimits, uInt64 maxCount) const |
weights, mask, ranges | |
virtual Bool | _populateTestArray (vector< AccumType > &ary, const DataIterator &dataBegin, Int64 nr, uInt dataStride, uInt maxElements) const |
no weights, no mask, no ranges | |
virtual Bool | _populateTestArray (vector< AccumType > &ary, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude, uInt maxElements) const |
ranges | |
virtual Bool | _populateTestArray (vector< AccumType > &ary, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, uInt maxElements) const |
mask | |
virtual Bool | _populateTestArray (vector< AccumType > &ary, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude, uInt maxElements) const |
mask and ranges | |
virtual Bool | _populateTestArray (vector< AccumType > &ary, const DataIterator &dataBegin, const WeightsIterator &weightBegin, Int64 nr, uInt dataStride, uInt maxElements) const |
weights | |
virtual Bool | _populateTestArray (vector< AccumType > &ary, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, Int64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude, uInt maxElements) const |
weights and ranges | |
virtual Bool | _populateTestArray (vector< AccumType > &ary, const DataIterator &dataBegin, const WeightsIterator &weightBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, uInt maxElements) const |
weights and mask | |
virtual Bool | _populateTestArray (vector< AccumType > &ary, const DataIterator &dataBegin, const WeightsIterator &weightBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude, uInt maxElements) const |
weights, mask, ranges | |
virtual void | _unweightedStats (StatsData< AccumType > &stats, uInt64 &ngood, LocationType &location, const DataIterator &dataBegin, Int64 nr, uInt dataStride) |
no weights, no mask, no ranges | |
virtual void | _unweightedStats (StatsData< AccumType > &stats, uInt64 &ngood, LocationType &location, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) |
no weights, no mask | |
virtual void | _unweightedStats (StatsData< AccumType > &stats, uInt64 &ngood, LocationType &location, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride) |
virtual void | _unweightedStats (StatsData< AccumType > &stats, uInt64 &ngood, LocationType &location, const DataIterator &dataBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) |
virtual void | _updateDataProviderMaxMin (const StatsData< AccumType > &threadStats) |
virtual void | _weightedStats (StatsData< AccumType > &stats, LocationType &location, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, Int64 nr, uInt dataStride) |
has weights, but no mask, no ranges | |
virtual void | _weightedStats (StatsData< AccumType > &stats, LocationType &location, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, Int64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) |
virtual void | _weightedStats (StatsData< AccumType > &stats, LocationType &location, const DataIterator &dataBegin, const WeightsIterator &weightBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride) |
virtual void | _weightedStats (StatsData< AccumType > &stats, LocationType &location, const DataIterator &dataBegin, const WeightsIterator &weightBegin, Int64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) |
Private Member Functions | |
vector< vector< uInt64 > > | _binCounts (vector< CountedPtr< AccumType > > &sameVal, const vector< typename StatisticsUtilities< AccumType >::BinDesc > &binDesc) |
tally the number of data points that fall into each bin provided by binDesc Any points that are less than binDesc.minLimit or greater than binDesc.minLimit + binDesc.nBins*binDesc.binWidth are not included in the counts. | |
void | _computeBins (vector< vector< uInt64 > > &bins, vector< CountedPtr< AccumType > > &sameVal, vector< Bool > &allSame, DataIterator dataIter, MaskIterator maskIter, WeightsIterator weightsIter, uInt64 count, const vector< typename StatisticsUtilities< AccumType >::BinDesc > &binDesc, const vector< AccumType > &maxLimit) |
void | _computeDataArray (vector< AccumType > &ary, DataIterator dataIter, MaskIterator maskIter, WeightsIterator weightsIter, uInt64 dataCount) |
void | _computeDataArrays (vector< vector< AccumType > > &arys, uInt64 ¤tCount, DataIterator dataIter, MaskIterator maskIter, WeightsIterator weightsIter, uInt64 dataCount, const vector< std::pair< AccumType, AccumType > > &includeLimits, uInt64 maxCount) |
void | _computeMinMax (CountedPtr< AccumType > &mymax, CountedPtr< AccumType > &mymin, DataIterator dataIter, MaskIterator maskIter, WeightsIterator weightsIter, uInt64 dataCount) |
void | _computeStats (StatsData< AccumType > &stats, uInt64 &ngood, LocationType &location, DataIterator dataIter, MaskIterator maskIter, WeightsIterator weightsIter, uInt64 count) |
void | _createDataArray (vector< AccumType > &array) |
Create an unsorted array of the complete data set. | |
void | _createDataArrays (vector< vector< AccumType > > &arrays, const vector< std::pair< AccumType, AccumType > > &includeLimits, uInt64 maxCount) |
vector< std::map< uInt64, AccumType > > | _dataFromMultipleBins (const vector< typename StatisticsUtilities< AccumType >::BinDesc > &binDesc, uInt64 maxArraySize, const vector< std::set< uInt64 > > &dataIndices, uInt64 nBins) |
extract data from multiple histograms given by binDesc . | |
vector< std::map< uInt64, AccumType > > | _dataFromSingleBins (const vector< uInt64 > &binNpts, uInt64 maxArraySize, const vector< std::pair< AccumType, AccumType > > &binLimits, const vector< std::set< uInt64 > > &dataIndices, uInt64 nBins) |
Int64 | _doNpts () |
Bool | _increment (Bool includeIDataset) |
increment the relevant loop counters | |
void | _incrementThreadIters (DataIterator &dataIter, MaskIterator &maskIter, WeightsIterator &weightsIter, uInt64 &offset, uInt nthreads) const |
increment thread-based iterators | |
std::map< uInt64, AccumType > | _indicesToValues (CountedPtr< uInt64 > knownNpts, CountedPtr< AccumType > knownMin, CountedPtr< AccumType > knownMax, uInt64 maxArraySize, const std::set< uInt64 > &dataIndices, Bool persistSortedArray, uInt64 nBins) |
get the values for the specified indices in the sorted array of all good data | |
void | _initIterators () |
void | _initLoopVars () |
void | _initThreadVars (uInt &nBlocks, uInt64 &extra, uInt &nthreads, PtrHolder< DataIterator > &dataIter, PtrHolder< MaskIterator > &maskIter, PtrHolder< WeightsIterator > &weightsIter, PtrHolder< uInt64 > &offset, uInt nThreadsMax) const |
Bool | _isNptsSmallerThan (vector< AccumType > &arrayToSort, uInt maxArraySize) |
Determine by scanning the dataset if the number of good points is smaller than maxArraySize . | |
std::set< uInt64 > | _medianIndices (CountedPtr< uInt64 > knownNpts) |
get the index (for odd npts) or indices (for even npts) of the median of the sorted array. | |
uInt | _nThreadsMax () const |
uInt | _threadIdx () const |
Bool | _valuesFromSortedArray (std::map< uInt64, AccumType > &values, CountedPtr< uInt64 > knownNpts, const std::set< uInt64 > &indices, uInt64 maxArraySize, Bool persistSortedArray) |
get values from sorted array if the array is small enough to be held in memory. | |
Static Private Member Functions | |
static void | _convertToAbsDevMedArray (vector< AccumType > &myArray, AccumType median) |
convert in place by taking the absolute value of the difference of the vector and the median | |
static void | _makeBins (typename StatisticsUtilities< AccumType >::BinDesc &bins, AccumType minData, AccumType maxData, uInt maxBins, Bool allowPad) |
If allowPad is True, then pad the lower side of the lowest bin and the higher side of the highest bin so that minData and maxData do not fall on the edge of their respective bins. | |
static void | _mergeResults (vector< vector< uInt64 > > &bins, vector< CountedPtr< AccumType > > &sameVal, vector< Bool > &allSame, const PtrHolder< vector< vector< uInt64 > > > &tBins, const PtrHolder< vector< CountedPtr< AccumType > > > &tSameVal, const PtrHolder< vector< Bool > > &tAllSame, uInt nThreadsMax) |
Private Attributes | |
StatsData< AccumType > | _statsData |
Int64 | _idataset |
Bool | _calculateAsAdded |
Bool | _doMaxMin |
Bool | _doMedAbsDevMed |
Bool | _mustAccumulate |
Bool | _hasData |
vector< DataIterator > ::const_iterator | _dend |
mutables, used to mitigate repeated code | |
vector< DataIterator > ::const_iterator | _diter |
vector< Int64 >::const_iterator | _citer |
vector< uInt >::const_iterator | _dsiter |
std::map< uInt, MaskIterator > | _masks |
maps data to masks | |
uInt | _maskStride |
std::map< uInt, WeightsIterator > | _weights |
maps data to weights | |
std::map< uInt, DataRanges > | _ranges |
std::map< uInt, Bool > | _isIncludeRanges |
Bool | _hasMask |
Bool | _hasRanges |
Bool | _hasWeights |
Bool | _myIsInclude |
DataRanges | _myRanges |
MaskIterator | _myMask |
DataIterator | _myData |
WeightsIterator | _myWeights |
uInt | _dataCount |
uInt | _myStride |
uInt64 | _myCount |
Static Private Attributes | |
static const uInt | CACHE_PADDING |
static const uInt | BLOCK_SIZE |
Class to calculate statistics in a "classical" sense, ie using accumulators with no special filtering beyond optional range filtering etc.
setCalculateAsAdded() allows one to specify if statistics should be calculated and updated on upon each call to set/addData(). If False, statistics will be calculated only when getStatistic(), getStatistics(), or similar methods are called. Setting this value to True allows the caller to not have to keep all the data accessible at once. Note however, that all data must be simultaneously accessible if quantile (eg median) calculations are desired.
I attempted to write this class using the Composite design pattern, with eg the _unweightedStats() and _weightedStats() methods in their own class, but for reasons I don't understand, that impacted performance significantly. So I'm using the current architecture, which I know is a bit a maintenance nightmare.
Definition at line 60 of file ClassicalStatistics.h.
casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::ClassicalStatistics | ( | ) |
casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::ClassicalStatistics | ( | const ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator > & | cs | ) |
copy semantics
virtual casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::~ClassicalStatistics | ( | ) | [virtual] |
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_accumNpts | ( | uInt64 & | npts, | |
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride | |||
) | const [protected, virtual] |
Reimplemented in casacore::ConstrainedRangeStatistics< CASA_STATP >.
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_accumNpts | ( | uInt64 & | npts, | |
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightsBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude | |||
) | const [protected, virtual] |
Reimplemented in casacore::ConstrainedRangeStatistics< CASA_STATP >.
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_accumNpts | ( | uInt64 & | npts, | |
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightsBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude | |||
) | const [protected, virtual] |
Reimplemented in casacore::ConstrainedRangeStatistics< CASA_STATP >.
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_accumNpts | ( | uInt64 & | npts, | |
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightsBegin, | |||
Int64 | nr, | |||
uInt | dataStride | |||
) | const [protected, virtual] |
Reimplemented in casacore::ConstrainedRangeStatistics< CASA_STATP >.
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_accumNpts | ( | uInt64 & | npts, | |
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude | |||
) | const [protected, virtual] |
Reimplemented in casacore::ConstrainedRangeStatistics< CASA_STATP >.
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_accumNpts | ( | uInt64 & | npts, | |
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride | |||
) | const [protected, virtual] |
Reimplemented in casacore::ConstrainedRangeStatistics< CASA_STATP >.
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_accumNpts | ( | uInt64 & | npts, | |
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude | |||
) | const [protected, virtual] |
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_accumNpts | ( | uInt64 & | npts, | |
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride | |||
) | const [inline, protected, virtual] |
scan through the data set to determine the number of good (unmasked, weight > 0, within range) points.
The first with no mask, no ranges, and no weights is trivial with npts = nr in this class, but is implemented here so that derived classes may override it.
Reimplemented in casacore::ConstrainedRangeStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >, casacore::HingesFencesStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >, and casacore::ConstrainedRangeStatistics< CASA_STATP >.
void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_accumulate | ( | StatsData< AccumType > & | stats, | |
const AccumType & | datum, | |||
const AccumType & | weight, | |||
const LocationType & | location | |||
) | [inline, protected] |
void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_accumulate | ( | StatsData< AccumType > & | stats, | |
const AccumType & | datum, | |||
const LocationType & | location | |||
) | [inline, protected] |
void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_addData | ( | ) | [protected, virtual] |
Allows derived classes to do things after data is set or added.
Default implementation does nothing.
Reimplemented from casacore::StatisticsAlgorithm< AccumType, DataIterator, MaskIterator, WeightsIterator >.
vector<vector<uInt64> > casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_binCounts | ( | vector< CountedPtr< AccumType > > & | sameVal, | |
const vector< typename StatisticsUtilities< AccumType >::BinDesc > & | binDesc | |||
) | [private] |
tally the number of data points that fall into each bin provided by binDesc
Any points that are less than binDesc.minLimit or greater than binDesc.minLimit + binDesc.nBins*binDesc.binWidth are not included in the counts.
A data point that falls exactly on a bin boundary is considered to be in the higher index bin. sameVal
will be non-null if all the good values in the histogram range are the same. In that case, the value held will be the value of each of those data points.
void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_clearData | ( | ) | [protected, virtual] |
void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_clearStats | ( | ) | [protected] |
void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_computeBins | ( | vector< vector< uInt64 > > & | bins, | |
vector< CountedPtr< AccumType > > & | sameVal, | |||
vector< Bool > & | allSame, | |||
DataIterator | dataIter, | |||
MaskIterator | maskIter, | |||
WeightsIterator | weightsIter, | |||
uInt64 | count, | |||
const vector< typename StatisticsUtilities< AccumType >::BinDesc > & | binDesc, | |||
const vector< AccumType > & | maxLimit | |||
) | [private] |
void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_computeDataArray | ( | vector< AccumType > & | ary, | |
DataIterator | dataIter, | |||
MaskIterator | maskIter, | |||
WeightsIterator | weightsIter, | |||
uInt64 | dataCount | |||
) | [private] |
void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_computeDataArrays | ( | vector< vector< AccumType > > & | arys, | |
uInt64 & | currentCount, | |||
DataIterator | dataIter, | |||
MaskIterator | maskIter, | |||
WeightsIterator | weightsIter, | |||
uInt64 | dataCount, | |||
const vector< std::pair< AccumType, AccumType > > & | includeLimits, | |||
uInt64 | maxCount | |||
) | [private] |
void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_computeMinMax | ( | CountedPtr< AccumType > & | mymax, | |
CountedPtr< AccumType > & | mymin, | |||
DataIterator | dataIter, | |||
MaskIterator | maskIter, | |||
WeightsIterator | weightsIter, | |||
uInt64 | dataCount | |||
) | [private] |
void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_computeStats | ( | StatsData< AccumType > & | stats, | |
uInt64 & | ngood, | |||
LocationType & | location, | |||
DataIterator | dataIter, | |||
MaskIterator | maskIter, | |||
WeightsIterator | weightsIter, | |||
uInt64 | count | |||
) | [private] |
static void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_convertToAbsDevMedArray | ( | vector< AccumType > & | myArray, | |
AccumType | median | |||
) | [static, private] |
convert in place by taking the absolute value of the difference of the vector and the median
void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_createDataArray | ( | vector< AccumType > & | array | ) | [private] |
Create an unsorted array of the complete data set.
If includeLimits
is specified, only points within those limits (including min but excluding max, as per definition of bins), are included.
void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_createDataArrays | ( | vector< vector< AccumType > > & | arrays, | |
const vector< std::pair< AccumType, AccumType > > & | includeLimits, | |||
uInt64 | maxCount | |||
) | [private] |
vector<std::map<uInt64, AccumType> > casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_dataFromMultipleBins | ( | const vector< typename StatisticsUtilities< AccumType >::BinDesc > & | binDesc, | |
uInt64 | maxArraySize, | |||
const vector< std::set< uInt64 > > & | dataIndices, | |||
uInt64 | nBins | |||
) | [private] |
extract data from multiple histograms given by binDesc
.
dataIndices
represent the indices of the sorted arrays of values to extract. There should be exactly one set of data indices to extract for each supplied histogram. The data indices are relative to the minimum value of the minimum bin in their repsective histograms. The ordering of the maps in the returned vector represent the ordering of histograms in binDesc
. binDesc
should contain non-overlapping histograms and the histograms should be specified in ascending order.
vector<std::map<uInt64, AccumType> > casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_dataFromSingleBins | ( | const vector< uInt64 > & | binNpts, | |
uInt64 | maxArraySize, | |||
const vector< std::pair< AccumType, AccumType > > & | binLimits, | |||
const vector< std::set< uInt64 > > & | dataIndices, | |||
uInt64 | nBins | |||
) | [private] |
void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_doMinMax | ( | AccumType & | vmin, | |
AccumType & | vmax | |||
) | [protected] |
scan dataset(s) to find min and max
Int64 casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_doNpts | ( | ) | [private] |
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_findBins | ( | vector< vector< uInt64 > > & | binCounts, | |
vector< CountedPtr< AccumType > > & | sameVal, | |||
vector< Bool > & | allSame, | |||
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride, | |||
const vector< typename StatisticsUtilities< AccumType >::BinDesc > & | binDesc, | |||
const vector< AccumType > & | maxLimit | |||
) | const [protected, virtual] |
Reimplemented in casacore::ConstrainedRangeStatistics< CASA_STATP >.
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_findBins | ( | vector< vector< uInt64 > > & | binCounts, | |
vector< CountedPtr< AccumType > > & | sameVal, | |||
vector< Bool > & | allSame, | |||
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightsBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude, | |||
const vector< typename StatisticsUtilities< AccumType >::BinDesc > & | binDesc, | |||
const vector< AccumType > & | maxLimit | |||
) | const [protected, virtual] |
Reimplemented in casacore::ConstrainedRangeStatistics< CASA_STATP >.
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_findBins | ( | vector< vector< uInt64 > > & | binCounts, | |
vector< CountedPtr< AccumType > > & | sameVal, | |||
vector< Bool > & | allSame, | |||
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightsBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude, | |||
const vector< typename StatisticsUtilities< AccumType >::BinDesc > & | binDesc, | |||
const vector< AccumType > & | maxLimit | |||
) | const [protected, virtual] |
Reimplemented in casacore::ConstrainedRangeStatistics< CASA_STATP >.
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_findBins | ( | vector< vector< uInt64 > > & | binCounts, | |
vector< CountedPtr< AccumType > > & | sameVal, | |||
vector< Bool > & | allSame, | |||
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightsBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const vector< typename StatisticsUtilities< AccumType >::BinDesc > & | binDesc, | |||
const vector< AccumType > & | maxLimit | |||
) | const [protected, virtual] |
Reimplemented in casacore::ConstrainedRangeStatistics< CASA_STATP >.
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_findBins | ( | vector< vector< uInt64 > > & | binCounts, | |
vector< CountedPtr< AccumType > > & | sameVal, | |||
vector< Bool > & | allSame, | |||
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude, | |||
const vector< typename StatisticsUtilities< AccumType >::BinDesc > & | binDesc, | |||
const vector< AccumType > & | maxLimit | |||
) | const [protected, virtual] |
Reimplemented in casacore::ConstrainedRangeStatistics< CASA_STATP >.
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_findBins | ( | vector< vector< uInt64 > > & | binCounts, | |
vector< CountedPtr< AccumType > > & | sameVal, | |||
vector< Bool > & | allSame, | |||
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride, | |||
const vector< typename StatisticsUtilities< AccumType >::BinDesc > & | binDesc, | |||
const vector< AccumType > & | maxLimit | |||
) | const [protected, virtual] |
Reimplemented in casacore::ConstrainedRangeStatistics< CASA_STATP >.
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_findBins | ( | vector< vector< uInt64 > > & | binCounts, | |
vector< CountedPtr< AccumType > > & | sameVal, | |||
vector< Bool > & | allSame, | |||
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude, | |||
const vector< typename StatisticsUtilities< AccumType >::BinDesc > & | binDesc, | |||
const vector< AccumType > & | maxLimit | |||
) | const [protected, virtual] |
Reimplemented in casacore::ConstrainedRangeStatistics< CASA_STATP >.
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_findBins | ( | vector< vector< uInt64 > > & | binCounts, | |
vector< CountedPtr< AccumType > > & | sameVal, | |||
vector< Bool > & | allSame, | |||
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const vector< typename StatisticsUtilities< AccumType >::BinDesc > & | binDesc, | |||
const vector< AccumType > & | maxLimit | |||
) | const [protected, virtual] |
Get the counts of data within the specified histogram bins.
The number of arrays within binCounts will be equal to the number of histograms in binDesc
. Each array within binCounts
will have the same number of elements as the number of bins in its corresponding histogram in binDesc
.
Reimplemented in casacore::ConstrainedRangeStatistics< CASA_STATP >.
Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_getDoMaxMin | ( | ) | const [inline, protected] |
Definition at line 349 of file ClassicalStatistics.h.
Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_getIDataset | ( | ) | const [inline, protected] |
Definition at line 351 of file ClassicalStatistics.h.
virtual StatsData<AccumType> casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_getInitialStats | ( | ) | const [protected, virtual] |
AccumType casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_getStatistic | ( | StatisticsData::STATS | stat | ) | [protected, virtual] |
StatsData<AccumType> casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_getStatistics | ( | ) | [protected, virtual] |
virtual const StatsData<AccumType>& casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_getStatsData | ( | ) | const [inline, protected, virtual] |
Reimplemented in casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >.
Definition at line 363 of file ClassicalStatistics.h.
virtual StatsData<AccumType>& casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_getStatsData | ( | ) | [inline, protected, virtual] |
retreive stats structure.
Allows derived classes to maintain their own StatsData structs.
Reimplemented in casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >.
Definition at line 361 of file ClassicalStatistics.h.
Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_increment | ( | Bool | includeIDataset | ) | [private] |
increment the relevant loop counters
void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_incrementThreadIters | ( | DataIterator & | dataIter, | |
MaskIterator & | maskIter, | |||
WeightsIterator & | weightsIter, | |||
uInt64 & | offset, | |||
uInt | nthreads | |||
) | const [private] |
increment thread-based iterators
std::map<uInt64, AccumType> casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_indicesToValues | ( | CountedPtr< uInt64 > | knownNpts, | |
CountedPtr< AccumType > | knownMin, | |||
CountedPtr< AccumType > | knownMax, | |||
uInt64 | maxArraySize, | |||
const std::set< uInt64 > & | dataIndices, | |||
Bool | persistSortedArray, | |||
uInt64 | nBins | |||
) | [private] |
get the values for the specified indices in the sorted array of all good data
void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_initIterators | ( | ) | [private] |
void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_initLoopVars | ( | ) | [private] |
void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_initThreadVars | ( | uInt & | nBlocks, | |
uInt64 & | extra, | |||
uInt & | nthreads, | |||
PtrHolder< DataIterator > & | dataIter, | |||
PtrHolder< MaskIterator > & | maskIter, | |||
PtrHolder< WeightsIterator > & | weightsIter, | |||
PtrHolder< uInt64 > & | offset, | |||
uInt | nThreadsMax | |||
) | const [private] |
Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_isNptsSmallerThan | ( | vector< AccumType > & | arrayToSort, | |
uInt | maxArraySize | |||
) | [private] |
Determine by scanning the dataset if the number of good points is smaller than maxArraySize
.
If so, arrayToSort
will contain the unsorted data values. If not, this vector will be empty.
static void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_makeBins | ( | typename StatisticsUtilities< AccumType >::BinDesc & | bins, | |
AccumType | minData, | |||
AccumType | maxData, | |||
uInt | maxBins, | |||
Bool | allowPad | |||
) | [static, private] |
If allowPad
is True, then pad the lower side of the lowest bin and the higher side of the highest bin so that minData and maxData do not fall on the edge of their respective bins.
If false, no padding so that minData and maxData are also exactly the histogram abscissa limits.
std::set<uInt64> casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_medianIndices | ( | CountedPtr< uInt64 > | knownNpts | ) | [private] |
get the index (for odd npts) or indices (for even npts) of the median of the sorted array.
If knownNpts is not null, it will be used and must be correct. If it is null, the value of _npts will be used if it has been previously calculated. If not, the data sets will be scanned to determine npts.
static void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_mergeResults | ( | vector< vector< uInt64 > > & | bins, | |
vector< CountedPtr< AccumType > > & | sameVal, | |||
vector< Bool > & | allSame, | |||
const PtrHolder< vector< vector< uInt64 > > > & | tBins, | |||
const PtrHolder< vector< CountedPtr< AccumType > > > & | tSameVal, | |||
const PtrHolder< vector< Bool > > & | tAllSame, | |||
uInt | nThreadsMax | |||
) | [static, private] |
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_minMax | ( | CountedPtr< AccumType > & | mymin, | |
CountedPtr< AccumType > & | mymax, | |||
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride | |||
) | const [protected, virtual] |
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_minMax | ( | CountedPtr< AccumType > & | mymin, | |
CountedPtr< AccumType > & | mymax, | |||
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightsBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude | |||
) | const [protected, virtual] |
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_minMax | ( | CountedPtr< AccumType > & | mymin, | |
CountedPtr< AccumType > & | mymax, | |||
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightsBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude | |||
) | const [protected, virtual] |
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_minMax | ( | CountedPtr< AccumType > & | mymin, | |
CountedPtr< AccumType > & | mymax, | |||
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightsBegin, | |||
Int64 | nr, | |||
uInt | dataStride | |||
) | const [protected, virtual] |
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_minMax | ( | CountedPtr< AccumType > & | mymin, | |
CountedPtr< AccumType > & | mymax, | |||
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude | |||
) | const [protected, virtual] |
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_minMax | ( | CountedPtr< AccumType > & | mymin, | |
CountedPtr< AccumType > & | mymax, | |||
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride | |||
) | const [protected, virtual] |
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_minMax | ( | CountedPtr< AccumType > & | mymin, | |
CountedPtr< AccumType > & | mymax, | |||
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude | |||
) | const [protected, virtual] |
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_minMax | ( | CountedPtr< AccumType > & | mymin, | |
CountedPtr< AccumType > & | mymax, | |||
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride | |||
) | const [protected, virtual] |
uInt casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_nThreadsMax | ( | ) | const [private] |
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateArray | ( | vector< AccumType > & | ary, | |
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude | |||
) | const [protected, virtual] |
weights, mask, ranges
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateArray | ( | vector< AccumType > & | ary, | |
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride | |||
) | const [protected, virtual] |
weights and mask
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateArray | ( | vector< AccumType > & | ary, | |
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightsBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude | |||
) | const [protected, virtual] |
weights and ranges
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateArray | ( | vector< AccumType > & | ary, | |
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightsBegin, | |||
Int64 | nr, | |||
uInt | dataStride | |||
) | const [protected, virtual] |
weights
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateArray | ( | vector< AccumType > & | ary, | |
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude | |||
) | const [protected, virtual] |
mask and ranges
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateArray | ( | vector< AccumType > & | ary, | |
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride | |||
) | const [protected, virtual] |
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateArray | ( | vector< AccumType > & | ary, | |
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude | |||
) | const [protected, virtual] |
ranges
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateArray | ( | vector< AccumType > & | ary, | |
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride | |||
) | const [protected, virtual] |
populate an unsorted array with valid data.
no weights, no mask, no ranges
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateArrays | ( | vector< vector< AccumType > > & | arys, | |
uInt64 & | currentCount, | |||
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude, | |||
const vector< std::pair< AccumType, AccumType > > & | includeLimits, | |||
uInt64 | maxCount | |||
) | const [protected, virtual] |
weights, mask, ranges
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateArrays | ( | vector< vector< AccumType > > & | arys, | |
uInt64 & | currentCount, | |||
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride, | |||
const vector< std::pair< AccumType, AccumType > > & | includeLimits, | |||
uInt64 | maxCount | |||
) | const [protected, virtual] |
weights and mask
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateArrays | ( | vector< vector< AccumType > > & | arys, | |
uInt64 & | currentCount, | |||
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightsBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude, | |||
const vector< std::pair< AccumType, AccumType > > & | includeLimits, | |||
uInt64 | maxCount | |||
) | const [protected, virtual] |
weights and ranges
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateArrays | ( | vector< vector< AccumType > > & | arys, | |
uInt64 & | currentCount, | |||
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightsBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const vector< std::pair< AccumType, AccumType > > & | includeLimits, | |||
uInt64 | maxCount | |||
) | const [protected, virtual] |
weights
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateArrays | ( | vector< vector< AccumType > > & | arys, | |
uInt64 & | currentCount, | |||
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude, | |||
const vector< std::pair< AccumType, AccumType > > & | includeLimits, | |||
uInt64 | maxCount | |||
) | const [protected, virtual] |
mask and ranges
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateArrays | ( | vector< vector< AccumType > > & | arys, | |
uInt64 & | currentCount, | |||
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride, | |||
const vector< std::pair< AccumType, AccumType > > & | includeLimits, | |||
uInt64 | maxCount | |||
) | const [protected, virtual] |
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateArrays | ( | vector< vector< AccumType > > & | arys, | |
uInt64 & | currentCount, | |||
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude, | |||
const vector< std::pair< AccumType, AccumType > > & | includeLimits, | |||
uInt64 | maxCount | |||
) | const [protected, virtual] |
ranges
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateArrays | ( | vector< vector< AccumType > > & | arys, | |
uInt64 & | currentCount, | |||
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const vector< std::pair< AccumType, AccumType > > & | includeLimits, | |||
uInt64 | maxCount | |||
) | const [protected, virtual] |
Create a vector of unsorted arrays, one array for each bin defined by includeLimits
.
includeLimits
should be non-overlapping and should be given in ascending order (the algorithm used assumes this). Once the sum of the lengths of all arrays equals maxCount
the method will return with no further processing. no weights, no mask, no ranges
virtual Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateTestArray | ( | vector< AccumType > & | ary, | |
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude, | |||
uInt | maxElements | |||
) | const [protected, virtual] |
weights, mask, ranges
virtual Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateTestArray | ( | vector< AccumType > & | ary, | |
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride, | |||
uInt | maxElements | |||
) | const [protected, virtual] |
weights and mask
virtual Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateTestArray | ( | vector< AccumType > & | ary, | |
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightsBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude, | |||
uInt | maxElements | |||
) | const [protected, virtual] |
weights and ranges
virtual Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateTestArray | ( | vector< AccumType > & | ary, | |
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
uInt | maxElements | |||
) | const [protected, virtual] |
weights
virtual Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateTestArray | ( | vector< AccumType > & | ary, | |
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude, | |||
uInt | maxElements | |||
) | const [protected, virtual] |
mask and ranges
virtual Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateTestArray | ( | vector< AccumType > & | ary, | |
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride, | |||
uInt | maxElements | |||
) | const [protected, virtual] |
mask
virtual Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateTestArray | ( | vector< AccumType > & | ary, | |
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude, | |||
uInt | maxElements | |||
) | const [protected, virtual] |
ranges
virtual Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_populateTestArray | ( | vector< AccumType > & | ary, | |
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
uInt | maxElements | |||
) | const [protected, virtual] |
no weights, no mask, no ranges
uInt casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_threadIdx | ( | ) | const [private] |
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_unweightedStats | ( | StatsData< AccumType > & | stats, | |
uInt64 & | ngood, | |||
LocationType & | location, | |||
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude | |||
) | [protected, virtual] |
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_unweightedStats | ( | StatsData< AccumType > & | stats, | |
uInt64 & | ngood, | |||
LocationType & | location, | |||
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride | |||
) | [protected, virtual] |
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_unweightedStats | ( | StatsData< AccumType > & | stats, | |
uInt64 & | ngood, | |||
LocationType & | location, | |||
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude | |||
) | [protected, virtual] |
no weights, no mask
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_unweightedStats | ( | StatsData< AccumType > & | stats, | |
uInt64 & | ngood, | |||
LocationType & | location, | |||
const DataIterator & | dataBegin, | |||
Int64 | nr, | |||
uInt | dataStride | |||
) | [protected, virtual] |
no weights, no mask, no ranges
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_updateDataProviderMaxMin | ( | const StatsData< AccumType > & | threadStats | ) | [protected, virtual] |
Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_valuesFromSortedArray | ( | std::map< uInt64, AccumType > & | values, | |
CountedPtr< uInt64 > | knownNpts, | |||
const std::set< uInt64 > & | indices, | |||
uInt64 | maxArraySize, | |||
Bool | persistSortedArray | |||
) | [private] |
get values from sorted array if the array is small enough to be held in memory.
Note that this is the array containing all good data, not data in just a single bin representing a subset of good data. Returns True if the data were successfully retrieved. If True is returned, the values map will contain a map of index to value.
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_weightedStats | ( | StatsData< AccumType > & | stats, | |
LocationType & | location, | |||
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude | |||
) | [protected, virtual] |
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_weightedStats | ( | StatsData< AccumType > & | stats, | |
LocationType & | location, | |||
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const MaskIterator & | maskBegin, | |||
uInt | maskStride | |||
) | [protected, virtual] |
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_weightedStats | ( | StatsData< AccumType > & | stats, | |
LocationType & | location, | |||
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightsBegin, | |||
Int64 | nr, | |||
uInt | dataStride, | |||
const DataRanges & | ranges, | |||
Bool | isInclude | |||
) | [protected, virtual] |
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_weightedStats | ( | StatsData< AccumType > & | stats, | |
LocationType & | location, | |||
const DataIterator & | dataBegin, | |||
const WeightsIterator & | weightsBegin, | |||
Int64 | nr, | |||
uInt | dataStride | |||
) | [protected, virtual] |
has weights, but no mask, no ranges
virtual StatisticsData::ALGORITHM casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::algorithm | ( | ) | const [inline, virtual] |
get the algorithm that this object uses for computing stats
Implements casacore::StatisticsAlgorithm< AccumType, DataIterator, MaskIterator, WeightsIterator >.
Reimplemented in casacore::ChauvenetCriterionStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >, casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >, and casacore::HingesFencesStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >.
Definition at line 77 of file ClassicalStatistics.h.
virtual AccumType casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::getMedian | ( | CountedPtr< uInt64 > | knownNpts = NULL , |
|
CountedPtr< AccumType > | knownMin = NULL , |
|||
CountedPtr< AccumType > | knownMax = NULL , |
|||
uInt | binningThreshholdSizeBytes = 4096 *4096 , |
|||
Bool | persistSortedArray = False , |
|||
uInt64 | nBins = 10000 | |||
) | [virtual] |
In the following group of methods, if the size of the composite dataset is smaller than binningThreshholdSizeBytes
, the composite dataset will be (perhaps partially) sorted and persisted in memory during the call.
In that case, and if persistSortedArray
is True, this sorted array will remain in memory after the call and will be used on subsequent calls of this method when binningThreshholdSizeBytes
is greater than the size of the composite dataset. If persistSortedArray
is False, the sorted array will not be stored after this call completes and so any subsequent calls for which the dataset size is less than binningThreshholdSizeBytes
, the dataset will be sorted from scratch. Values which are not included due to non-unity strides, are not included in any specified ranges, are masked, or have associated weights of zero are not considered as dataset members for quantile computations. If one has a priori information regarding the number of points (npts) and/or the minimum and maximum values of the data set, these can be supplied to improve performance. Note however, that if these values are not correct, the resulting median and/or quantile values will also not be correct (although see the following notes regarding max/min). Note that if this object has already had getStatistics() called, and the min and max were calculated, there is no need to pass these values in as they have been stored internally and used (although passing them in shouldn't hurt anything). If provided, npts, the number of points falling in the specified ranges which are not masked and have weights > 0, should be exactly correct. min
can be less than the true minimum, and max
can be greater than the True maximum, but for best performance, these should be as close to the actual min and max as possible. In order for quantile computations to occur over multiple datasets, all datasets must be available. This means that if setCalculateAsAdded() was previously called by passing in a value of True, these methods will throw an exception as the previous call indicates that there is no guarantee that all datasets will be available. If one uses a data provider (by having called setDataProvider()), then this should not be an issue.
get the median of the distribution. For a dataset with an odd number of good points, the median is just the value at index int(N/2) in the equivalent sorted dataset, where N is the number of points. For a dataset with an even number of points, the median is the mean of the values at indices int(N/2)-1 and int(N/2) in the sorted dataset. nBins
is the number of bins, per histogram, to use to bin the data. More bins decrease the likelihood that multiple passes of the data set will be necessary, but also increase the amount of memory used. If nBins is set to less than 1,000, it is automatically increased to 1,000; there should be no reason to ever set nBins to be this small.
Implements casacore::StatisticsAlgorithm< AccumType, DataIterator, MaskIterator, WeightsIterator >.
Reimplemented in casacore::ConstrainedRangeStatistics< CASA_STATP >.
virtual AccumType casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::getMedianAbsDevMed | ( | CountedPtr< uInt64 > | knownNpts = NULL , |
|
CountedPtr< AccumType > | knownMin = NULL , |
|||
CountedPtr< AccumType > | knownMax = NULL , |
|||
uInt | binningThreshholdSizeBytes = 4096 *4096 , |
|||
Bool | persistSortedArray = False , |
|||
uInt64 | nBins = 10000 | |||
) | [virtual] |
get the median of the absolute deviation about the median of the data.
Implements casacore::StatisticsAlgorithm< AccumType, DataIterator, MaskIterator, WeightsIterator >.
Reimplemented in casacore::ConstrainedRangeStatistics< CASA_STATP >.
virtual AccumType casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::getMedianAndQuantiles | ( | std::map< Double, AccumType > & | quantiles, | |
const std::set< Double > & | fractions, | |||
CountedPtr< uInt64 > | knownNpts = NULL , |
|||
CountedPtr< AccumType > | knownMin = NULL , |
|||
CountedPtr< AccumType > | knownMax = NULL , |
|||
uInt | binningThreshholdSizeBytes = 4096 *4096 , |
|||
Bool | persistSortedArray = False , |
|||
uInt64 | nBins = 10000 | |||
) | [virtual] |
If one needs to compute both the median and quantile values, it is better to call getMedianAndQuantiles() rather than getMedian() and getQuantiles() separately, as the first will scan large data sets fewer times than calling the separate methods.
The return value is the median; the quantiles are returned in the quantiles
map. Values in the fractions
set represent the locations in the CDF and should be between 0 and 1, exclusive.
Implements casacore::StatisticsAlgorithm< AccumType, DataIterator, MaskIterator, WeightsIterator >.
Reimplemented in casacore::ConstrainedRangeStatistics< CASA_STATP >.
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::getMinMax | ( | AccumType & | mymin, | |
AccumType & | mymax | |||
) | [virtual] |
scan the dataset(s) that have been added, and find the min and max.
This method may be called even if setStatsToCaclulate has been called and MAX and MIN has been excluded. If setCalculateAsAdded(True) has previously been called after this object has been (re)initialized, an exception will be thrown.
Reimplemented in casacore::ConstrainedRangeStatistics< CASA_STATP >.
virtual uInt64 casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::getNPts | ( | ) | [virtual] |
scan the dataset(s) that have been added, and find the number of good points.
This method may be called even if setStatsToCaclulate has been called and NPTS has been excluded. If setCalculateAsAdded(True) has previously been called after this object has been (re)initialized, an exception will be thrown.
Reimplemented in casacore::ConstrainedRangeStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >, casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >, and casacore::ConstrainedRangeStatistics< CASA_STATP >.
virtual std::map<Double, AccumType> casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::getQuantiles | ( | const std::set< Double > & | fractions, | |
CountedPtr< uInt64 > | knownNpts = NULL , |
|||
CountedPtr< AccumType > | knownMin = NULL , |
|||
CountedPtr< AccumType > | knownMax = NULL , |
|||
uInt | binningThreshholdSizeBytes = 4096 *4096 , |
|||
Bool | persistSortedArray = False , |
|||
uInt64 | nBins = 10000 | |||
) | [virtual] |
Get the specified quantiles.
fractions
must be between 0 and 1, noninclusive.
Implements casacore::StatisticsAlgorithm< AccumType, DataIterator, MaskIterator, WeightsIterator >.
Reimplemented in casacore::ConstrainedRangeStatistics< CASA_STATP >.
virtual std::pair<Int64, Int64> casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::getStatisticIndex | ( | StatisticsData::STATS | stat | ) | [virtual] |
see base class description
Implements casacore::StatisticsAlgorithm< AccumType, DataIterator, MaskIterator, WeightsIterator >.
Reimplemented in casacore::ConstrainedRangeStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >, and casacore::ConstrainedRangeStatistics< CASA_STATP >.
Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::hasData | ( | ) | const [inline] |
Has any data been added to this object? Will return False if the object has been reset and no data have been added afterward.
Definition at line 182 of file ClassicalStatistics.h.
ClassicalStatistics<AccumType, DataIterator, MaskIterator, WeightsIterator>& casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::operator= | ( | const ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator > & | other | ) |
copy semantics
Reimplemented from casacore::StatisticsAlgorithm< AccumType, DataIterator, MaskIterator, WeightsIterator >.
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::reset | ( | ) | [virtual] |
reset object to initial state.
Clears all private fields including data, accumulators, etc.
Reimplemented in casacore::ChauvenetCriterionStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >, casacore::ConstrainedRangeStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >, casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >, casacore::HingesFencesStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >, and casacore::ConstrainedRangeStatistics< CASA_STATP >.
virtual void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::setCalculateAsAdded | ( | Bool | c | ) | [virtual] |
Should statistics be updated with calls to addData or should they only be calculated upon calls to getStatistics etc? Beware that calling this will automatically reinitialize the object, so that it will contain no references to data et al.
after this method has been called.
Reimplemented in casacore::ChauvenetCriterionStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >, casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >, and casacore::HingesFencesStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >.
void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::setDataProvider | ( | StatsDataProvider< AccumType, DataIterator, MaskIterator, WeightsIterator > * | dataProvider | ) |
An exception will be thrown if setCalculateAsAdded(True) has been called.
void casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::setStatsToCalculate | ( | std::set< StatisticsData::STATS > & | stats | ) | [virtual] |
Provide guidance to algorithms by specifying a priori which statistics the caller would like calculated.
Reimplemented from casacore::StatisticsAlgorithm< AccumType, DataIterator, MaskIterator, WeightsIterator >.
Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_calculateAsAdded [private] |
Definition at line 657 of file ClassicalStatistics.h.
vector<Int64>::const_iterator casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_citer [mutable, private] |
Definition at line 662 of file ClassicalStatistics.h.
uInt casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_dataCount [mutable, private] |
Definition at line 674 of file ClassicalStatistics.h.
vector<DataIterator>::const_iterator casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_dend [mutable, private] |
mutables, used to mitigate repeated code
Definition at line 661 of file ClassicalStatistics.h.
vector<DataIterator>::const_iterator casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_diter [mutable, private] |
Definition at line 661 of file ClassicalStatistics.h.
Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_doMaxMin [private] |
Definition at line 657 of file ClassicalStatistics.h.
Referenced by casacore::ClassicalStatistics< CASA_STATP >::_getDoMaxMin().
Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_doMedAbsDevMed [private] |
Reimplemented in casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >.
Definition at line 657 of file ClassicalStatistics.h.
vector<uInt>::const_iterator casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_dsiter [mutable, private] |
Definition at line 663 of file ClassicalStatistics.h.
Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_hasData [private] |
Definition at line 657 of file ClassicalStatistics.h.
Referenced by casacore::ClassicalStatistics< CASA_STATP >::hasData().
Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_hasMask [mutable, private] |
Definition at line 669 of file ClassicalStatistics.h.
Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_hasRanges [mutable, private] |
Definition at line 669 of file ClassicalStatistics.h.
Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_hasWeights [mutable, private] |
Definition at line 669 of file ClassicalStatistics.h.
Int64 casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_idataset [private] |
Definition at line 656 of file ClassicalStatistics.h.
Referenced by casacore::ClassicalStatistics< CASA_STATP >::_getIDataset().
std::map<uInt, Bool> casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_isIncludeRanges [mutable, private] |
Reimplemented from casacore::StatisticsAlgorithm< AccumType, DataIterator, MaskIterator, WeightsIterator >.
Definition at line 668 of file ClassicalStatistics.h.
std::map<uInt, MaskIterator> casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_masks [mutable, private] |
maps data to masks
Reimplemented from casacore::StatisticsAlgorithm< AccumType, DataIterator, MaskIterator, WeightsIterator >.
Definition at line 664 of file ClassicalStatistics.h.
uInt casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_maskStride [mutable, private] |
Definition at line 665 of file ClassicalStatistics.h.
Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_mustAccumulate [private] |
Definition at line 657 of file ClassicalStatistics.h.
uInt64 casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_myCount [mutable, private] |
Definition at line 675 of file ClassicalStatistics.h.
DataIterator casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_myData [mutable, private] |
Definition at line 672 of file ClassicalStatistics.h.
Bool casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_myIsInclude [mutable, private] |
Definition at line 669 of file ClassicalStatistics.h.
MaskIterator casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_myMask [mutable, private] |
Definition at line 671 of file ClassicalStatistics.h.
DataRanges casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_myRanges [mutable, private] |
Definition at line 670 of file ClassicalStatistics.h.
uInt casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_myStride [mutable, private] |
Definition at line 674 of file ClassicalStatistics.h.
WeightsIterator casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_myWeights [mutable, private] |
Definition at line 673 of file ClassicalStatistics.h.
std::map<uInt, DataRanges> casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_ranges [mutable, private] |
Definition at line 667 of file ClassicalStatistics.h.
StatsData<AccumType> casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_statsData [private] |
Reimplemented in casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >.
Definition at line 655 of file ClassicalStatistics.h.
Referenced by casacore::ClassicalStatistics< CASA_STATP >::_getStatsData().
std::map<uInt, WeightsIterator> casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_weights [mutable, private] |
maps data to weights
Reimplemented from casacore::StatisticsAlgorithm< AccumType, DataIterator, MaskIterator, WeightsIterator >.
Definition at line 666 of file ClassicalStatistics.h.
const uInt casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::BLOCK_SIZE [static, private] |
Definition at line 678 of file ClassicalStatistics.h.
const uInt casacore::ClassicalStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::CACHE_PADDING [static, private] |
Definition at line 677 of file ClassicalStatistics.h.