The biweight algorithm is a robust iterative algorithm that computes two quantities called the "location" and the "scale", which are analogous to the mean and the standard deviation.
More...
|
| | BiweightStatistics (Int maxNiter=3, Double c=6.0) |
| |
| | BiweightStatistics (const BiweightStatistics< CASA_STATP > &other) |
| | copy semantics More...
|
| |
| virtual | ~BiweightStatistics () |
| |
| BiweightStatistics< CASA_STATP > & | operator= (const BiweightStatistics< CASA_STATP > &other) |
| | copy semantics More...
|
| |
| virtual StatisticsData::ALGORITHM | algorithm () const |
| | get the algorithm that this object uses for computing stats More...
|
| |
virtual StatisticsAlgorithm
< CASA_STATP > * | clone () const |
| | Clone this instance. More...
|
| |
| virtual AccumType | getMedian (CountedPtr< uInt64 > knownNpts=nullptr, CountedPtr< AccumType > knownMin=nullptr, CountedPtr< AccumType > knownMax=nullptr, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt nBins=10000) |
| | these statistics are not supported. More...
|
| |
| virtual AccumType | getMedianAndQuantiles (std::map< Double, AccumType > &quantileToValue, const std::set< Double > &quantiles, CountedPtr< uInt64 > knownNpts=nullptr, CountedPtr< AccumType > knownMin=nullptr, CountedPtr< AccumType > knownMax=nullptr, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt nBins=10000) |
| |
| virtual AccumType | getMedianAbsDevMed (CountedPtr< uInt64 > knownNpts=nullptr, CountedPtr< AccumType > knownMin=nullptr, CountedPtr< AccumType > knownMax=nullptr, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt nBins=10000) |
| |
virtual std::map< Double,
AccumType > | getQuantiles (const std::set< Double > &quantiles, CountedPtr< uInt64 > npts=nullptr, CountedPtr< AccumType > min=nullptr, CountedPtr< AccumType > max=nullptr, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt nBins=10000) |
| |
| virtual std::pair< Int64, Int64 > | getStatisticIndex (StatisticsData::STATS stat) |
| | see base class description More...
|
| |
| Int | getNiter () const |
| | returns the number of iterations performed to compute the current location and scale values More...
|
| |
| virtual void | reset () |
| | reset object to initial state. More...
|
| |
| virtual void | setCalculateAsAdded (Bool c) |
| | If c is True, an exception is thrown; this algorithm does not support computing stats as data are added. More...
|
| |
| virtual void | setStatsToCalculate (std::set< StatisticsData::STATS > &stats) |
| | Provide guidance to algorithms by specifying a priori which statistics the caller would like calculated. More...
|
| |
| | ClassicalStatistics () |
| |
| | ClassicalStatistics (const ClassicalStatistics &cs) |
| | copy semantics More...
|
| |
| virtual | ~ClassicalStatistics () |
| |
| ClassicalStatistics & | operator= (const ClassicalStatistics &other) |
| | copy semantics More...
|
| |
| virtual CASA_STATP | getMedian (CountedPtr< uInt64 > knownNpts=nullptr, CountedPtr< CASA_STATP > knownMin=nullptr, CountedPtr< CASA_STATP > knownMax=nullptr, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt nBins=10000) |
| | In the following group of methods, if the size of the composite dataset is smaller than binningThreshholdSizeBytes, the composite dataset will be (perhaps partially) sorted and persisted in memory during the call. More...
|
| |
| virtual CASA_STATP | getMedianAndQuantiles (std::map< Double, CASA_STATP > &quantiles, const std::set< Double > &fractions, CountedPtr< uInt64 > knownNpts=nullptr, CountedPtr< CASA_STATP > knownMin=nullptr, CountedPtr< CASA_STATP > knownMax=nullptr, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt nBins=10000) |
| | If one needs to compute both the median and quantile values, it is better to call getMedianAndQuantiles() rather than getMedian() and getQuantiles() separately, as the first will scan large data sets fewer times than calling the separate methods. More...
|
| |
| virtual CASA_STATP | getMedianAbsDevMed (CountedPtr< uInt64 > knownNpts=nullptr, CountedPtr< CASA_STATP > knownMin=nullptr, CountedPtr< CASA_STATP > knownMax=nullptr, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt nBins=10000) |
| | get the median of the absolute deviation about the median of the data. More...
|
| |
virtual std::map< Double,
CASA_STATP > | getQuantiles (const std::set< Double > &fractions, CountedPtr< uInt64 > knownNpts=nullptr, CountedPtr< CASA_STATP > knownMin=nullptr, CountedPtr< CASA_STATP > knownMax=nullptr, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt nBins=10000) |
| | Get the specified quantiles. More...
|
| |
| virtual void | getMinMax (CASA_STATP &mymin, CASA_STATP &mymax) |
| | scan the dataset(s) that have been added, and find the min and max. More...
|
| |
| virtual void | getMinMaxNpts (uInt64 &npts, CASA_STATP &mymin, CASA_STATP &mymax) |
| |
| virtual uInt64 | getNPts () |
| | scan the dataset(s) that have been added, and find the number of good points. More...
|
| |
| virtual void | setDataProvider (StatsDataProvider< CASA_STATP > *dataProvider) |
| | An exception will be thrown if setCalculateAsAdded(True) has been called. More...
|
| |
| void | setQuantileComputer (CountedPtr< ClassicalQuantileComputer< CASA_STATP >> qc) |
| | Allow derived objects to set the quantile computer object. More...
|
| |
| virtual | ~StatisticsAlgorithm () |
| |
| void | addData (const DataIterator &first, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False) |
| | Add a dataset to an existing set of datasets on which statistics are to be calculated. More...
|
| |
| void | addData (const DataIterator &first, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False) |
| |
| void | addData (const DataIterator &first, const const Bool *&maskFirst, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1) |
| |
| void | addData (const DataIterator &first, const const Bool *&maskFirst, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1) |
| |
| void | addData (const DataIterator &first, const DataIterator &weightFirst, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False) |
| |
| void | addData (const DataIterator &first, const DataIterator &weightFirst, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False) |
| |
| void | addData (const DataIterator &first, const DataIterator &weightFirst, const const Bool *&maskFirst, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1) |
| |
| void | addData (const DataIterator &first, const DataIterator &weightFirst, const const Bool *&maskFirst, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1) |
| |
| CASA_STATP | getQuantile (Double quantile, CountedPtr< uInt64 > knownNpts=nullptr, CountedPtr< CASA_STATP > knownMin=nullptr, CountedPtr< CASA_STATP > knownMax=nullptr, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt nBins=10000) |
| | Purposefully not virtual. More...
|
| |
| CASA_STATP | getStatistic (StatisticsData::STATS stat) |
| | get the value of the specified statistic. More...
|
| |
| StatsData< CASA_STATP > | getStatistics () |
| | Return statistics. More...
|
| |
| void | setData (const DataIterator &first, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False) |
| | setdata() clears any current datasets or data provider and then adds the specified data set as the first dataset in the (possibly new) set of data sets for which statistics are to be calculated. More...
|
| |
| void | setData (const DataIterator &first, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False) |
| |
| void | setData (const DataIterator &first, const const Bool *&maskFirst, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1) |
| |
| void | setData (const DataIterator &first, const const Bool *&maskFirst, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1) |
| |
| void | setData (const DataIterator &first, const DataIterator &weightFirst, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False) |
| |
| void | setData (const DataIterator &first, const DataIterator &weightFirst, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False) |
| |
| void | setData (const DataIterator &first, const DataIterator &weightFirst, const const Bool *&maskFirst, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1) |
| |
| void | setData (const DataIterator &first, const DataIterator &weightFirst, const const Bool *&maskFirst, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1) |
| |
|
| void | _computeStats () |
| |
| virtual StatsData< AccumType > | _getStatistics () |
| |
| | ClassicalStatistics (CountedPtr< ClassicalQuantileComputer< CASA_STATP > > qc) |
| | This constructor should be used by derived objects in order to set the proper quantile computer object. More...
|
| |
| virtual void | _accumNpts (uInt64 &npts, const DataIterator &dataBegin, uInt64 nr, uInt dataStride) const |
| | scan through the data set to determine the number of good (unmasked, weight > 0, within range) points. More...
|
| |
| virtual void | _accumNpts (uInt64 &npts, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const |
| |
| virtual void | _accumNpts (uInt64 &npts, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const const Bool *&maskBegin, uInt maskStride) const |
| |
| virtual void | _accumNpts (uInt64 &npts, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const const Bool *&maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const |
| |
| virtual void | _accumNpts (uInt64 &npts, const DataIterator &dataBegin, const DataIterator &weightsBegin, uInt64 nr, uInt dataStride) const |
| |
| virtual void | _accumNpts (uInt64 &npts, const DataIterator &dataBegin, const DataIterator &weightsBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const |
| |
| virtual void | _accumNpts (uInt64 &npts, const DataIterator &dataBegin, const DataIterator &weightsBegin, uInt64 nr, uInt dataStride, const const Bool *&maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const |
| |
| virtual void | _accumNpts (uInt64 &npts, const DataIterator &dataBegin, const DataIterator &weightBegin, uInt64 nr, uInt dataStride, const const Bool *&maskBegin, uInt maskStride) const |
| |
| void | _accumulate (StatsData< CASA_STATP > &stats, const CASA_STATP &datum, const LocationType &location) |
| |
| void | _accumulate (StatsData< CASA_STATP > &stats, const CASA_STATP &datum, const CASA_STATP &weight, const LocationType &location) |
| |
| void | _addData () |
| | Allows derived classes to do things after data is set or added. More...
|
| |
| void | _clearStats () |
| |
| Bool | _getDoMaxMin () const |
| |
| virtual StatsData< CASA_STATP > | _getInitialStats () const |
| |
| virtual CASA_STATP | _getStatistic (StatisticsData::STATS stat) |
| |
| virtual StatsData< CASA_STATP > & | _getStatsData () |
| | Retrieve stats structure. More...
|
| |
virtual const StatsData
< CASA_STATP > & | _getStatsData () const |
| |
| virtual void | _minMax (CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, uInt64 nr, uInt dataStride) const |
| |
| virtual void | _minMax (CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const |
| |
| virtual void | _minMax (CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const const Bool *&maskBegin, uInt maskStride) const |
| |
| virtual void | _minMax (CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const const Bool *&maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const |
| |
| virtual void | _minMax (CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, const DataIterator &weightsBegin, uInt64 nr, uInt dataStride) const |
| |
| virtual void | _minMax (CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, const DataIterator &weightsBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const |
| |
| virtual void | _minMax (CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, const DataIterator &weightsBegin, uInt64 nr, uInt dataStride, const const Bool *&maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const |
| |
| virtual void | _minMax (CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, const DataIterator &weightBegin, uInt64 nr, uInt dataStride, const const Bool *&maskBegin, uInt maskStride) const |
| |
| virtual void | _minMaxNpts (uInt64 &npts, CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, uInt64 nr, uInt dataStride) const |
| | Sometimes we want the min, max, and npts all in one scan. More...
|
| |
| virtual void | _minMaxNpts (uInt64 &npts, CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const |
| |
| virtual void | _minMaxNpts (uInt64 &npts, CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const const Bool *&maskBegin, uInt maskStride) const |
| |
| virtual void | _minMaxNpts (uInt64 &npts, CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const const Bool *&maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const |
| |
| virtual void | _minMaxNpts (uInt64 &npts, CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, const DataIterator &weightsBegin, uInt64 nr, uInt dataStride) const |
| |
| virtual void | _minMaxNpts (uInt64 &npts, CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, const DataIterator &weightsBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const |
| |
| virtual void | _minMaxNpts (uInt64 &npts, CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, const DataIterator &weightsBegin, uInt64 nr, uInt dataStride, const const Bool *&maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const |
| |
| virtual void | _minMaxNpts (uInt64 &npts, CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, const DataIterator &weightBegin, uInt64 nr, uInt dataStride, const const Bool *&maskBegin, uInt maskStride) const |
| |
CountedPtr
< StatisticsAlgorithmQuantileComputer
< CASA_STATP > > | _getQuantileComputer () |
| |
| virtual void | _unweightedStats (StatsData< CASA_STATP > &stats, uInt64 &ngood, LocationType &location, const DataIterator &dataBegin, uInt64 nr, uInt dataStride) |
| | no weights, no mask, no ranges More...
|
| |
| virtual void | _unweightedStats (StatsData< CASA_STATP > &stats, uInt64 &ngood, LocationType &location, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) |
| | no weights, no mask More...
|
| |
| virtual void | _unweightedStats (StatsData< CASA_STATP > &stats, uInt64 &ngood, LocationType &location, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const const Bool *&maskBegin, uInt maskStride) |
| |
| virtual void | _unweightedStats (StatsData< CASA_STATP > &stats, uInt64 &ngood, LocationType &location, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const const Bool *&maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) |
| |
| virtual void | _updateDataProviderMaxMin (const StatsData< CASA_STATP > &threadStats) |
| |
| virtual void | _weightedStats (StatsData< CASA_STATP > &stats, LocationType &location, const DataIterator &dataBegin, const DataIterator &weightsBegin, uInt64 nr, uInt dataStride) |
| | has weights, but no mask, no ranges More...
|
| |
| virtual void | _weightedStats (StatsData< CASA_STATP > &stats, LocationType &location, const DataIterator &dataBegin, const DataIterator &weightsBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) |
| |
| virtual void | _weightedStats (StatsData< CASA_STATP > &stats, LocationType &location, const DataIterator &dataBegin, const DataIterator &weightBegin, uInt64 nr, uInt dataStride, const const Bool *&maskBegin, uInt maskStride) |
| |
| virtual void | _weightedStats (StatsData< CASA_STATP > &stats, LocationType &location, const DataIterator &dataBegin, const DataIterator &weightBegin, uInt64 nr, uInt dataStride, const const Bool *&maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) |
| |
| | StatisticsAlgorithm () |
| |
| | StatisticsAlgorithm (const StatisticsAlgorithm &other) |
| | use copy semantics, except for the data provider which uses reference semantics More...
|
| |
| StatisticsAlgorithm & | operator= (const StatisticsAlgorithm &other) |
| | use copy semantics, except for the data provider which uses reference semantics More...
|
| |
const StatisticsDataset
< CASA_STATP > & | _getDataset () const |
| | These methods are purposefully not virtual. More...
|
| |
| StatisticsDataset< CASA_STATP > & | _getDataset () |
| |
const std::set
< StatisticsData::STATS > | _getStatsToCalculate () const |
| |
virtual const std::set
< StatisticsData::STATS > & | _getUnsupportedStatistics () const |
| |
| void | _setUnsupportedStatistics (const std::set< StatisticsData::STATS > &stats) |
| | Derived classes should normally call this in their constructors, if applicable. More...
|
| |
|
| void | _computeLocationAndScaleSums (AccumType &sxw2, AccumType &sw2, AccumType &sx_M2w4, AccumType &ww_4u2, DataIterator dataIter, MaskIterator maskIter, WeightsIterator weightsIter, uInt64 dataCount, const typename StatisticsDataset< CASA_STATP >::ChunkData &chunk) |
| |
| void | _computeLocationSums (AccumType &sxw2, AccumType &sw2, DataIterator dataIter, MaskIterator maskIter, WeightsIterator weightsIter, uInt64 dataCount, const typename StatisticsDataset< CASA_STATP >::ChunkData &chunk) |
| |
| void | _computeScaleSums (AccumType &sx_M2w4, AccumType &ww_4u2, DataIterator dataIter, MaskIterator maskIter, WeightsIterator weightsIter, uInt64 dataCount, const typename StatisticsDataset< CASA_STATP >::ChunkData &chunk) const |
| |
| void | _doLocationAndScale () |
| |
| void | _doLocation () |
| |
| void | _doScale () |
| |
| void | _locationAndScaleSums (AccumType &sxw2, AccumType &sw2, AccumType &sx_M2w4, AccumType &ww_4u2, const DataIterator &dataBegin, uInt64 nr, uInt dataStride) const |
| | sxw2 = sum(x_i*(1 - u_i^2)^2) sw2 = sum((1-u_i^2)^2) sx_M2w4 = sum((x_i - _location)^2 * (1 - u_i^2)^4) = sum((x_i - _location)^2 * w_i^4) ww_4u2 = sum((1 - u_i^2) * (1 - 5*u_i^2)) = sum(w_i * (w_i - 4*u_i^2)) More...
|
| |
| void | _locationAndScaleSums (AccumType &sxw2, AccumType &sw2, AccumType &sx_M2w4, AccumType &ww_4u2, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const |
| |
| void | _locationAndScaleSums (AccumType &sxw2, AccumType &sw2, AccumType &sx_M2w4, AccumType &ww_4u2, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride) const |
| |
| void | _locationAndScaleSums (AccumType &sxw2, AccumType &sw2, AccumType &sx_M2w4, AccumType &ww_4u2, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const |
| |
| void | _locationAndScaleSums (AccumType &sxw2, AccumType &sw2, AccumType &sx_M2w4, AccumType &ww_4u2, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, uInt64 nr, uInt dataStride) const |
| |
| void | _locationAndScaleSums (AccumType &sxw2, AccumType &sw2, AccumType &sx_M2w4, AccumType &ww_4u2, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const |
| |
| void | _locationAndScaleSums (AccumType &sxw2, AccumType &sw2, AccumType &sx_M2w4, AccumType &ww_4u2, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const |
| |
| void | _locationAndScaleSums (AccumType &sxw2, AccumType &sw2, AccumType &sx_M2w4, AccumType &ww_4u2, const DataIterator &dataBegin, const WeightsIterator &weightBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride) const |
| |
| void | _locationSums (AccumType &sxw2, AccumType &sw2, const DataIterator &dataBegin, uInt64 nr, uInt dataStride) const |
| | sxw2 = sum(x_i*(1 - u_i^2)^2) sw2 = sum((1-u_i^2)^2) More...
|
| |
| void | _locationSums (AccumType &sxw2, AccumType &sw2, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const |
| |
| void | _locationSums (AccumType &sxw2, AccumType &sw2, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride) const |
| |
| void | _locationSums (AccumType &sxw2, AccumType &sw2, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const |
| |
| void | _locationSums (AccumType &sxw2, AccumType &sw2, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, uInt64 nr, uInt dataStride) const |
| |
| void | _locationSums (AccumType &sxw2, AccumType &sw2, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const |
| |
| void | _locationSums (AccumType &sxw2, AccumType &sw2, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const |
| |
| void | _locationSums (AccumType &sxw2, AccumType &sw2, const DataIterator &dataBegin, const WeightsIterator &weightBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride) const |
| |
| void | _scaleSums (AccumType &sx_M2w4, AccumType &ww_4u2, const DataIterator &dataBegin, uInt64 nr, uInt dataStride) const |
| | sx_M2w4 = sum((x_i - _location)^2 * (1 - u_i^2)^4) = sum((x_i - _location)^2 * w_i^4) ww_4u2 = sum((1 - u_i^2) * (1 - 5*u_i^2)) = sum(w_i * (w_i - 4*u_i^2)) More...
|
| |
| void | _scaleSums (AccumType &sx_M2w4, AccumType &ww_4u2, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const |
| |
| void | _scaleSums (AccumType &sx_M2w4, AccumType &ww_4u2, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride) const |
| |
| void | _scaleSums (AccumType &sx_M2w4, AccumType &ww_4u2, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const |
| |
| void | _scaleSums (AccumType &sx_M2w4, AccumType &ww_4u2, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, uInt64 nr, uInt dataStride) const |
| |
| void | _scaleSums (AccumType &sx_M2w4, AccumType &ww_4u2, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const |
| |
| void | _scaleSums (AccumType &sx_M2w4, AccumType &ww_4u2, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const |
| |
| void | _scaleSums (AccumType &sx_M2w4, AccumType &ww_4u2, const DataIterator &dataBegin, const WeightsIterator &weightBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride) const |
| |
template<class AccumType, class DataIterator, class MaskIterator = const Bool*, class WeightsIterator = DataIterator>
class casacore::BiweightStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >
The biweight algorithm is a robust iterative algorithm that computes two quantities called the "location" and the "scale", which are analogous to the mean and the standard deviation.
Important equations are
A. How to compute u_i values, which are related to the weights, w_i = (1 - u_i*u_i) if abs(u_i) < 1, 0 otherwise, using the equation
u_i = (x_i - c_bi)/(c*s_bi) (1)
where x_i are the data values, c_bi is the biweight location, c is a configurable constant, and s_bi is the biweight scale. For the initial computation of the u_i values, c_bi is set equal to the median of the distribution and s_bi is set equal to the normalized median of the absolute deviation about the median (that is the median of the absolute deviation about the median multiplied by the value of the probit function at 0.75). B The location, c_bi, is computed from
c_bi = sum(x_i * w_i^2)/sum(w_i^2) (2)
where only values of u_i which satisfy abs(u_i) < 1 (w_i > 0) are used in the sums. C. The scale value is computed using
n * sum((x_i - c_bi)^2 * w_i^4) s_bi^2 = _______________________________ (3) p * max(1, p - 1)
where n is the number of points for the entire distribution (which includes all the data for which abs(u_i) >= 1) and p is given by
p = abs(sum((w_i) * (w_i - 4*u_i^2)))
Again, the sums include only data for which abs(u_i) < 1.
The algorithm proceeds as follows.
- Compute initial u_i values from equation (1), setting c_bi equal to the median of the distribution and s_bi equal to the normalized median of the absolute deviation about the median.
- Compute the initial value of the scale using the u_i values computed in step 1. using equation 3.
- Recompute u_i values using the most recent previous scale and location values.
- Compute the location using the u_i values from step 3 and equation (2).
- Recompute u_i values using the most recent previous scale and location values.
- Compute the new scale value using the the u_i values computed in step 5 and the value of the location computed in step 4.
- Steps 3. - 6. are repeated until convergence occurs or the maximum number of iterations (a configurable parameter) is reached. The convergence criterion is given by
abs(1 - s_bi/s_bi,prev) < 0.03 * sqrt(0.5/(n - 1))
where s_bi,prev is the value of the scale computed in the previous iteration.
SPECIAL CASE TO FACILITATE SPEED
In the special case where maxNiter is specified to be negative, the algorithm proceeds as follows
- Compute u_i values using the median for the location and the normalized median of the absolute deviation about the median as the scale
- Compute the location and scale (which can be carried out simultaneously) using the u_i values computed in step 1. The value of the location is just the median that is used in equation (3) to compute the scale
IMPORTANT NOTE REGARDING USER SPECIFIED WEIGHTS
Although user-specified weights can be supplied, they are effectively ignored by this algorithm, except for data which have weights of zero, which are ignored.
This is a derived class of ClassicalStatistics, rather than ConstrainedRangeStatistics, because if behaves differently from ConstrainedRangeStatistics and does not need to use any methods in that class, so making it a specialization of the higher level ClassicalStatistics seems the better choice.
Definition at line 121 of file BiweightStatistics.h.
Public Member Functions inherited from casacore::ClassicalStatistics< CASA_STATP >
1.8.5