Compute and display various statistics from a lattice. More...

#include <LatticePADD.h>

Inheritance diagram for casacore::LatticeStatistics< T >:

Public Types
typedef NumericTraits< T > ::PrecisionType	AccumType

typedef std::pair< String, String >	stat_element

typedef std::list< stat_element >	stat_list

Public Types inherited from casacore::LatticeStatsBase
enum	StatisticsTypes { NPTS, SUM, SUMSQ, MEDIAN, MEDABSDEVMED, QUARTILE, Q1, Q3, MIN, MAX, MEAN, VARIANCE, SIGMA, RMS, FLUX, NSTATS, NACCUM }
	This `enum StatisticTypes` is provided for use with the `LatticeStatistics<T>::setPlotting` function. More...

Public Member Functions
	LatticeStatistics (const MaskedLattice< T > &lattice, LogIO &os, Bool showProgress=True, Bool forceDisk=False, Bool clone=True)
	Constructor takes the lattice and a `LogIO` object for logging. More...

	LatticeStatistics (const MaskedLattice< T > &lattice, Bool showProgress=True, Bool forceDisk=False, Bool clone=True)
	Constructor takes the lattice only. More...

	LatticeStatistics (const LatticeStatistics< T > &other)
	Copy constructor. More...

virtual	~LatticeStatistics ()
	Destructor. More...

LatticeStatistics< T > &	operator= (const LatticeStatistics< T > &other)
	Assignment operator. More...

Bool	setAxes (const Vector< Int > &cursorAxes)
	Set the cursor axes (0 relative). More...

Bool	setInExCludeRange (const Vector< T > &include, const Vector< T > &exclude, Bool setMinMaxToInclude=False)
	You may specify a pixel intensity range as either one for which all pixels in that range are included or one for which all pixels in that range are excluded. More...

Bool	setList (const Bool &doList)
	This function allows you to control whether the statistics are written to the output stream if you are also making a plot. More...

Bool	display ()
	Display the statistics by listing and/or plotting them. More...

Bool	getLayerStats (String &stats, Double area, Int zAxis=-1, Int zLayer=-1, Int hAxis=-1, Int hLayer=-1)

Bool	getLayerStats (stat_list &stats, Double area, Int zAxis=-1, Int zLayer=-1, Int hAxis=-1, Int hLayer=-1)

Vector< Int >	displayAxes () const
	Return the display axes. More...

Bool	getStatistic (Array< AccumType > &stat, LatticeStatsBase::StatisticsTypes type, Bool dropDeg=True)
	Recover the desired Statistic into an array. More...

Bool	getConvertedStatistic (Array< T > &stat, LatticeStatsBase::StatisticsTypes type, Bool dropDeg=True)

Bool	getMinMaxPos (IPosition &minPos, IPosition &maxPos)
	Recover position of min and max. More...

Bool	getStats (Vector< AccumType > &, const IPosition &pos, const Bool posInLattice=False)
	This function gets a vector containing all the statistics for a given location. More...

void	resetError ()
	Reset argument error condition. More...

Bool	getFullMinMax (T &dataMin, T &dataMax)
	Get full lattice min and max only. More...

String	errorMessage () const
	Recover last error message. More...

Bool	setNewLattice (const MaskedLattice< T > &lattice, Bool clone=True)
	Set a new MaskedLattice object. More...

Bool	hasLogger () const
	Did we construct with a logger ? More...

Bool	configureBiweight (Int maxIter, Double c)
	The configure methods return True if reconfiguration is actually necessary (ie if the underlying storage lattice needs to be recomputed). More...

Bool	configureClassical ()
	configure object to use Classical Statistics The time, t_x, it takes to compute classical statistics using algorithm x, can be modeled by t_x = n_sets(a_x + b_xn_el) where n_sets is the number of independent sets of data to compute stats on, each containing n_el number of elements. More...

Bool	configureClassical (Double aOld, Double bOld, Double aNew, Double bNew)

Bool	configureFitToHalf (FitToHalfStatisticsData::CENTER centerType=FitToHalfStatisticsData::CMEAN, FitToHalfStatisticsData::USE_DATA useData=FitToHalfStatisticsData::LE_CENTER, AccumType centerValue=0)
	configure to use fit to half algorithm. More...

Bool	configureHingesFences (Double f)
	configure to use hinges-fences algorithm More...

Bool	configureChauvenet (Double zscore=-1, Int maxIterations=-1)
	configure to use Chauvenet's criterion More...

void	forceUseStatsFrameworkUsingDataProviders ()
	The force* methods are really only for testing. More...

void	forceUseStatsFrameworkUsingArrays ()

void	forceUseOldTiledApplyMethod ()

void	forceAllowCodeDecideWhichAlgortihmToUse ()

std::map< String, uInt >	getChauvenetNiter () const
	get number of iterations associated with Chauvenet criterion algorithm More...

void	setComputeQuantiles (Bool b)
	should quantile-like stats (median, quartiles, medabsdevmed) be computed? When the stats framework is used, It is better to set this before computing any statistics, to avoid unnecessary duplicate creations of the stats algorithm objects. More...

Protected Member Functions
virtual Bool	_canDoFlux () const
	Virtual Functions. More...

virtual Quantum< AccumType >	_flux (Bool &, AccumType, Double) const

virtual void	listMinMax (ostringstream &osMin, ostringstream &osMax, Int oWidth, DataType type)

virtual Bool	listStats (Bool hasBeam, const IPosition &dPos, const Matrix< AccumType > &ord)
	List the statistics to the logger. More...

virtual Bool	listLayerStats (const Matrix< AccumType > &ord, ostringstream &rslt, Int zLayer)

IPosition	locInLattice (const IPosition &storagePosition, Bool relativeToParent=True) const
	Given a location in the storage lattice, convert those locations on the non-statistics axis (the last one) and optionally account for the lattice subsectioning. More...

void	setStream (ostream &os, Int oPrec)
	Non-virtual functions. More...

IPosition	_storageLatticeShape () const
	get the storage lattice shape More...

virtual Bool	_computeFlux (Array< AccumType > &flux, const Array< AccumType > &npts, const Array< AccumType > &sum)

virtual Bool	_computeFlux (Quantum< AccumType > &flux, AccumType sum, const IPosition &pos, Bool posInLattice)

void	_latticePosToStoragePos (IPosition &storagePos, const IPosition &latticePos)
	convert a position in the input lattice to the corresponding position in the stats storage lattice. More...

StatisticsData::ALGORITHM	_getAlgorithm () const

Protected Attributes
LogIO	os_p

Vector< Int >	cursorAxes_p

Vector< Int >	displayAxes_p

Bool	goodParameterStatus_p

Bool	haveLogger_p

Bool	fixedMinMax_p

Bool	doRobust_p
	doRobust means that when the storage lattice is generated, the robust statistics are generated as well More...

Bool	doList_p

IPosition	minPos_p

IPosition	maxPos_p

IPosition	blcParent_p

String	error_p

Private Types
enum	LatticeStatsAlgorithm { STATS_FRAMEWORK_ARRAYS, STATS_FRAMEWORK_DATA_PROVIDERS, TILED_APPLY }

Private Member Functions
void	_setDefaultCoeffs ()

virtual void	summStats ()
	Summarize the statistics found over the entire lattice. More...

virtual void	displayStats (AccumType nPts, AccumType sum, AccumType median, AccumType medAbsDevMed, AccumType quartile, AccumType sumSq, AccumType mean, AccumType var, AccumType rms, AccumType sigma, AccumType dMin, AccumType dMax, AccumType q1, AccumType q3)

Bool	calculateStatistic (Array< AccumType > &slice, LatticeStatsBase::StatisticsTypes type, Bool dropDeg)
	Calculate statistic from storage lattice and return in an array. More...

template<class U , class V >
void	_computeQuantiles (AccumType &median, AccumType &medAbsDevMed, AccumType &q1, AccumType &q3, CountedPtr< StatisticsAlgorithm< AccumType, U, V > > statsAlg, uInt64 knownNpts, AccumType knownMin, AccumType knownMax) const

template<class U , class V >
void	_computeQuantilesForStatsFramework (StatsData< AccumType > &stats, AccumType &q1, AccumType &q3, CountedPtr< StatisticsAlgorithm< AccumType, U, V > > statsAlg) const

void	generateRobust ()
	Find the median per cursorAxes chunk. More...

Bool	generateStorageLattice ()
	Create a new storage lattice. More...

IPosition	locInStorageLattice (const IPosition &latticePosition, LatticeStatsBase::StatisticsTypes type) const
	Given a location in the lattice and a statistic type, work out where to put it in the storage lattice. More...

void	minMax (Bool &none, AccumType &dMin, AccumType &dMax, const Vector< AccumType > &d, const Vector< AccumType > &n) const
	Find min and max of good data in arrays specified by pointers. More...

Bool	retrieveStorageStatistic (Array< AccumType > &slice, const LatticeStatsBase::StatisticsTypes type, const Bool dropDeg)
	Retrieve a statistic from the storage lattice and return in an array. More...

Bool	retrieveStorageStatistic (Vector< AccumType > &slice, const IPosition &pos, const Bool posInLattice)
	Retrieve a statistic from the storage lattice at the specified location and return in an array. More...

IPosition	statsSliceShape () const
	Find the shape of slice from the statistics lattice at one spatial pixel. More...

Bool	someGoodPoints ()
	See if there were some valid points found in the storage lattice. More...

void	stretchMinMax (AccumType &dMin, AccumType &dMax) const
	Stretch min and max by 5%. More...

void	_configureDataProviders (LatticeStatsDataProvider< T > &lattDP, MaskedLatticeStatsDataProvider< T > &maskedLattDP) const

void	_doStatsLoop (uInt nsets, CountedPtr< LattStatsProgress > progressMeter)

void	_computeStatsUsingArrays (SubLattice< T > subLat, CountedPtr< LattStatsProgress > progressMeter, const IPosition &cursorShape)

void	_computeStatsUsingLattDataProviders (LatticeStepper &stepper, SubLattice< T > subLat, Slicer &slicer, CountedPtr< LattStatsProgress > progressMeter, uInt nsets)

IPosition	_cursorShapeForArrayMethod (uInt64 setSize) const

void	_doComputationUsingArrays (std::vector< CountedPtr< StatisticsAlgorithm< AccumType, typename Array< T >::const_iterator, Array< Bool >::const_iterator > > > &sa, T &overallMin, T &overallMax, IPosition &arrayShape, std::vector< Array< T > > &dataArray, std::vector< Array< Bool > > &maskArray, std::vector< IPosition > &curPos, uInt nthreads, const SubLattice< T > &subLat, Bool isChauv, Bool isMasked, Bool isReal, CountedPtr< const DataRanges > range)

void	_fillStorageLattice (T currentMin, T currentMax, const IPosition &curPos, const StatsData< AccumType > &stats, Bool doQuantiles, AccumType q1=0, AccumType q3=0)

void	_updateMinMaxPos (T &overallMin, T &overallMax, T currentMin, T currentMax, const IPosition &minPos, const IPosition &maxPos, Bool atStart, const SubLattice< T > &subLat)

Static Private Member Functions
static AccumType	_mean (const AccumType &sum, const AccumType &npts)

static AccumType	_rms (const AccumType &sumsq, const AccumType &npts)

Private Attributes
const MaskedLattice< T > *	pInLattice_p

std::shared_ptr< const MaskedLattice< T > >	_inLatPtrMgr

CountedPtr< TempLattice < AccumType > >	pStoreLattice_p

Vector< Int >	nxy_p

Vector< Int >	statsToPlot_p

Vector< T >	range_p

Bool	noInclude_p

Bool	noExclude_p

Bool	needStorageLattice_p

Bool	doneSomeGoodPoints_p

Bool	someGoodPointsValue_p

Bool	showProgress_p

Bool	forceDisk_p

T	minFull_p

T	maxFull_p

Bool	doneFullMinMax_p

StatisticsAlgorithmFactory < AccumType, const T , const Bool >	_saf

std::map< String, uInt >	_chauvIters

Double	_aOld

Double	_bOld

Double	_aNew

Double	_bNew

PtrHolder< LatticeStatsAlgorithm >	_latticeStatsAlgortihm
	unset means let the code decide More...

Additional Inherited Members
Static Public Member Functions inherited from casacore::LatticeStatsBase
static Vector< Int >	toStatisticTypes (const String &statistics, const Regex &delimiter)
	Helper function to convert a String containing a list of desired statistics to the correct Vector<Int> required for the LatticeStatistics<T>::setPlotting function. More...

static Vector< Int >	toStatisticTypes (const Vector< String > &statistics)

static String	toStatisticName (StatisticsTypes type)
	Convert type to string. More...

static String	toStatisticName (Int type)

static Int	toStatisticType (const String &statistic)
	Returns -1 if the statistic string is not valid. More...

static Bool	setNxy (Vector< Int > &nxy, ostream &os)
	Check and fill in defaults for a `Vector<Int>` containing the number of subplots in x and y to be put on a plot. More...

static void	setStorageImageShape (IPosition &storeImageShape, const Bool &last, const Int &axisSize, const Vector< Int > &displayAxes, const IPosition &shape)
	A storage image is used to accumulate information as a function of the display axes as an image is iterated through. More...

static void	stretchMinMax (Float &min, Float &max)
	Stretch a range by 10%. More...

static std::set< Double >	quartileFracs ()

Detailed Description

template<class T>
class casacore::LatticeStatistics< T >

Compute and display various statistics from a lattice.

Intended use:

Public interface

Review Status

Date Reviewed:: yyyy/mm/dd

Prerequisite

Etymology

This is a class designed to display and retrieve statistics from lattices

Synopsis

This class enable you to display and/or retrieve statistics evaluated over specified regions of a lattice. The dimension of the region is arbitrary, but the size of each dimension is always the shape of the corresponding lattice axis. The statistics are displayed as a function of location of the axes not used to evaluate the statistics over. The axes which you evaluate the statistics over are called the cursor axes, the others are called the display axes.

For example, consider a lattice cube (call the axes xyz or [0,1,2]). You could display statistics from xy planes (cursor axes [0,1]) as a function of z (display axes [2]). Or you could retrieve statistics from the z axis (cursor axes [2]) for each [x,y] location (display axes [0,1]).

This class inherits from LatticeStatsBase This base class provides an enum defining allowed statistics types and a helper function to convert between a String and a Vector<Int> describing the desired statistics to plot. An example is shown below.

This class can list, plot and retrieve statistics. When it lists statistics, it always lists all the available statistics. When you plot statistics, you must specify which ones you would like to see.

This class generates a "storage lattice" into which it writes the accumulated statistical sums. It is from this storage lattice that the plotting and retrieval arrays are drawn. The storage lattice is either in core or on disk depending upon its size (if > 10% of memory given by.aipsrc system.resources.memory then it goes into a disk-based PagedArray). If on disk, the storage lattice is deleted when the LatticeStatistics class object destructs. However, currently, if the process is terminated ungracefully, the storage lattice will be left over.

Tip: This class has a few virtual functions; they are not part of a nice general polymorphic interface; rather they have specialized functionality; The idea of these is that you can derive a class from LatticeStatistics, such as ImageStatistics which provides you with a little more information when displaying/logging the statistics (such as world coordinates) The virtual functions are

getBeamArea can be used to return the synthesized beam area so that the FLUX statistic can be computed

listStats is used to list the statistics to the logger

getLabelsM find the X-axis label and the title label for the plotting;

Tip: If you ignore return error statuses from the functions that set the state of the class, the internal status of the class is set to bad; This means it will just keep on returning error conditions until you explicitly recover the situation; A message describing the last error condition can be recovered with function errorMessage;

Example

// Construct PagedImage (which isA MaskedLattice) from file name
 
 PagedImage<Float> inImage(inName);
 
// Construct statistics object
 
 LogOrigin or("myClass", "myFunction(...)", WHERE);
 LogIO os(or);
 LatticeStatistics<Float> stats(SubImage<FLoat>(inImage), os);
 
// Set cursor axes to see statistics of yz planes (0 relative)
 
 Vector<Int> cursorAxes(2)
 cursorAxes(0) = 1;
 cursorAxes(1) = 2;
 if (!stats.setAxes(cursorAxes)) return 1;
 
// Set to list and plot mean, sigma and rms
 
 if (!stats.setList(True)) return 1;
 String device = "/xs";
 Vector<Int> nxy(2);
 nxy(0) = 1;
 nxy(1) = 1;
 Vector<Int> statsToPlot = LatticeStatsBase::toStatisticTypes("mean,rms,sigma");
 if (!stats.setPlotting(statsToPlot, device, nxy)) return 1;
 
// Now activate actual listing and plotting
 
 if (!stats.display ()) return 1;
 
// Retrieve statistics into array
 
 Array<Double> sum;
 if (!stats.getStatistic(sum, LatticeStatsBase::SUM)) return 1;

In this example, a PagedImage is constructed (which isA MaskedLattice) with. We set the cursor axes to be the y and z axes, we specify to list the statistics if we plot them, and we ask to plot the mean, standard deviation, and root mean square of each yz plane as a function of x location on the PGPLOT device "/xs" with 1 subplot per page (there will be only one in this case). After the plotting and listing, we also retrieve the sum of the selected pixels as a function of x location into an array.

Motivation

The generation of statistical information from a lattice is a basic and necessary capability.

To Do

Implement plotting for complex lattices
Retrieve statistics at specified location of display axes

Definition at line 47 of file LatticePADD.h.

Member Typedef Documentation

template<class T>

typedef NumericTraits<T>::PrecisionType casacore::LatticeStatistics< T >::AccumType

Definition at line 202 of file LatticeStatistics.h.

template<class T>

typedef std::pair<String,String> casacore::LatticeStatistics< T >::stat_element

Definition at line 287 of file LatticeStatistics.h.

template<class T>

typedef std::list<stat_element> casacore::LatticeStatistics< T >::stat_list

Definition at line 288 of file LatticeStatistics.h.

Member Enumeration Documentation

template<class T>

enum casacore::LatticeStatistics::LatticeStatsAlgorithm

private

Enumerator
STATS_FRAMEWORK_ARRAYS
STATS_FRAMEWORK_DATA_PROVIDERS
TILED_APPLY

Definition at line 519 of file LatticeStatistics.h.

Constructor & Destructor Documentation

template<class T>

casacore::LatticeStatistics< T >::LatticeStatistics	(	const MaskedLattice< T > &	lattice,
		LogIO &	os,
		Bool	showProgress = `True`,
		Bool	forceDisk = `False`,
		Bool	clone = `True`
	)

Constructor takes the lattice and a LogIO object for logging.

You can specify whether you want to see progress meters or not. You can force the storage lattice to be disk based, otherwise the decision for core or disk is taken for you. If clone is True, the input lattice will be cloned, so the caller can make changes to the input lattice, but the statistics will reflect the lattice as it was at construction. If False, a reference to the input lattice is used, and so the caller shouldn't make changes to the input lattice between construction and calling statistics computation methods, unless it calls setNewLattice() to update the changed lattice. Obviously, cloning the lattice impacts performance and memory usage.

template<class T>

casacore::LatticeStatistics< T >::LatticeStatistics	(	const MaskedLattice< T > &	lattice,
		Bool	showProgress = `True`,
		Bool	forceDisk = `False`,
		Bool	clone = `True`
	)

Constructor takes the lattice only.

In the absence of a logger you get no messages. This includes error messages and potential listing of the statistics. You can specify whether you want to see progress meters or not. You can force the storage lattice to be disk based, otherwise the decision for core or disk is taken for you.

template<class T>

casacore::LatticeStatistics< T >::LatticeStatistics ( const LatticeStatistics< T > & other )

Copy constructor.

Copy semantics are followed. Therefore any storage lattice that has already been created for other is copied to *this

template<class T>

virtual casacore::LatticeStatistics< T >::~LatticeStatistics ( )

virtual

Destructor.

Member Function Documentation

template<class T>

virtual Bool casacore::LatticeStatistics< T >::_canDoFlux ( ) const

inlineprotectedvirtual

Virtual Functions.

See implementation to figure it all out !

FIXME The indirect dependence of this class on ImageInterface related issues (eg flux density) breaks encapsulation. All the ImageInterface related code should be encapsulated in ImageStatistics. Unfortunately, that requires significantly more time than I have atm. A return value of False means that the object in question cannot compute flux density values. The default implementation returns False.

Reimplemented in casacore::ImageStatistics< T >.

Definition at line 452 of file LatticeStatistics.h.

template<class T>

virtual Bool casacore::LatticeStatistics< T >::_computeFlux	(	Array< AccumType > &	flux,
		const Array< AccumType > &	npts,
		const Array< AccumType > &	sum
	)

protectedvirtual

Reimplemented in casacore::ImageStatistics< T >.

template<class T>

virtual Bool casacore::LatticeStatistics< T >::_computeFlux	(	Quantum< AccumType > &	flux,
		AccumType	sum,
		const IPosition &	pos,
		Bool	posInLattice
	)

protectedvirtual

Reimplemented in casacore::ImageStatistics< T >.

template<class T>

template<class U , class V >

void casacore::LatticeStatistics< T >::_computeQuantiles	(	AccumType &	median,
		AccumType &	medAbsDevMed,
		AccumType &	q1,
		AccumType &	q3,
		CountedPtr< StatisticsAlgorithm< AccumType, U, V > >	statsAlg,
		uInt64	knownNpts,
		AccumType	knownMin,
		AccumType	knownMax
	)		const

private

template<class T>

template<class U , class V >

void casacore::LatticeStatistics< T >::_computeQuantilesForStatsFramework	(	StatsData< AccumType > &	stats,
		AccumType &	q1,
		AccumType &	q3,
		CountedPtr< StatisticsAlgorithm< AccumType, U, V > >	statsAlg
	)		const

private

template<class T>

void casacore::LatticeStatistics< T >::_computeStatsUsingArrays	(	SubLattice< T >	subLat,
		CountedPtr< LattStatsProgress >	progressMeter,
		const IPosition &	cursorShape
	)

private

template<class T>

void casacore::LatticeStatistics< T >::_computeStatsUsingLattDataProviders	(	LatticeStepper &	stepper,
		SubLattice< T >	subLat,
		Slicer &	slicer,
		CountedPtr< LattStatsProgress >	progressMeter,
		uInt	nsets
	)

private

template<class T>

void casacore::LatticeStatistics< T >::_configureDataProviders	(	LatticeStatsDataProvider< T > &	lattDP,
		MaskedLatticeStatsDataProvider< T > &	maskedLattDP
	)		const

private

template<class T>

IPosition casacore::LatticeStatistics< T >::_cursorShapeForArrayMethod ( uInt64 setSize ) const

private

template<class T>

void casacore::LatticeStatistics< T >::_doComputationUsingArrays	(	std::vector< CountedPtr< StatisticsAlgorithm< AccumType, typename Array< T >::const_iterator, Array< Bool >::const_iterator > > > &	sa,
		T &	overallMin,
		T &	overallMax,
		IPosition &	arrayShape,
		std::vector< Array< T > > &	dataArray,
		std::vector< Array< Bool > > &	maskArray,
		std::vector< IPosition > &	curPos,
		uInt	nthreads,
		const SubLattice< T > &	subLat,
		Bool	isChauv,
		Bool	isMasked,
		Bool	isReal,
		CountedPtr< const DataRanges >	range
	)

private

template<class T>

void casacore::LatticeStatistics< T >::_doStatsLoop	(	uInt	nsets,
		CountedPtr< LattStatsProgress >	progressMeter
	)

private

template<class T>

void casacore::LatticeStatistics< T >::_fillStorageLattice	(	T	currentMin,
		T	currentMax,
		const IPosition &	curPos,
		const StatsData< AccumType > &	stats,
		Bool	doQuantiles,
		AccumType	q1 = `0`,
		AccumType	q3 = `0`
	)

private

template<class T>

virtual Quantum<AccumType> casacore::LatticeStatistics< T >::_flux	(	Bool &	,
		AccumType	,
		Double
	)		const

inlineprotectedvirtual

Reimplemented in casacore::ImageStatistics< T >.

Definition at line 454 of file LatticeStatistics.h.

template<class T>

StatisticsData::ALGORITHM casacore::LatticeStatistics< T >::_getAlgorithm ( ) const

protected

template<class T>

void casacore::LatticeStatistics< T >::_latticePosToStoragePos	(	IPosition &	storagePos,
		const IPosition &	latticePos
	)

protected

convert a position in the input lattice to the corresponding position in the stats storage lattice.

The number of elements in storagePos will not be changed and only the first N elements will be modified where N = the number of elements in latticePos. storagePos must therefore have at least as many elements as latticePos. Returns False if latticePos is inconsistent with the input lattice.

template<class T>

static AccumType casacore::LatticeStatistics< T >::_mean	(	const AccumType &	sum,
		const AccumType &	npts
	)

inlinestaticprivate

Definition at line 662 of file LatticeStatistics.h.

template<class T>

static AccumType casacore::LatticeStatistics< T >::_rms	(	const AccumType &	sumsq,
		const AccumType &	npts
	)

inlinestaticprivate

Definition at line 666 of file LatticeStatistics.h.

template<class T>

void casacore::LatticeStatistics< T >::_setDefaultCoeffs ( )

inlineprivate

coefficients from timings run on PagedImages on etacarinae.cv.nrao.edu (dmehring's development machine)

Definition at line 547 of file LatticeStatistics.h.

template<class T>

IPosition casacore::LatticeStatistics< T >::_storageLatticeShape ( ) const

inlineprotected

get the storage lattice shape

Definition at line 493 of file LatticeStatistics.h.

template<class T>

void casacore::LatticeStatistics< T >::_updateMinMaxPos	(	T &	overallMin,
		T &	overallMax,
		T	currentMin,
		T	currentMax,
		const IPosition &	minPos,
		const IPosition &	maxPos,
		Bool	atStart,
		const SubLattice< T > &	subLat
	)

private

template<class T>

Bool casacore::LatticeStatistics< T >::calculateStatistic	(	Array< AccumType > &	slice,
		LatticeStatsBase::StatisticsTypes	type,
		Bool	dropDeg
	)

private

Calculate statistic from storage lattice and return in an array.

template<class T>

Bool casacore::LatticeStatistics< T >::configureBiweight	(	Int	maxIter,
		Double	c
	)

The configure methods return True if reconfiguration is actually necessary (ie if the underlying storage lattice needs to be recomputed).

If no reconfiguration is necessary, False is returned.

configure to use biweight algorithm.

template<class T>

Bool casacore::LatticeStatistics< T >::configureChauvenet	(	Double	zscore = `-1`,
		Int	maxIterations = `-1`
	)

configure to use Chauvenet's criterion

template<class T>

Bool casacore::LatticeStatistics< T >::configureClassical ( )

configure object to use Classical Statistics The time, t_x, it takes to compute classical statistics using algorithm x, can be modeled by t_x = n_sets*(a_x + b_x*n_el) where n_sets is the number of independent sets of data to compute stats on, each containing n_el number of elements.

a_x is the time it takes to compute stats a a single set of data, and b_x is the time it takes to accumulate a single point. The old algorithm was developed in the early history of the project, I'm guessing by Neil Kileen, while the new algorithm was developed in 2015 by Dave Mehringer as part of the stats framework project. The old algorithm is faster in the regime of large n_sets and small n_el, while the new algorithm is faster in the regime of small n_sets and large n_el. If one always wants to use one of these algorithms, that algorithm's coefficients should be set to 0, while setting the other algorithm's coefficients to positive values. Note that it's the relative, not the absolute, values of these coeffecients that is important The version that takes no parameters uses the default values of the coefficients;

template<class T>

Bool casacore::LatticeStatistics< T >::configureClassical	(	Double	aOld,
		Double	bOld,
		Double	aNew,
		Double	bNew
	)

template<class T>

Bool casacore::LatticeStatistics< T >::configureFitToHalf	(	FitToHalfStatisticsData::CENTER	centerType = `FitToHalfStatisticsData::CMEAN`,
		FitToHalfStatisticsData::USE_DATA	useData = `FitToHalfStatisticsData::LE_CENTER`,
		AccumType	centerValue = `0`
	)

configure to use fit to half algorithm.

template<class T>

Bool casacore::LatticeStatistics< T >::configureHingesFences ( Double f )

configure to use hinges-fences algorithm

template<class T>

Bool casacore::LatticeStatistics< T >::display ( )

Display the statistics by listing and/or plotting them.

If you don't call this function then you won't see anything ! A return value of False indicates an invalid plotting device, or that the internal state of the class is bad.

template<class T>

Vector<Int> casacore::LatticeStatistics< T >::displayAxes ( ) const

inline

Return the display axes.

The returned vector will be valid only if setAxes has been called, or if one of the active "display" or "get*" methods has been called.

Definition at line 295 of file LatticeStatistics.h.

template<class T>

virtual void casacore::LatticeStatistics< T >::displayStats	(	AccumType	nPts,
		AccumType	sum,
		AccumType	median,
		AccumType	medAbsDevMed,
		AccumType	quartile,
		AccumType	sumSq,
		AccumType	mean,
		AccumType	var,
		AccumType	rms,
		AccumType	sigma,
		AccumType	dMin,
		AccumType	dMax,
		AccumType	q1,
		AccumType	q3
	)

privatevirtual

Reimplemented in casacore::ImageStatistics< T >.

template<class T>

String casacore::LatticeStatistics< T >::errorMessage ( ) const

inline

Recover last error message.

Definition at line 344 of file LatticeStatistics.h.

template<class T>

void casacore::LatticeStatistics< T >::forceAllowCodeDecideWhichAlgortihmToUse ( )

template<class T>

void casacore::LatticeStatistics< T >::forceUseOldTiledApplyMethod ( )

template<class T>

void casacore::LatticeStatistics< T >::forceUseStatsFrameworkUsingArrays ( )

template<class T>

void casacore::LatticeStatistics< T >::forceUseStatsFrameworkUsingDataProviders ( )

The force* methods are really only for testing.

They in general shouldn't be called in production code. The last one to be called will be the one to be attempted to be used.

template<class T>

void casacore::LatticeStatistics< T >::generateRobust ( )

private

Find the median per cursorAxes chunk.

template<class T>

Bool casacore::LatticeStatistics< T >::generateStorageLattice ( )

private

Create a new storage lattice.

template<class T>

std::map<String, uInt> casacore::LatticeStatistics< T >::getChauvenetNiter ( ) const

inline

get number of iterations associated with Chauvenet criterion algorithm

Definition at line 420 of file LatticeStatistics.h.

template<class T>

Bool casacore::LatticeStatistics< T >::getConvertedStatistic	(	Array< T > &	stat,
		LatticeStatsBase::StatisticsTypes	type,
		Bool	dropDeg = `True`
	)

template<class T>

Bool casacore::LatticeStatistics< T >::getFullMinMax	(	T &	dataMin,
		T &	dataMax
	)

Get full lattice min and max only.

Returns False if no unmasked data, else returns True. Honours any include or exclude range if set.

template<class T>

Bool casacore::LatticeStatistics< T >::getLayerStats	(	String &	stats,
		Double	area,
		Int	zAxis = `-1`,
		Int	zLayer = `-1`,
		Int	hAxis = `-1`,
		Int	hLayer = `-1`
	)

template<class T>

Bool casacore::LatticeStatistics< T >::getLayerStats	(	stat_list &	stats,
		Double	area,
		Int	zAxis = `-1`,
		Int	zLayer = `-1`,
		Int	hAxis = `-1`,
		Int	hLayer = `-1`
	)

template<class T>

Bool casacore::LatticeStatistics< T >::getMinMaxPos	(	IPosition &	minPos,
		IPosition &	maxPos
	)

Recover position of min and max.

Only works if there are no display axes (i.e. statistics found over entire image), otherwise, the returned values are resized to 0 shape. A return value of False indicates that the internal state of the class is bad.

template<class T>

Bool casacore::LatticeStatistics< T >::getStatistic	(	Array< AccumType > &	stat,
		LatticeStatsBase::StatisticsTypes	type,
		Bool	dropDeg = `True`
	)

Recover the desired Statistic into an array.

If you choose to use the T version, be aware that the values in the AccumType version of the Array may not be representable in the T version (e.g. large values for SumSq). The shape of the array is the shape of the display axes (e.g. if the shape of the lattice is [nx,ny,nz] and you ask for the mean of the y axis the shape of the returned array would be [nx,nz]. A returned array of zero shape indicates that there were no good values. A return value of False indicates that the internal state of the class is bad.

template<class T>

Bool casacore::LatticeStatistics< T >::getStats	(	Vector< AccumType > &	,
		const IPosition &	pos,
		const Bool	posInLattice = `False`
	)

This function gets a vector containing all the statistics for a given location.

If posInLattice=True then the location is a location in the input lattice. Any positions on the display axes are ignored. Otherwise, you should just give locations for the display axes only. Use can use the enum in class LatticeStatsBase to find out which locations in the vector contain which statistics. A returned vector of zero shape indicates that there were no good values. A return value of False indicates that the internal state of the class is bad.

template<class T>

Bool casacore::LatticeStatistics< T >::hasLogger ( ) const

inline

Did we construct with a logger ?

Definition at line 358 of file LatticeStatistics.h.

template<class T>

virtual Bool casacore::LatticeStatistics< T >::listLayerStats	(	const Matrix< AccumType > &	ord,
		ostringstream &	rslt,
		Int	zLayer
	)

protectedvirtual

template<class T>

virtual void casacore::LatticeStatistics< T >::listMinMax	(	ostringstream &	osMin,
		ostringstream &	osMax,
		Int	oWidth,
		DataType	type
	)

protectedvirtual

Reimplemented in casacore::ImageStatistics< T >.

template<class T>

virtual Bool casacore::LatticeStatistics< T >::listStats	(	Bool	hasBeam,
		const IPosition &	dPos,
		const Matrix< AccumType > &	ord
	)

protectedvirtual

List the statistics to the logger.

The implementation here is adequate for all lattices. See ImageStatistics for an example of where extra information is provided. hasBeam is the return value of getBeamArea. If it is true, that means that the FLUX statistics will be available in the storage lattice. dPos is the location of the start of the cursor in the storage image for this row. stats(j,i) is the statistics matrix. for the jth point and the ith statistic. The return value is False if something goes wrong ! Have a look at the implementation to see what you really have to do.

Reimplemented in casacore::ImageStatistics< T >.

template<class T>

IPosition casacore::LatticeStatistics< T >::locInLattice	(	const IPosition &	storagePosition,
		Bool	relativeToParent = `True`
	)		const

protected

Given a location in the storage lattice, convert those locations on the non-statistics axis (the last one) and optionally account for the lattice subsectioning.

template<class T>

IPosition casacore::LatticeStatistics< T >::locInStorageLattice	(	const IPosition &	latticePosition,
		LatticeStatsBase::StatisticsTypes	type
	)		const

private

Given a location in the lattice and a statistic type, work out where to put it in the storage lattice.

template<class T>

void casacore::LatticeStatistics< T >::minMax	(	Bool &	none,
		AccumType &	dMin,
		AccumType &	dMax,
		const Vector< AccumType > &	d,
		const Vector< AccumType > &	n
	)		const

private

Find min and max of good data in arrays specified by pointers.

template<class T>

LatticeStatistics<T>& casacore::LatticeStatistics< T >::operator= ( const LatticeStatistics< T > & other )

Assignment operator.

Deletes any storage lattice associated with the object being assigned to and copies any storage lattice that has already been created for "other".

template<class T>

void casacore::LatticeStatistics< T >::resetError ( )

inline

Reset argument error condition.

If you specify invalid arguments to one of the above set functions, an internal flag will be set which will prevent the work functions from doing anything (should you have chosen to ignore the Boolean return values of the set functions). This function allows you to reset that internal state to good.

Definition at line 337 of file LatticeStatistics.h.

template<class T>

Bool casacore::LatticeStatistics< T >::retrieveStorageStatistic	(	Array< AccumType > &	slice,
		const LatticeStatsBase::StatisticsTypes	type,
		const Bool	dropDeg
	)

private

Retrieve a statistic from the storage lattice and return in an array.

template<class T>

Bool casacore::LatticeStatistics< T >::retrieveStorageStatistic	(	Vector< AccumType > &	slice,
		const IPosition &	pos,
		const Bool	posInLattice
	)

private

Retrieve a statistic from the storage lattice at the specified location and return in an array.

template<class T>

Bool casacore::LatticeStatistics< T >::setAxes ( const Vector< Int > & cursorAxes )

Set the cursor axes (0 relative).

A return value of False indicates you have asked for an invalid axis. The default state of the class is to set the cursor axes to all axes in the lattice.

template<class T>

void casacore::LatticeStatistics< T >::setComputeQuantiles ( Bool b )

should quantile-like stats (median, quartiles, medabsdevmed) be computed? When the stats framework is used, It is better to set this before computing any statistics, to avoid unnecessary duplicate creations of the stats algorithm objects.

Unnecessary recreation of these is a performance bottleneck for iterative stats algorithms (eg Chauvenet), especially for large images (CAS-10947/10948).

template<class T>

Bool casacore::LatticeStatistics< T >::setInExCludeRange	(	const Vector< T > &	include,
		const Vector< T > &	exclude,
		Bool	setMinMaxToInclude = `False`
	)

You may specify a pixel intensity range as either one for which all pixels in that range are included or one for which all pixels in that range are excluded.

One or the other of include and exclude must therefore be a zero length vector if you call this function. If you are setting an include range, then if you set setMinMaxToInclude=True, the minimum and maximum values that this class returns will always be the minimum and maximum of the include range, respectively. A return value of False indicates that you have given both an include and an exclude range. A vector of length 1 for include and/or exclude means that the range will be set to (say for include) -abs(include(0)) to abs(include(0)). A return value of False indicates that both an inclusion and exclusion range were given or that the internal state of the class is bad. If you don't call this function, the default state of the class is to include all pixels.

template<class T>

Bool casacore::LatticeStatistics< T >::setList ( const Bool & doList )

This function allows you to control whether the statistics are written to the output stream if you are also making a plot.

A return value of False indicates that the internal state of the class is bad. If you have created the LatticeStatistics object without a LogIO object, you won't see any listings, but no error conditions will be generated. The default state of the class is to not list the output when making a plot.

template<class T>

Bool casacore::LatticeStatistics< T >::setNewLattice	(	const MaskedLattice< T > &	lattice,
		Bool	clone = `True`
	)

Set a new MaskedLattice object.

A return value of False indicates the lattice had an invalid type or that the internal state of the class is bad. If clone is True, the input lattice will be cloned, so the caller can make changes to the input lattice, but the statistics will reflect the lattice as it was at construction. If False, a reference to the input lattice is used, and so the caller shouldn't make changes to the input lattice between construction and calling statistics computation methods, unless it calls setNewLattice() to update the changed lattice. Obviously, cloning the lattice impacts performance and memory usage.

template<class T>