 News |
FAQ
|
| Search
|
Home
|
| Getting Started | Documentation | Glish | Learn More | Programming | Contact Us |
|
| Version 1.9 Build 1367 |
|
We now summarize our immediate, short range, and long range plans for Single Dish Calibration and Imaging.
All of the classes we have implemented for this initial single dish development and calibration effort need improvements. Many of these have been suggested in previous sections of this document.
Building upon our experience in implementing SDMeasurementSet, Peter Teuben is outlining a more general MeasurementSet. This MeasurementSet will be used by all of AIPS++ . While it may be necessary to derive more specific MeasurementSets from the base MeasurementSet, it is our feeling that this should be avoided when practical. Initially we plan that both interferometric and single dish data will share the same MeasurementSet.
Sharing the same MeasurementSet does not mean that the table organization will be the same. One of the strengths of the planned MeasurementSet is that it is a repository of standard column names. A few columns will likely be required while most will be optional. Rather than derive specific MeasurementSets that require more column names, it is felt that the same result can be accomplished through member functions of the MeasurementSet. For example, one might have
Bool isSingleDish();
to verify that the MeasurementSet is a valid single dish MeasurementSet (it has the appropriate required column names) or
Bool isVLASet();
to verify that the MeasurementSet is a valid VLA MeasurementSet. If there are other member functions required by a specific type of MeasurementSet, then this suggests that a new class should be derived. However, since the MeasurementSet is primarily a Table that holds the data with little additional functionality, the need for specific derived MeasurementSets seems remote at this time.
As described earlier in this document, the key aspect of the new MeasurementSet is the use of an enumeration listing the columns with a predefined meaning. This replaces the multiple member functions, one for each column, that are found in the initial SDMeasurementSet. Here is one possible implementation that is being considered (using pseudo-C++).
class MeasurementSetInfo {
public:
// Columns with a predefined meaning
enum PredefinedColumns {
NON_EXISTENT=0, // "TRUE" columns exist.
DATA, // Data vector (spectrum, time series, ...)
ROW_FLAG, // The per row flag
CHANNEL_FLAG, // The vector of channel flags
...
OTHER};
virtual String columnName(PredefinedColumns which); // avoid typos
virtual PredefinedColumns columnType(String name);
virtual Bool isScalar(PredefinedColumns which);
virtual Bool isArray(PredefinedColumns which);
virtual Bool exists(PredefinedColumns which);
virtual Fallible< ScalarColumn<Int> >getIntColumn(PredefinedColumns which);
// ... for all column types.
};
MeasurementSetInfo defines all the standard MS columns and allows access to them in a way which catches typos at compile time. This could then be used within a MeasurementSet.
Once the basic form of MeasurementSet is agreed upon, the larger task of agreeing on standard definitions for column names will need to be addressed. In order for the UV Calibration and Imaging group to proceed early next year, both of the above will need to be in place. We expect that the next version of the MeasurementSet will be available for initial use by the end of December, 1994.
It is imperative that AIPS++ be able to fill directly from any of a number of telescope specific data formats. For 12-m OTF data, this is important for two reasons. First, the time spent converting 12-m data (which is in the UniPOPS Single Dish Data (SDD) format), is a significant part of the time spent creating an OTF image. It is unlikely that the calibration and imaging will be able to keep up with the projected data rates given the current time required for conversion from SDD to SD-FITS and from SD-FITS to an AIPS++ Table. Second, observers need to see the image as soon as possible, preferably as the data arrives so that the gridded image can build up as the data is deposited on disk. Mistakes in the observing parameters, pointing problems, unexpected emission (requiring the observer to map a larger region), atmospheric problems, etc, need to be dealt with as soon as possible so as to make the most efficient use of limited observing time. At 40 or more minutes for a modest OTF image, the observer simply can not afford to wait until the data has been taken to grid the data (wasting even more time) and finally to see the results. A simple mistake has the ability to waste up to an hour of valuable observing time.
The existing SDMSSource will be used as a starting point for development of a more general MSSource base class. From this base class the single dish FITS and SDD specific versions will be derived. The SDD specific version will be able to handle data from the 12-m as well as from the 140' telescope. Alternative binary table FITS fillers should be easily derivable from this base class using the single dish FITS filler as an example. We expect to be able to fill the general MeasurementSet from SDD data by the end of January, 1995.
We expect to be able to implement on-demand calibration of position switched data using virtual columns shortly. It is likely to be a few days worth of work and should be easily completed by the end of December, 1994.
The other planned improvement is to implement a true ``sucking'' TelescopeComponent constructor. As described previously, this would allow the user to edit the calibration tables without altering the raw MeasurementSet. This is estimated to be about 1 weeks worth of work. However, the ability to truly edit these tables depends on a working table browser. Furthermore, the coordination between a running application which is doing calibration on-demand and a table editor that would allow the user to alter the calibration tables used by the application (allowing feedback to the application) needs to be worked out.
We will develop additional TelescopeComponents as required. The simple nature of most current single dish calibration techniques means that this step is not difficult for most planned calibration techniques. Some TelescopeComponents may be sufficiently simple that they can be implemented as a glish script. This ability provides the flexibility that single dish observers require (where a new observing mode or a slightly different calibration technique are often developed during an observing session) while maintaining the framework of the TelescopeComponent and TelescopeModel approaches to calibration.
The ultimate near-term goal of the single dish calibration and imaging effort is to produce a tool for gridding 12-m OTF data as it arrives on disk in the raw telescope file during observation. The user will be able to see the gridded image build up as the data is taken, row by row. Some analysis of the data will be possible (the user will be able to capture snapshots by getting copies of the data cube at any phase of the imaging). The user will have control over the calibration (both the type of calibration, pre-imaging as is now down and post-imaging, described below, as well as the calibration parameters as described above).
Once the capability to read SDD is available, most of the above functionality should be achievable in a few weeks. We expect to have a useful OTFtool by the end of February which will display the gridded image as the data arrives and allow the user to analyze a snapshot of the gridding process at any time during the process. Control of the calibration step will likely take some additional time (this depends on an addition described in the next section).
The MeasurementSet is flexible. The price for flexibility is that applications must do a fair amount of checking to verify that a specific MeasurementSet has the characteristic they desire and to deal with missing information appropriately. In other words, the fundamental telescope-specific nature of a MeasurementSet is not convenient for general, non-telescope-specific applications. We find a need for a more general class to simplify the interface to MeasurementSet. Some non-telescope-specific applications that would use such a class include imaging applications (thus allowing these imaging application to image data from different telescopes), data editors and displayers where the basic astronomical information is to be edited and displayed.
Our current term for such an object is a Spectrum, although with a suitable renaming, this object might be general enough to encompass time series and other vector objects. A SpectrumSet is a collection of Spectra presented in some order. A SpectrumIterator may be a more appropriate name. These concepts are not specific to single dish data.
The SpectrumSet as a whole has
Each Spectrum has
These would be base classes. The obvious implementation is to attach them to a MeasurementSet. However, they might also be attached to an on-line system or even to an image cube for single dish data.
The later observation, that a single dish SpectrumSet can be attached to an Image, will allow us to do post-imaging calibration. The TelescopeModel can be instructed to apply a calibration to a SpectrumSet. If the SpectrumSet is attached to an Image, this would immediately alter the image.
Most of the discussion in this document concerns spectral line calibration and imaging. We have been driven by need rather than complete functionality. UniPOPS' most obvious deficiency is in dealing with complex spectral line data sets and we have chosen one clear example of that to focus on, 12-m OTF spectral line data. The MeasurementSet is sufficiently flexible to allow for continuum data. Currently it is not high on our priority. We would, however, like to fill a MeasurementSet with continuum data shortly after we have a useful OTFtool simply to demonstrate that it can be done.
One of the strongest aspects of UniPOPS is it's versatile procedure language. The 10 1-dimensional arrays in UniPOPS can be manipulated directly, element-by-element, within a procedure or through a number of specific verbs, which operate on an entire vector. glish is more versatile since vectors (and object of higher dimensionality) are built in to the language. It will be possible to replace the functionality of many UniPOPS verbs with glish scripts.
One long term task is to mirror as much of the functionality of UniPOPS as possible with glish scripts. Specific applications (i.e.,using C++rather than glish) will be created where required (by poor execution times). This is a time consuming but not particularly challenging task.
AIPS++ will gradually replace UniPOPS as the single dish package of choice within NRAO. In order for this to be a replacement of choice, AIPS++ must contain all of the functionality of UniPOPS. As stated above, this is a time consuming but not particularly challenging task. We expect to have most of the key functionality of UniPOPS available within an AIPS++ environment within the next year.
Users of UniPOPS as well as other analysis packages will not choose to use AIPS++ unless it does something not available in their current package or if AIPS++ simply does that thing better. UniPOPS is adequate for most single dish users. It is unlikely that simply supplying AIPS++ with the functionality of UniPOPS will be sufficient to draw users to AIPS++ . This is especially true for experienced UniPOPS users. The payoff for learning a new ``language'' must be worth the price. Consequently, the primary focus of the single dish calibration and imaging effort within AIPS++ is initially on providing tools to do tasks not available or difficult within UniPOPS. One such example is the OTFtool described previously.
The GBT is relying on using AIPS++ for it's data analysis needs. There is sufficient time that this should not be a problem. However, there are specific near-term analysis needs for the GBT that can be met by AIPS++ . One specific example is the holography test planned for early 1995. We believe that there are sufficient parts of the AIPS++ infrastructure currently in place and sufficient experience with single dish data for AIPS++ to provide the analysis needs for this test. We will work with the GBT to formalize the list of requirements for this test and should begin work on this as soon as possible.
The next analysis software need at the 12-m after OTFtool is support for the 8-beam receiver. This will be available for use sometime during 1995. As with the GBT, a fair amount of planning needs to occur so that we can anticipate the analysis needs rather than react to the analysis needs as we are doing with OTFtool.
Users will use AIPS++ only by choice. They payoff must be worth the price of learning the new language. Single dish analysis needs are generally not well supported. The support staff frequently have their hands full maintaining an existing package and providing support to new instruments and observing techniques.
There is one primary reason why single dish analysis will migrate to AIPS++ (at least from UniPOPS, although this same reason likely applies to other packages). UniPOPS is difficult to maintain. New instruments and observing techniques are increasingly difficult to shoehorn into the old analysis package. UniPOPS deals well with moderately sized, 1-dimensional chunks of data. It offers little help in managing the data base (keeping track of which objects were observed when with what instrument, etc). It is increasingly common for single dish telescopes to generate huge data bases during a single observing session. Large regions of the sky can be mapped with ease. Multi-feed (multi-beam) arrays are becoming common. Even the individual spectra are getting quite large (in terms of number of channels). These advances in instrumentation and observing techniques strain or break UniPOPS.
One solution to these advances would be to use a different package to handle the new techniques while retaining the old package for the more mundane tasks it does so well. This is the immediate solution being used at the 12-m to handle OTF spectral line data. It is also the technique they have used for quite some time to handle complex continuum mapping observations. OTF data from the 12-m is currently handled within classic AIPS. It is adequate although it does not allow the flexibility of calibration that is desired. It is also awkward for the observer to use two different analysis environments. Clearly it is preferable for all analysis to occur within the same system.
One major obstacle that will keep single dish observers from using AIPS++ is filling a MeasurementSet from their native data format. A primary job of the single dish development will be to assist consortium members and ultimately non-consortium members in getting their data into a MeasurementSet. A large portion of single dish observers use the CLASS reduction package. The single dish calibration and imaging group will have as a goal the conversion of CLASS data into AIPS++ as well as AIPS++ data into CLASS format. This will make it easier for a number of other groups to begin to explore AIPS++ .