 News |
FAQ
|
| Search
|
Home
|
| Getting Started | Documentation | Glish | Learn More | Programming | Contact Us |
|
| Version 1.9 Build 1367 |
|
T.J. Cornwell, NRAO
1997 October 8
AIPS++ is now in second beta release (of three expected before a limited public release). In addition, AIPS++ is in use at a number of consortium observatories:
In Single Dish support, we continued development of the single dish analysis program, concentrating on improving the plotter capabilities. Much work this last quarter has been dedicated to improving our handling of SDFITS. We continue to provide support for the use of AIPS++ by the Green Bank Telescope engineers.
In Synthesis support, as planned, we have re-written the gridding and FFT code to improve the performance for both spectral line and large continuum processing. We also considerably improved the performance of the sort and selection routines. We added support for manipulating lists of components, most particularly converting them to images.
We continue development of a sophisticated Glish-based tool for visualizing visibility data from a MeasurementSet. Jan Noordam is writing this application entirely in Glish. As such it is an excellent test and demonstration of the Glish programmability. For efficiency purposes, we have provided C++-based access to the MeasurementSet via a Distributed Object. This type of development (astronomer in Glish, project support via C++ and Glish-bound Distributed Objects) is a good model for future development by astronomers in AIPS++. The UV visualizer will be released into the system after testing by the local staff in Dwingeloo.
We have had discussions with Bob Sault of ATNF over the completeness of the Measurement Equation implementation. None of the hard cases that came up in discussion are thought to present serious problems.
In Measures, we continued adding capabilities to the system. We now have the JPL DE-200 ephermeris built-in so that the position of e.g. Mars is available from C++ and from Glish. We continue to add Table support for Measures. We expect this to be very useful in many different applications.
In Glish support, our main activity was to respond to bug reports. We are concerned that our development of GUIs in Glish may be stressing Glish too much. An initial implementation of a sophisticated plotter object shows unacceptable memory and resource usage. Implementation of garbage collection inside Glish has not helped this significantly. We are currently re-implementing some key parts of the GUI framework to see if the memory usage can be improved. In AIPS++ Infrastructure, our work continues to be driven by the needs of applications development. We also continue to develop a graphical user interface for standard AIPS++ Distributed Objects, based upon the tk widgets now bound to Glish. The Table system was augmented by the completion of the Table Query Language (TaQL), an SQL-like query language. We made considerable improvements to the tablebrowser: editing of values, user-controlled formatting, generation of sub-tables by TaQL-query, improved scrolling speed, etc.
In Visualization and Image Analysis, we continued development of the image display library, a joint undertaking of ATNF (Tom Oosterloo) and BIMA/NCSA (John Pixton). The core library is finished and well-documented but as yet only a few demonstration and test programs have been written. This work is now in hiatus following the departure of both Tom and John. We expect that some small amount of work will occur in the next few months, but our major development in this area will resume with the arrival in February of David Barnes at Epping to work in AIPS++.
In the System area, we started (and are closing to finishing) a port to the IRIX native compiler, including optimizations for parallelization.
We also continued an evaluation of the Kuck and Associates C++ compiler. This includes a number of optimizations not present in other C++ compilers and may improve the performance of our code. This work now continues at NFRA., If this is successful, we plan to use this compiler for generating binaries for distribution.
We have normalized and documented the names of user control variables.
We have started re-implementation of our ``Do-It-Yourself'' template handling mechanism. The goals of this are to cut down compilation time by suitably grouping templates and by eliminating un-used templates. We expect to complete the implementation of this approach in the next quarter.
We held long dicussions of proposals for rearranging the directory structure. This will be needed soon to support the delivery of data to AIPS++ sites (for e.g. the Measures system), and also to rationalize the distribution of source code files inside the system. In view of the complicated nature of the set of changes that must be made, we have deferred this until after the next beta release.
In Documentation, we continued adding material to the Reference manual. We have deferred production of a cookbook until the interfaces, most particularly the GUIS, settle down. We have hired a Information Services Coordinator in Socorro (50% time) to work on improving the Web interface for our documentation and to provide technical editing for our documentation.
In Management, we conducted a Birds-Of-a-Feather session (jointly with the AIPS project) at the ADASS meeting in Sonthofen. In addition, we gave a computer demonstration of AIPS++. Cornwell, Glendenning and Kemball visited Dwingeloo for several days following the meetings. One purpose of the meeting was to discuss the proposed alternate implementation of the MeasurementEquation (``Megi Number Servers''). After intense discussion, we decided to defer any further consideration for at least a year. In addition, we held discussions with the NFRA TMS group and the JIVE Correlator group. The main purpose of these meetings was to exchange information on the use of the system by both groups. TMS is currently using AIPS++ as part of the operational interface of the WSRT telescope. JIVE intends to use AIPS++ for data acquistion and re-formatting. Both projects need loss-less mechanisms for exporting and archiving in FITS format. As a result, we have made this a high priority. JIVE is not expected to contribute back to the AIPS++ project in the short term but is interested in becoming a full partner in the longer term.
Tim Cornwell visited ATNF (both Epping and Narrabri) for three weeks to work with the local group on a variety of issues, most importantly the performance of the synthesis code in collaboration with Mark Wieringa.
The second beta release was made on September 11, 1997. Little feedback has been received. Some problems with FITS reading and writing were noted (and will be fixed in the next patch). There is some concern that the current model for spectral line processing needs augmentation by addition of a ``channel-0'' pseudo-continuum. Our response is to fix this in the short term by providing improved visualization capabilities, and in the longer term by incorporating channel-0 capabilities in the data access routines needed for cross-calibration.
The current list of milestones is as follows:
We are concerned about exhausting our beta-testers with too many releases so we intend to hold off on the third beta release until we have a clear core of new capabilities for testing.
The principal activity for the next quarter will be support of beta testing, and preparation of the next release.
In Single Dish support, we will continue development of the Single Dish analysis program, with the goal of making this part of the third beta release. We will continue work towards the formalization of the SDFITS standard, providing a draft standard and some example SDFITS files.
In Synthesis support, we will increase our activity considerably:
This activity will be coordinated via weekly phone conferences following the usual Monday Meetings.
In Measures, we will complete the addition of support for Measures in Tables.
In Glish support, we will continue to fix bugs as they occur in testing. We will issue release 2.6 of Glish.
In AIPS++ Infrastructure, we will continue to track the needs of applications programmers. We will finish the improvements to the table browser. We will integrate history and logging. We will finish the DO Version 2 GUI support including support for plug-in components. We will complete optimization and cleanup of the FFTServer class.
In Visualization and Image Analysis, we will do some minor testing and application development using the display library.
In the System area, we will complete the port to the KAI compiler. We will implement fully the improvements to the DIY template system that were described above.
In Documentation, we expect to make progress on the intermediate level documentation requested by our beta testers (cookbooks and migration guides). We will review our Web presence.
In Management, as the first stage in implementation of the Operations plan, we will set up the Quality Assurance Group.
People
For this quarter, the ATNF had 4 people working in AIPS++. These are Neil Killeen (25-75%, also local manager), Wim Brouw (90%), Tom Oosterloo (75%) and Mark Wieringa (30%). For the coming quarter, we will be three as Tom Oosterloo has now left the project and returned to Italy. We have hired David Barnes to replace Tom; David will join us in the middle of February 1998 after he completes his PhD. His prime software responsibility will be to continue implementation of the display library and to generate applications using it.
I would like to thank Tom for his contribution to AIPS++, in particular for designing and implementing (in collaboration with John Pixton) the display library, and wish him success in his future.
System
Visits
Nobody went anywhere. I should note here that the ATNF was unsuccessful in trying to get funding from an Australian government agency for AIPS++ international travel. In the past we have received some funding from them. We will apply again next year, but this shortfall will probably curtail some of our ability to travel in the coming year.
Tim Cornwell made an enjoyable visit (at least, we enjoyed it) to the ATNF for about 3 weeks during this quarter. We took the opportunity during his visit to hold some discussions about mosaicing and single dish integration with interferometers. Tim spent time with Mark Wieringa discussing the continuing effort to implement the Measurement Equation. He also discussed a range of ME implemention and functionality aspects with Bob Sault (one of its creators). With Wim Brouw and Neil Killeen he discussed how to incorporate system data into AIPS++. We also made an arrangement with Tim that Wim Brouw would spend some time implementing mosaicing related software later in the Measurement Equation development.
Individuals
Wim Brouw's responsibility is mainly to designing and
implementing the Measure and related classes.
Mark Wieringa works with Tim Cornwell implementing the Measurement
equation.
Tom Oosterloo was working on the display library.
Neil Killeen spends his AIPS++ time attending to local ATNF AIPS++ management
issues as well as generating image applications.
During the past quarter we had 6 people working on AIPS++. Those continuing from before were Dick Crutcher (20%), John Pixton (100%), Harold Ravlin (100%), Doug Roberts (75%), and Peter Teuben (10%). Dan Briggs joined the group in August at 100% effort; John Pixton leaves the group October 1. Accomplishments during the past quarter are described below under each person.
Dick Crutcher served as local AIPS++ project manager and as BIMA director in charge of our participation in AIPS++. In addition to usual management duties, he attended the ADASS97 meeting in Germany and presented a paper on "VRML and Networked Environments: New Tools for Networked Visualization". He also spent considerable time arranging for the ADASS98 meeting to be at the University of Illinois, sponsored by the University of Illinois Astronomy Department and NCSA. Crutcher became Chair of the Astronomy Department in August and moved the AIPS++ group from the Beckman Institute building to the Astronomy Building. Much effort was put into negotiations with NCSA about continuing support of the AIPS++ group once Crutcher's NSF Grand Challenge grant which has provided the majority of funding expires next year; these negotiations have not yet been concluded.
John Pixton has focused primarily on the AIPS++ display library. Working closely with Tom Oosterloo (CSIRO, Australia), he has tried to fill in more of the design details regarding the WorldCanvas. He implemented and tested all the code necessary for the successful display of images, including rescaling of the image and conversion of its real values into color values. He then needed to go back and implement translation ability in display lists and clipping control for the PixelCanvas to help optimize the redraw process and make it easy to draw the same object in different places on the screen. He completed documentation for the classes involved in the display library, and added the ability for the user to control the target buffer for graphical output, supporting double-buffering for smooth movies and overlay graphics. He also implemented but has not fully tested the code required to handle the rendering of RGB and HSV color images. He also attended the SIGGRAPH '97 meeting in Los Angeles August 2-9.
Harold Ravlin administers our AIPS++ workstations, which takes about 1/3 of his time. He spent considerable time on the move of all of our workstations from NCSA to the Astronomy Building. He attended SIGGRAPH '97. Aside from workstation support for the group, his primary responsibility was Aipsview. Much of this was bug fixes, including fixing some incompatibilities with Motif 2.x. He also changed the profile code so a PGPLOT preview window is available and the X display allows overlaid plots. The color and line style of the X display are also changeable. Contour line widths are now adjustable, and emphasized contours are available. Line widths now track better between profile and axis displays and X11 and PostScript outputs.
Doug Roberts finished porting the AIPS++ system to the NCSA Origin2000 computer using the SGI native C++ compiler. He attempted to install AIPS++ under g++ on the Origin2000, but the gnu software had been improperly installed by NCSA and he was unable to proceed. He has been editing the Aipsview manual to reflect functional changes that Harold Ravlin has been putting into Aipsview. He has tested the AIPS++ calibration, imaging, and deconvolution routines against the same ones in MIRIAD by processing the same BIMA data set completely through the two systems; he found them to give the identical results within the noise. He has also written a users' manual on processing BIMA data through AIPS++ to help BIMA MIRIAD users transition to using AIPS++. The spectral-line Beta version of AIPS++ has been installed on the Illinois Astronomy computer system. He has given two talks to the BIMA group here at Illinois on the background of AIPS++ and specifically how to use it to process BIMA data; the talks include a demonstration script to calibrate, image, and deconvolve BIMA data in AIPS++.
Dan Briggs joined the NCSA AIPS++ group in mid-August. He has spent the month coming up to speed on the local computing infrastructure, and preparing for a 6 week AIPS++ training session in Socorro which began on 29 September. His initial project within AIPS++ will be implementation of a wide field imaging task, similar to the SDE program, dragon. This may be in the form of extensions to imager or as a specialized task. Beyond duplication of existing capabilities, special attention will be paid to spectral line capability and high end parallelization on the Origin 2000 at NCSA.
Peter Teuben has continued work on the BIMA filler.
Local project members: Ger van Diepen (GVD), Michael Haller (MH), Jan Noordam (JEN, local manager), Friso Olnon (FMO), Henk Vosmeijer (HJV).
After ADASS, the AIPS++ Project Management paid a visit to Dwingeloo. The main topic was a discussion about interfaces between AIPS++ and TMS and the JIVE correlator. This included UVFITS-issues and the structure of calibration tables. It was further agreed that, in view of the large amount of work that remains to be done on the basic package, any plans for alternative implementations of the Measurement Equation would be shelved for a year.
The first module of the new WSRT correlator (DZB) was delivered at the end of August. This coincided beautifully with the second beta release of AIPS++. Subsequent commissioning of the DZB in conjunction with the new on-line system (TMS) uncovered the usual crop of small problems, but went well on the whole. When regular observations start this fall, AIPS++ will be an essential link in the use of a major instrument.
At this point, the use of AIPS++ is still limited. The emphasis lies on the use of the Table System and Measures by TMS, and the use of the Measurement Set as the output data format. Off-line reduction tends to take place in other packages like NEWSTAR and AIPS, and some extra effort is needed to streamline those two links. The expectation is that AIPS++ will be used more for off-line reduction as beta testing makes them more familiar with it, and (most importantly) when AIPS++ will feature cross-calibration. Friso Olnon rounded off his work on the prototype dzbfiller, a program which creates AIPS++ MeasurementSets from "header" data gotten from the TMS (Telescope Management System) database and "actual" data from the new digital backend, DZB. The classes have been released to the TMS group for further development, testing and maintenance. Olnon also started an upgrade of the wsrtfiller, which fills an AIPS++ MeasurementSet from WSRT archived data. The filler will be extended to handle multiple spectral windows (8 continuum bands for the DCB), multiple pointing positions (for mosaicking), and IF data (stored in WSRT-specific extra columns).
The uv-data visualiser (Jan Noordam) will play an increasingly important role in the near future. It has been split into two modules: a data-selector and a data inspection/manipulation tool. These two functions constitute the bulk of most application programs, whose actual processing algorithm usually only consists of a few lines of code. If well-designed, these two modules would make it possible to put together new applications very rapidly. This concept will be pioneered with the various WSRT setup programs, which require relatively little processing and are not time-critical.
"Regular" maintenance of the local AIPS++ system costs Friso Olnon somewhat less than one day a week. This more and more includes support to programmers using AIPS++ code (in the JIVE and TMS groups). In addition, Ger van Diepen and Henk Vosmeijer spend some time on inspection, debugging and repair. The inhale and sneeze operations are now usually successfull apart from some minor glitches, which are easily and quickly repaired. But about half the time we suppress the weekly inhale/sneeze to provide user/developers with a more stable environment.
Extra activities this quarter were the beta-0.9 release (preparation and installation) and the ADASS'97 conference in Sonthofen (including the preparation for AIPS++ demos during that conference).
The Table system has been enhanced considerably by Ger van Diepen, in particular the TaQL (Table Query Language).
The Table system has been adopted by the TMS-group in Dwingeloo. Therefore TaQL had to be extended with the possibilities of handling arrays and subqueries. The new possibilities are:
All functionality, except subqueries, is also accessible from C++.
A few other changes have been made to the table system and some utility classes:
Too little time could be spent on the Lattice module. A few additions have been made:
Under the guidance of Ger van Diepen, Michael Haller has spent a month learning C++ and AIPS++, especially the Tables and Measures modules. His first project is the implementation of Measures in Tables. He has also made initial investigation/analysis of the possible development of a BTrees module which is being considered for inclusion into AIPS++. He reviewed the new AIPS++ module Aipsrc prior to inclusion into AIPS++ package.
Some work has been done for the KAI compiler (which conforms the draft C++ standard). It appears that in the new standard the iostream package uses stdio instead of file descriptors. This required a change in the class FilebufIO and also resulted in a new class FiledesIO.
On September 1st Eelke Klein, a student from the HIO Leeuwarden, started working on a C++ Software Quality project. It consists of configuring QAC++ and assembling some metrics for the AIPS++ and TMS code.
Some UNIX administration support has been provided especially to the Linux systems.
The core NRAO AIPS++ group is now Cornwell (100%), Garwood (90%, 10% going to Unipops support), Glendenning (100%), Kemball(100%), Marson (100%), McMullin (100%), Schiebel (100%), and Uphoff (100%). In addition, Young of the AOC Computer Group works with us, and as described above, Murphy of the CV computer group also contributes. We continue to have one unfilled position, in Socorro, for a VLBI software specialist, and one (funded by the collaboration with NCSA) for a programmer to work on parallelization of the AIPS++ code. We expect to fill the latter position in a few weeks. We have also extended an offer to an Information Services Coordinator to work 50% on AIPS++ documentation.
Most of the NRAO staff work on general AIPS++ capabilities. The exceptions are Joe McMullin, who although he is an AIPS++ Project member, concentrates on Green Bank and GBT support concerns, and Wes Young, who is a member of the AOC Computer Division, and thus spends a substantial (and increasing) amount of his time on VLA-specific developments.
Wes has worked on a number of VLA specific capabilities in AIPS++, the most recent of which is a filler from VLA archive tapes into AIPS++ Tables. When completed, this will give VLA engineers the capability to examine VLA monitor data using AIPS++ in much the same way that GBT engineers can examine GBT monitor data. From there, it is a relatively short step to fill into a fully-fledged MeasurementSet, and thus give a path for visibility data directly from the Modcomps into AIPS++.
The principal activity for the next quarter was be support of beta testing, and preparation of the next release.
In Single Dish support, we did continue development of the Single Dish analysis program, with the goal of making this part of the third beta release.
In Synthesis support, we did continue development of the synthesis code, starting the design of a fully-fledged calibration system using the current Measurement Equation implementation. We did not yet finish a visibility data visualizer being developed by Noordam, but expect to do so within a few weeks. We did add the various capabilities needed for dealing with list of discrete components, as needed for WSRT calibration.
In Measures, we did not implement the data distribution scheme (see above). We did incorporate JPL planetary data in the Measures system. We did work towards but did not finishing adding support for Measures in Tables.
In Glish support, we did continue to fix bugs as they occur in testing. We did not yet issue release 2.6 of Glish.
In AIPS++ Infrastructure, we did continue to track the needs of applications programmers. We made a number of revisions to the Lattice, Image, and Table classes. We did improve the table browser.
In Visualization and Image Analysis, we did continue development of the image display library. We do have a complete design and description, an implementation of the major components, but we do not yet have any example applications.
In the System area, we did not yet complete the port to the KAI compiler. We did not simplify and consolidate the disposition of Glish script files in the code management system.
In Documentation, we did not yet make progress on the intermediate level documentation requested by our beta testers (cookbooks and migration guides).
In Management, we did demonstrate AIPS++ at the ADASS in Germany in September and conduct an ADASS Birds-Of-a-Feather session on AIPS and AIPS++. We did not proceed with the planned name change. We did complete the operations plan.
In this section, we give the names of people in the various AIPS++ groups and the nominal fraction of time allocated to AIPS++.
The ATNF group is: Neil Killeen (25-75%), Wim Brouw (90%), and Mark Wieringa (30%). Tom Oosterloo's position is to be filled by David Barnes in mid February.
The BIMA/NCSA group is: Dick Crutcher (25%), Dan Briggs (100%), Harold Ravlin (100%), Doug Roberts (75%), and Peter Teuben (10%).
The NFRA group is: Ger van Diepen (100%), Michael Haller (100%), Jan Noordam (25%), Friso Olnon (50%), and Henk Vosmeijer (50%).
The NRAO group is: Tim Cornwell (100%), Bob Garwood (90%), Brian Glendenning (100%), Athol Kemball (100%), Ralph Marson (100%), Joe McMullin (100%), Pat Murphy (30%), Darrell Schiebel (100%), Jeff Uphoff (100%) and Wes Young (90%). We have an open position in Socorro for a VLBI software specialist, and an open position funded by the NCSA-NRAO collaboration for someone to work on parallelization of AIPS++ code.
Thus, in aggregate, we have 24 people contributing just over 17 FTEs to the AIPS++ Project. Of these, 11 are employed by NRAO, and contribute 9.1 FTEs. The numbers for the other partners are: ATNF 3 and 1.45, BIMA/NCSA 5 and 3.10, NFRA 5 and 3.25.
