News| Getting Started | Documentation | Glish | Learn More | Programming | Contact Us |
|
| Version 1.9 Build 000 |
News |
AIPS++ Quarterly Report: 1995 Q3
Tim Cornwell, AIPS++ Project Manager
October 5, 1995
This report summarizes the status of the AIPS++ Project at the end of the third quarter of 1995. It describes the achievements during that quarter and reviews the developments promised in the previous quarterly report. The report then gives a list of planned developments for the fourth quarter of 1995. Reports from each active AIPS++ site are included in an appendix.
Progress was made in many areas of the project. The most important achievement was the participation in the tests of the GBT Monitor and Control system on the NRAO 140' telescope. This was the first mission-critical use of AIPS++ and demonstrated the usability and reliability of several AIPS++ components: the Glish system, the Table system, and various fitting and plotting capabilities. In other areas, development proceeded as required to support this work and application development in other areas. We continue with our strategy of developing a few key applications with the goals of testing the AIPS++ infrastructure and attracting early adopters to AIPS++.
The consortium interactions are working well. The Project Manager visited ATNF in August (see the ATNF Site report in Appendix A) and took part in a number of useful discussions. This has led to a collaboration between ATNF and NCSA/BIMA on Image Analysis and Visualization.
In this section, I describe the work that actually was done in the last quarter. The overarching rationale behind all of the work performed in AIPS++ remains the push to applications development, both in Single Dish and Synthesis.
In Single Dish support, we continued our participation in the tests of the GBT Monitor and Control system on the 140' telescope. In these tests, designed to test both the M&C system and various parts of the electronics, AIPS++ was to provide the analysis capabilities. The goals were to load monitor and astronomical data from a number of FITS binary tables into the AIPS++ Table system and to provide analysis and plotting capabilities. The timeline was this development was agreed in collaboration with the GBT M&C group in a series of meeting starting in late March 1995. The first tests took place in July 1995. This was the first "mission-critical" use of AIPS++, and so our ability to fulfill expectations was on trial. AIPS++ met all the specified goals in functionality. In addition, the combination of the Glish system and the plotting and analysis tools was found to provide an accommodating environment for debugging. To track down unexpected behavior in some of the data, the group running the tests (not from the AIPS++ group) were able to try different and unanticipated ways of plotting the data. A demonstration of this type of use of AIPS++ is available on the WWW. These tests are continuing throughout the fall. AIPS++ will continue to provide support but it is anticipated that much of the burden of writing Glish scripts for the analysis will be able to be assumed by the GBT data analysis group.
In Synthesis support, we decided to adopt the Hamaker, Bregman, Sault formalism for as the basis for AIPS++ polarization calibration and imaging. A number of memos have been written, principally by Noordam and Cornwell, on this approach (see the AIPS++ Implementation Notes). Cornwell and Wieringa are now embarking upon design of classes.
In AIPS++ Infrastructure, Glish has seen substantial enhancements and improvements. These and the plans for the future are documented in the Glish Development Plan. By agreement with the author of the package, Vern Paxson, we have assumed responsibility for distributing Glish. This is good for AIPS++ since Glish lies at the heart of our development strategy in both command line interface development and task control. The design of the support for task control within AIPS++ has been slower than expected, mainly due to oversubscription of the time of Brian Glendenning but it is now complete and we are starting on implementation. We made a number of substantial improvements to the Lattice and Image classes that will be vital for applications development. We completed a design for a generic filler program to load telescope data into AIPS++, and are now proceeding to implement the design. We designed and started to implement a tiled storage manager for the AIPS++ Table system. We started design and implementation of the Coordinate classes (by Wim Brouw at ATNF).
In Visualization, AIPSView went through a number of alpha test phases and was released in version beta 1. The specific improvements to AIPSView are described in the BIMA site report. In addition to the 2D display present in AIPSView, we have developed a prototype for 3D visual analysis based on the OpenGL and OpenInventor libraries. Inventor AIPSview allows full 24-bit color, simultaneous display of multiple data cubes mixing or matching among simultaneous display modes integrated into a single view. Display modes include 3D slicing parallel to the three faces of data cubes, isosurface rendering, contouring, volumentric rendering, and others.
In System, we participated in beta testing of the new Sun native code C++ compiler. As a result of the feedback that we gave, we are now able to adopt the compiler to some considerable advantage in speed of compilation and ease of use. We installed AIPS++ on an SGI PowerChallenge computer. We added an e-mail exploder archive. We can now do rebuilds of AIPS++ in parallel on a number of CPUs.
In Personnel, the Project center completed hiring of two new members in Socorro. These will bring skills in a number of areas: radio and optical Synthesis, signal processing, parallel processing in C++, grand challenge computing, and numerical astrophysics. There have been a small number of short-term visits between sites. Mark Wieringa is visiting the AOC for 6 months to work on Synthesis processing.
In Documentation, we started a Documentation group headed by Alan Bridle. Most of the work in the quarter was in the area of User Documentation.
In Management, we negotiated a collaboration between NCSA/BIMA and ATNF on Image Analysis and Visualization (see below). We took part in discussions on the possible future use of AIPS++ in a number of areas: pulsars and multibeam observations at ATNF, and VLBI correlator support at NRAO and the JIVE. None of these discussions have yet led to any concrete action. We commissioned new requirements documents for NFRA and ATNF, describing the most urgent needs for AIPS++ development.
This section has been written to mirror the section in the last quarterly report on plans for 1995 Q3. The purpose is to delineate goals and actual achievements and to establish whether the Project has any systemic problems in keeping to agreed targets.
In Single Dish support, we chose to make our highest priority support of the GBT systems integration work on the NRAO 140' telescope. Because of this and a lack of support at the telescope, we dropped development of the On-The-Fly mapping tool developed for and installed at the NRAO 12m telescope. We worked on specification of SDCalc (for analysis of Single Dish spectra) and SDImager (for imaging from Single Dish data) but did not reach a final specification.
In Synthesis support, we changed strategy as described above. We had planned to develop tools for advanced imaging using the Briggs NNLS deconvolution algorithm in a selfcalibration cycle. Instead, we adopted the Hamaker-Bregman-Sault formalism. The reason for this change in strategy is that it allows unified development of calibration and imaging for all consortium interferometers (and probably single dishes). As a result, the telescope-specific lines of development that previously showed up in the development plan are expected to be substantially less time-consuming. Little actual time was lost in this change in strategy. It is expected that the Briggs NNLS algorithm will be implemented once the basic framework is in place.
In AIPS++ Infrastructure, we did develop the Glish system (the User Interface and Control Hub) according to the Glish Development Plan. We did not yet complete design of the Coordinate classes but it is proceeding satisfactorily. This is expected to be done by the middle of the next quarter. We are continuing to add support for tiled hypercube access to the AIPS++ Table System.
In Visualization, we did continue to add functionality to AIPSView but not as directly in response to need as hoped. In addition, extension of AIPSView to include 3D visualization did continue but will change with the advent of a collaboration between NCSA and ATNF to work on Image Analysis and Visualization We did select a package for line graphics inside AIPS++ (the Fermilab widgets) to replace the commercial widget (Xrt/Graph) currently used in the GBT work.
In the System area, we did not add support for automated testing of AIPS++ classes. We did investigate use of the new Sun native code C++ compiler and are adopting it. We deferred re-assessment of any possible use of the IBM compiler.
In Personnel, we completed hiring of two positions in Socorro, one of which will be supported initially by funding from NFRA. The arrival dates are 1 November and 1 January. Wieringa is visiting Socorro for a 6 month period starting in September (Funding for this comes from ATNF and NFRA).
In Documentation, we did continue the development of the overall documentation system, paying some more attention to the programmer level documentation.
In Management, the Project Manager worked on communication by visiting ATNF and paying particular attention to a new requirements document. We did not formalize work structuring by adopting approaches similar to those proposed by the NFRA Software Engineering Group. This has been deferred pending more experience with the results from that group.
In reviewing the work promised and that achieved, we were alert to serious systemic problems in meeting targets. The target dates system for tracking work has been quite satisfactory. The biggest problem in keeping to target dates has been the unexpected. Key personnel who know many things about the system are particularly liable to this since they are best able to solve these unexpected problems. The AIPS++ code maintenance system is generally very robust but it is an example of an area where the unexpected can occur. Mark Calabretta has provided very good response to all problems in this area, but it does take a large fraction of his time that is allocated to AIPS++ with the result that some planned system-related enhancements have had to be deferred. Brian Glendenning is another key person whose advice is sought after by others in the Project. However, leaving aside interruptions, most of the AIPS++ group are quite accurate in estimating the time required to complete a project, if the task is well-defined. If the task is not well-defined then estimation is of course difficult.
In summary, I think it is clear that the project has overall had a good success rate at setting and meeting targets and that there are no serious systemic problems that need rectifying. The one caveat is that perhaps we could be more conservative in allowing for interruptions in the work of key personnel.
The development plans for the next 12-18 months are given in the AIPS++ Development Plan . Here I summarize the expected developments over the next quarter. Again, the rationale behind the priorities is the push to applications development.
In Single Dish support, we will continue support of the GBT tests. We will consolidate some of the work performed so far, principally by working to further improve some of the Glish scripts. We will start on a path of evolving the existing capabilities into a longer term, more substantial single dish capability. A development plan for Single Dish is targeted for mid November.
In Synthesis support, we will continue development of the measurement equation for a general interferometer. We expect to be able to make images, deconvolve and perhaps self-calibrate polarization measurements. It is probable that the initial goals will be a Clean algorithm for simultaneous deconvolution of Stokes I,Q,U and V, and self-calibration of the antenna-based polarization leakage terms.
In AIPS++ Infrastructure, we will finish work on the tiled storage manager for the Table system (needed for efficient access of multidimensional data). We will finish design and start implementation of classes for Task control and interaction. We will continue with the development of Coordinate classes now proceeding under the supervision of Wim Brouw. We will write improved linear algebra classes (needed for the Coordinate classes, and other areas).
In Visualization and Image Analysis, ATNF and NCSA will collaborate. ATNF will be primarily responsible for Image Analysis and volume rendering in AIPSView, and NCSA will be primarily responsible for Visualization, 1D graphics, and the GUI for visualization. We will use a commercially available library, OpenGL/Inventor, for high-level 3D visualization work. Volume rendering algorithms developed using Richard Gooch's Karma library will be implemented in AIPSView. We will investigate the feasibility of incorporating vector graphics (the Fermilab widgets) into AIPSView. A small pilot project in Visualization and Image Analysis will be started, with the target being interactive fitting of either a galaxy model or Gaussian profiles to a spectral cube.
In the System area, we will implement shared libraries (thus reducing executable size), start using by default the new Sun native code compiler (speeding compilations substantially), implement a bug reporting system, and implement release versions. We will move the AIPS++ master code respository to Socorro.
In Documentation, we will continue the push for user and programmer documentation. We will start development of some of the software tools needed to extract documentation from code and to interface to the design of tasking inside AIPS++.
In Management, we will continue to monitor the success of the NFRA Software Engineering Group with a view to possible adoption of some of their methods.
In addition, we will give a number of presentations at the annual Astronomical Data Analysis Software and Systems meeting in Tucson.
Approved by Board.
The ATNF currently has 3 active workers, as well as Neil Killeen, the local manager. These are Mark Calabretta, Wim Brouw, and Mark Wieringa. Bob Sault acts in a consultant mode.
Mark Calabretta's main responsibility is to the code distribution system. His time in the last quarter has been spent on :
In addition, as a project separate from AIPS++, Mark worked on writing a Fortran library for WCSLIB (World Coordinate System) which be used by AIPS++. Including this, Mark has spent approximately 60% of his time on AIPS++ related work. His nominal AIPS++ allocation is 50% at present. This level will be sustained indefinitely.
Wim Brouw's responsibility is mainly to designing and implementing the Measure and related Classes. His time has been spent on:
Wim has spent 10% (July), 30% (August) and 60% (September) on AIPS++. His nominal contribution is 50%. At present, this level will be sustained for at least the next year.
Mark Wieringa is a part of the team working on the Synthesis related classes. He has just gone (beginning September) to Socorro for 6 months. Following his return from 3 months in Dwingeloo earlier this year, Mark's AIPS++ activity has been low because the Synthesis team was not yet assembled. This was acceptable to the management. He has spent his AIPS++ time on:
Mark has spent 10% of his time on AIPS++ before Sep 8, and 90% of his time thereafter. His nominal AIPS++ allocation is 50%. Mark's involvement is negotiated yearly with Narrabri since he has been seconded to the project from he Narrabri Computer Group. Currently the agreement is until mid-1996.
Neil Killeen currently does no actual code work and spends his AIPS++ time attending to local ATNF AIPS++ management issues. These include:
Neil has no nominal AIPS++ time allocation, and the load varies tremendously. Averaged over the last quarter, Neil has spent 20% of his time on AIPS++, with peaks of close to 100% in the weeks before Tim's visit.
Tim Cornwell visited the ATNF for 1 week in August 1995. This was useful for all parties, and numerous discussions with the astronomy community were held. In particular, the Parkes multibeam project is considering using AIPS++ as their software environment.
The ATNF has CSIRO/IISE AIPS++ demonstrator funding for two 3-year positions. Currently, we have been allowed to merge these into one 6 year position in order to make an attractive offer to the person we would like to hire; he will make a decision soon (early October). It anticipated that the position will involve visualization collaboration with the Radiophyics virtual workbench project currently starting up, as well as serving core AIPS++ needs in the image analysis and visualization areas.
AIPS++ activity at NCSA during the past quarter was entirely in the area of continuing development of visualization software. We are working on two separate visualization systems. One, called AIPSview, requires only unix, C++, X Window, and Motif, in order to meet the AIPS++ criterion that no (other) software which costs money be used. The other, here called Inventor AIPSview, is built on OpenGL and OpenInventor, for which run-time licenses are free on SGIs but which cost several hundred dollars on other platforms. Progress on each tool is described separately.
AIPSview provides display of 2D images, display of any orthogonal 2D slice from a 3D data cube, interactive display of values, coordinates and line profiles, animated sweeping (movies) through slices along any dimension, synchronized picking of coordinates, data values, profiles for multiple data sets, synchronized animated sweeping of multiple data cubes, blinking between images, a complete colormap editor, and other features. We completed phases 2 and 3 of our alpha program (involving AIPS++ and selected external testers) for this new software tool for two-dimensional visual data analysis, had an NCSA internal alpha 4 release, and released a beta 1 version, with source, to the radio astronomy community. We established a WWW display describing radiosynthesis work at NCSA (http://monet.ncsa.uiuc.edu) which includes a link to our AIPSview tool; the tool is described, there is a multi-media tour of AIPSview capabilities, and there are download capabilities for both executables and source code. AIPSview source was made available for checkin to AIPS++.
Specific AIPSview activities included the following:
11. Demo of AIPSview at BIMA Board meeting, August 26. Preparation for invited talk and demo of AIPSview and Inventor AIPSview (see below) at ADASS.
We have developed a prototype for 3D visual analysis based on the OpenGL and OpenInventor libraries. Inventor AIPSview allows full 24-bit color,simultaneous display of multiple data cubes mixing or matching among simultaneous display modes integrated into a single view. Display modes include 3D slicing parallel to the three faces of data cubes, isosurface rendering, contouring, volumentric rendering, and others. Development during the past quarter has included a complete change of the GUI to one easier to use, finishing a volume renderer based on optimized polygon drawing with transparencies, adding this renderer as a subclass to the Inventor class libraries, investigating using SGI 3D texture hardware for volume rendering, updating the slice viewing by integrating our SvSlice class into an Inventor subclass, updating rendering intensity as height by integrating our Surface class into an Inventor subclass, working on some tools to enable easier viewing of 3D, such as tools that 'rock' the data for volumetric and other data representations. Inventor AIPSview has been released to NCSA friendly users. Additionally, we are exploring the design decisions that must be made in order to use the software across both desktop flat and stereoscopic displays and advanced virtual environments such as the CAVE, PowerWall, and ImmersaDesk.
Ravlin attended Siggraph 95; Baker and Baxter will attend Visualization 95;Crutcher will attend ADASS 95 and demo AIPSview and Inventor AIPSview; Crutcher, Baker, and Pixton will attend Supercomputing 95 and present a demo which includes Inventor AIPSview in a virtual environment), Baker will attend the first VRML conference.
Percentages are actual, not nominal, during the past quarter:
Richard Crutcher: 10% time AIPS++ NCSA site and personnel management.
15% time AIPSview planning, testing, writing user documentation.
Polly Baker: 25% time AIPSview design, development.
50% time Inventor AIPSview design, development.
Harold Ravlin: 80% time AIPSview development.
20% time managing NCSA AIPS++ workstations.
John Pixton: 50% time AIPSview development.
50% time Inventor AIPSview development.
George Baxter: 50% time Inventor AIPSview development.
The NRAO AIPS++ group is split into two subgroups. In Socorro, the group works on Project Management (Cornwell and Glendenning), Synthesis support (Cornwell, Glendenning and Wieringa (visiting)), Documentation (Young), and Library Support (Glendenning and Roberts). In Charlottesville, the group works on Single Dish support (Garwood - group leader, Schiebel, and Shannon), Glish support (Schiebel) , Graphics (Shannon), Documentation (Bridle - group leader, Schiebel), and Library Support (Garwood, Schiebel and Shannon). All group members are 100% AIPS++, except Garwood (90%), Young (75%), and Bridle (50%).
We installed a server (tarzan.aoc.nrao.edu) at the AOC to handle development by AOC AIPS++ group members. The Charlottesville server will remain there. The master AIPS++ installation will move to the AOC this fall.
All members of the Charlottesville group have made extensive visits to Green Bank to support the GBT work. In addition, Paul Shannon visits for two days every week.
The group in Socorro is no large enough to support bimonthly technical meetings. We have just started such a series of talks.
The local project members: Ger van Diepen (GVD), Jan Noordam (JEN, local manager), Friso Olnon (FMO), Jayaram Chengalur (JNC, local Project Scientist).
This quarter has seen the emergence of a matrix-based Measurement Equation (M.E.) for a generic radio telescope. It seems to open the way to a truly generic formulation of AIPS++ calibration and imaging, which will be valid for all existing and planned radio telescopes. It is based on three related papers ("Understanding Radio Polarimetry I,II, and III") by Hamaker, Bregman and Sault, which have now been accepted by A+A. Noordam has written a series of AIPS++ Implementation notes on the subject.
Olnon has finished his concept study for a WSRT uv-data `filler', followed by a detailed design. The latter is `generic', i.e. it can be used for other telescopes than the WSRT. Implementation has started.
The local AIPS++ installation in Dwingeloo causes very few problems. Olnon checks the weekly `inhales' and makes small changes in the configuration where necessary. At the request of the Center he made a full installation, to test procedures and instructions.
Van Diepen continues to work on the Table System, under the guidance of Brian Glendenning at the Center. The current emphasis is on the tiled hypercube storage manager. The design is finished, and implementation has started. In addition, the Table Client for Glish has been considerably extended. In September, Van Diepen spent three weeks at the Center in Socorro, as he does regularly.
Olnon is responsible for the AIPS++ modules `Utilities' and `Inputs', in particular the documentation of their public interfaces.
Van Diepen has devoted some time to the evaluation of QAC++. This package analyses C++ code to check its compliance with standards. It also generates various metrics for assessing the complexity and maintainability of code. This work will continue.
NFRA shares with ATNF the cost of sending Mark Wieringa to the Center, and contributes to the salary of one of the new programmers. NFRA is still looking for an experienced AIPS++ application programmer, to strengthen the local team in Dwingeloo.
NFRA is increasing its general level of experience in the areas of OOP and C++. All members of the local AIPS++ group and the TMS project team have been exposed to a C++ course in September. Olnon has made a draft "C++ Coding Standards at NFRA".
Olnon and Van Diepen are members of the NFRA Software Engineering Group, led by Marco de Vos. Some of the standards and procedures developed by this group may find their way to the AIPS++ project. All SEG documents, which are accessible via the SEG pages on the web, are in English (URL: http://www.nfra.nl/~olnon/seg).