No Title

This document reports on the status of the AIPS++ Project at the conclusion of development cycle 1.5. We discuss the overall status and the work completed in the development cycle. We also review the health of the package as revealed by an analysis of the defects.

Bearing in mind the increasing interest in AIPS++, we have written this report with a newcomer to the project in mind, so some material well known to readers of these reports may be repeated.

Overview of project status

Development priorities:

The AIPS++ Project development priorities reflect the fact that AIPS++ is a package for astronomical data reduction, geared towards scientific end users. The project management has steered AIPS++ firmly towards this objective over the past several years, and this focus has intensified in the cycle. This objective is expressed as the need to achieve scientific completeness and to expand the use of the package in the community as a tool for producing scientific results from existing and future scientific instruments.

The natural evolution of the project has proceeded through overlapping stages of requirements specification, design, infrastructure development, applications development, and operational deployment of the package in the user community. At present we are in the closing stage of this process, best described as a phase of scientific integration. This phase encompasses refining the package for scientific completeness, as defined in terms of scientific data reduction capabilities and its ease of user by scientific end users, as well as proving the robustness and correctness of all data reduction paths.

Development priorities are established in AIPS++ for six-month development cycles, each culminating in a public release. The development plan for each release establishes a work breakdown structure, expressed as code development or other targets. This is available for v1.5 at:
http://aips2.nrao.edu/docs/notes/239/239.html. We have ensured that the priorities in the development cycle reflect the overall scientific direction for the project outlined above. Infrastructure development, in particular, has been tapered off and carefully assessed against competing needs in scientific integration. These themes have been emphasized in all AIPS++ consortium meetings.

The refinement of end-to-end connected-element capabilities has been an overriding priority in the last cycle. For synthesis developers, this effort has focused tightly on demonstrating these capabilities for consortium telescopes, and improving their ease of use for scientific end users. Work in this area has included improvements in data filling, editing and visualization, calibration and imaging, and automated scripts to verify correctness for sample datasets. At NRAO and BIMA, we have worked with scientific test groups in this effort, as described further below, and the development work has been based closely on data reduction experiences with real datasets in a range of observing modes. We have encouraged all consortium members to follow the same strategy at their local sites for their own connected-element instruments. While NRAO plays a large role in the development of mainstream synthesis capabilities, including connected-element, and these capabilities are applicable to a broad range of instruments by design, the responsibility for testing and utility applications for specific instruments remains the responsibility of each AIPS++ consortium site separately.

We note that VLBI is not a separate layer in AIPS++, and is fully integrated in the design of the data format and calibration and imaging formalism. In this respect it differs from other packages, which, of necessity, added VLBI capabilities later in their evolution as VLBI emerged in the scientific community. As such, AIPS++ has already added VLBI capabilities in many areas in earlier routine infrastructure development. In this cycle we have added preliminary high-level VLBI applications for filling FITS-IDI VLBI data, and for fringe fitting, which exploit this existing infrastructure. This effort has been secondary to connected-element refinement, however.

While the synthesis group has been focused on connected-element work as outlined above, the single-dish development group has concentrated in the main on GBT commissioning support. This has included significant efforts in improving the AIPS++ filler for GBT data, demonstrating imaging capabilities for single-dish data, and in scripts used for specific commissioning tasks, such as pointing determination and DCR analysis. This effort has proceeded in close consultation with the commissioning scientists and engineers at the GBT, through regular contact and scheduled weekly meetings.

User testing

We have encouraged the user groups to submit defects to our central tracking system in all areas, including failures modes, usability improvements, and missing capabilities required for scientific completeness. We have assigned several developers to work directly with the user groups at the AOC, and to handle defects as expeditiously as possible for the VLA. Note that defect correction is an overall responsibility of all developers in the project however. The defect breakdown and response to resolution statistics are described elsewhere in this report. As noted in that section, the overall allocation of time for defect correction within the project as a whole has been increased from 20% to 30%, as part of this overall effort, and a large number of the defects filed by the user groups have been addressed in a timely manner. There has been significant direct interaction between AIPS++ developers and the testers as they reduce data at their workstations.

Part of the user group interaction at NRAO has been a tutorial series both at the AOC and in Charlottesville to describe and demonstrate the package in an interactive format, which encourages and allows significant open discussion with the user groups and other interested scientific staff. The Charlottesville tutorial was conducted over a week in December, 2000, with a schedule chosen to allow daily presentations as well as separate direct interactions with individual scientific staff using the system. The Charlottesville tutorial series will be expanded by additional regular visits of this type. We have endeavored to schedule regular tutorials at the AOC for the local user group. The Charlottesville user group has participated in these via the internal NRAO video-conferencing system. We have also added additional cookbook-level chapters to assist our user groups, and improved the documentation quality in other areas based on their experience.

At NCSA, the user group interaction has focused around providing documentation for scientists. Two outreach activities occurred: a series of short introductory talks, and a worked introduction to the use of multi-scale cleaning.

An important part of scientific integration is refining and improving high-level user interfaces. AIPS++ is a toolkit, which allows flexible and powerful programming and scripting by scientific end users, as specified in the original project requirements. It is also required to provide high-level applications similar to existing monolithic tasks in AIPS or MIRIAD. We provide this though straightforward amalgamation of existing toolkit capabilities in AIPS++. We have devoted a significant effort to improving these interfaces over the cycle. Interaction with scientists is vital in this regard, and the user groups have been especially important in such matters. Our in-house user groups have been using the toolkit capabilities in many areas, both to test the underlying capabilities and to allow scripting tests.

The current phase of scientific integration in AIPS++ implies a sharp increase in user support and involvement with user requests, as is appropriate and expected. We have emphasized this priority in our project planning, and made significant efforts to ensure that the project makes this transition in a smooth manner through the appropriate allocation of development resources. The regular release schedule is essential in this operational phase. Although interaction with user groups at the NRAO has been dominant, we have benefited from defects submitted from outside users, and we expect this to increase sharply as we go forward. Scientific users groups have also been established at other AIPS++ consortium sites, with a similar charter.

We have benefited greatly from the scientific staff that has been assigned to work in the various user groups, and this initiative will remain a priority as we go forward. It is a vital part of scientific integration.

User outreach

The AIPS++ Project management recognizes that there are adverse opinions about the package in parts of the community. Our fundamental belief is that user perceptions will be determined by the scientific capabilities actually delivered by the package, and demonstrated in scientific use, and for our part, we have tried to address community concerns in this manner. There are also mechanisms for community input to AIPS++, described separately in this report.

We recognize that there are other sociological factors that may influence community perceptions. AIPS++ had a difficult beginning and we expect these early perceptions to persist for some time. It also marked a transition in radio astronomy to large-scale software development, which requires a more comprehensive software engineering approach. Large software projects are costly and require a level of engineering practice commensurate to their scale. These have recently been updated in AIPS++ (see http://aips2.nrao.edu/docs/notes/237/237.html). More formal software engineering is dictated by the scope of a modern data reduction packages, as set by the scientific needs of current and future radio astronomy instruments. The new requirements and scale have been a sociological transition for users. These practices are more established in other institutions (e.g. NASA or ESO). The total investment in software remains a low fraction of overall expenditures in radio astronomy; as a community we still often underestimate the cost of software and the engineering implications of such development. We feel that this unease in the community is similarly best addressed through delivering demonstrated scientific capabilities to the end users in AIPS++.

We believe that there are ranges of factors that influence user adoption of AIPS++. These include: a) the scientific completeness of the package; b) ease of use, including documentation and user interfaces; c) correctness and robustness; and, d) new data reduction capabilities which allow new science. Our approach to user outreach, which is aimed exclusively at expanding the user community, is based on a balanced approach to these factors considered as a whole. We note that some have conflicting implications for development planning. The first three are our highest priority, and fall within the scientific integration initiative discussed above. They are best summarized as the ability to do regular data reduction easily, correctly and completely. Factor (d) is addressed partly through the new data reduction approaches we were earlier able to design into AIPS++ as a new project, as well as its unique existing scripting capabilities, but also through specific advanced development often in separately funded initiatives such as parallelization. This is discussed further below.

We have also tried to reach the scientific community through our user test groups, described above, as well as by continuing our program of visiting individual institutions for demonstrations and user interaction. We have also occupied booths at both AAS meetings held in the cycle, in continuation of past practice in this area. We also conducted a demonstration and interest group meeting at the ADASS meeting held in October 2000 in Boston. Efforts in all these areas will continue as an important priority.

Organizational outreach

As AIPS++ matures towards a complete package, more organizations have expressed interest in using it. Here we review activities this cycle in adoption of the package by various groups. We discuss only those groups for which some concrete agreement or understanding has been reached.

The AIPS++ Consortium partners are all making quite extensive use of AIPS++ but with quite different emphasis.

ATNF

Since 1997, the Parkes Multibeam Project has used software developed using AIPS++ in regular science observing and reduction. The scientific payoff has been substantial: see e.g.the HIPASS survey at:
http://www.atnf.csiro.au/research/multibeam/release/; and also:
http://www.atnf.csiro.au/research/multibeam/.

A prototype pipeline for synthesis processing of ATCA data has been constructed but is not in everyday use.

Local testing is concentrating on the Display Library image applications. The viewer tool is undergoing user testing and comparison with other display applications.

ASTRON

AIPS++ tools are used in the WSRT Real Time observing system, the Telescope Management System. The prime tools are the table system, the measures system, and a large number of specialized Glish-based tools.

A prototype pipeline is being developed to handle WSRT data calibration and imaging.

Local scientists and engineers make extensive use of the combination of Glish, the table system, and the measures system for general-purpose scientific and engineering computations, including for example work on SKA.

The correlator operated by the Joint Institute for VLBI in Europe at ASTRON uses AIPS++ as the backend, performing quality checks and data reformatting.

Relatively little scientific use of the synthesis code beyond the pipeline has yet occurred.

NCSA

The synthesis capabilities of AIPS++ are being tested for use on BIMA observations. A group of UIUC scientists are working with the local AIPS++ group on testing and writing documentation.

NRAO

Engineers and scientists have used AIPS++ since 1995 during the construction phase of the GBT.

Various tools in AIPS++ have been used during the GBT commissioning to determine e.g. pointing, focus curves, etc. In addition, the first images from the GBT have been made using AIPS++.

AIPS++ is being intensively tested in Socorro and Charlottesville for synthesis data reduction, and at Green Bank for single dish reduction.

AIPS++ will be used for the EVLA pipeline and offline data reduction (see below).

NRAO, NCSA, and Cornell/NAIC have proposed to NSF to construct a Common grid-Based Radio Archive (COBRA) that will provide archives and pipelines for radio astronomical observations.

An acceptance test for the use of AIPS++ as the reduction package for ALMA has been negotiated with the project (see below).

The Navy Prototype Optical Interferometer uses AIPS++ for data analysis including imaging (http://ad.usno.navy.mil/npoi/)

The Auto-Correlation Spectrometer and Imaging System (ACSIS) developed by HIA/DRAO, ROE, and DAO is built using AIPS++ class libraries and Glish to calibrate and image focal plane array data in real time. Glish is used to control the processing on a Beowulf cluster. This is a very exciting development that illustrates the construction of an entirely novel type of radio instrument. It is also of direct relevance to the planned use of focal plane arrays and bolometer arrays on the GBT and other single dishes. For more details, see the URL at:
http://www.drao.nrc.ca/science/jcmt_correlator/acsis.html.

HIA is using AIPS++ for simulations of SKA observations, and for the development of high dynamic range imaging algorithms needed for a time-variable primary beam.

None of these uses have constituted a significant load on the project. Indeed, we have benefited considerably from these interactions, receiving extra returns in the form of defect reports, user feedback, sub-system designs and software. The ACSIS collaboration has been especially fruitful, providing testing of Glish script clients, testing of the core library, and a design and accompanying body of code for the processing of focal plane array data. It has also lead to the adoption of AIPS++ for general JCMT single dish observing: John Lightfoot of ROE is now writing a MeasurementSet filler for the JCMT native (GSD) format.

ALMA: The possibility of use of AIPS++ for ALMA data reduction has been discussed for a number of years in various contexts. AIPS++ has of course been designed for the type of processing envisaged for ALMA. Now that the ALMA project is planning Phase II, this discussion has gained more urgency. Recently, Robert Lucas of the ALMA Science Software Requirements committee proposed an acceptance test for AIPS++ use in ALMA. The test is to determine whether AIPS+ can reduce data from the Plateau de Bure interferometer, for which it was not explicitly designed. An additional part of the test is to determine the learning curve for a developer new to the package. The AIPS++ project will therefore work with the PdB group to demonstrate processing of PdB data inside AIPS++, including amplitude calibration and phase transfer. A developer from IRAM will learn the details of the necessary code in AIPS++. There will be two phases: in the first, the AIPS++ group will develop and demonstrate processing on a few key data sets, and in the second, PdB staff will use AIPS++ to reduce other data sets. The test will run from June 2001 to April 2002. We welcome this objective and well-defined acceptance test. Parallel to this test, the ALMA Computing division is actively considering applying for membership of the ALMA consortium.

EVLA: The EVLA project started in May 2001. The project management plans that some parts of the processing will be contracted internally to the NRAO Data Management group, of which NRAO AIPS++ is a part. The pipeline and post-processing will definitely be implemented in AIPS++. The other parts are still under consideration.

NAIC: Our understanding is that AIPS++ will be made available for use by visiting observers at Arecibo. We have worked closely with ARO staff on the adaptation of AIPS++ for Arecibo observations.

NVO: NRAO and NCSA are partners in a large multi-organization NSF proposal to develop the software technology for the National Virtual Observatory. NCSA will work mainly on meta-data definitions, whereas NRAO will be responsible for developing connections between AIPS++ and the NVO.

SMA: The SMA is evaluating AIPS++ at present for use during array commissioning and in post-processing. They have developed Glish scripts for data display and exploration that have been used during commissioning activities so far. They have also agreed to take over FITS-IDI filler development as this format is written by the SMA correlator and therefore represents a shared interest. Reciprocal site visits have taken place between AIPS++ and the SMA in the past year to assist them in getting their development effort started.

The ALMA, COBRA, NVO, and SMA connections will all bring new developers into the AIPS++ project. Although NAIC has not devoted any staff to the development of the package, our interactions with the NAIC scientific staff have provided very useful feedback.

External user review committee

The AIPS++ User Group (AUG) held the first of its review meetings on Jan 29-31, 2001 hosted by NCSA in Champaign-Urbana. The AUG members were Dana Balser (NRAO), Helene Dickel (UIUC), Ed Fomalont (NRAO), Robert Lucas (IRAM), Karen O'Neil (NAIC), Tom Pauls (NRAL; Chair), Lister Staveley-Smith (ATNF), Huib van Langevelde (JIVE), Francois Viallefond (Obs. De Paris), and Tony Willis (DRAO). Their report can be found on the AIPS++ web page at http://aips2.nrao.edu/docs/notes/241/241.html. The AUG was constituted as astronomer-dominated, from a sample of people put forward by the consortium as known active users, or astronomers with direct experience in using the package. Their report will strongly influence our planning as we go forward.

A separate technical advisory committee (TAC) will be formed in late 2001 to review technical issues in AIPS++. This is expected to be a much smaller group, and to consist of developers or astronomers with strong software development experience in projects of a comparable size in radio astronomy or the broader astronomical community.

User Groups

NRAO AIPS++ Users Group (NAUG) has two parts: a synthesis group chaired by Frazer Owen and a single dish group chaired by Phil Jewell. The synthesis group was largely responsible for the user testing described above. We expect that these groups will merge eventually. A report on release 1.5 was prepared at the initiative of the NAUG (see http://aips2.nrao.edu/docs/notes/244/244.html)

These groups have been very effective at testing the package, both applications and documentation, and have provided many defect reports and other feedback to the project. We would like to see similar groups at the other consortium sites.

Developers meeting

The third AIPS++ Developers meeting was held in Green Bank during the week of April 22, 2001. This meeting is open to active consortium-based developers of AIPS++, and all but two were able to attend. The meeting started with a review of the AIPS++ User Group report, followed by reviews of user outreach activities at the different consortium sites. Then followed a review of each part of the package, given by the person responsible, including recommendations for work in the next development cycle. In addition, a day was set aside specifically for discussion of complex long-term issues that are difficult to address via email and phone meetings. The meeting concluded with the project management drawing up and discussing specific plans for the next development cycle.

Workload

In table below, we show the workload per consortium partner for the last three development cycles, including that planned for the next release (1.6). The work is assigned in the planning for each cycle in units of weeks. Defect fixing is set at a fixed fraction of the overall time per developer. There are 24 people nominally associated with the project, so on average we are getting a fraction about 13/24 of each developer. Note that the NCSA Alliance provides funding for two positions at NRAO. These are currently being re-filled at the moment.

Table 1: AIPS++ workload

	V1.4	V1.5	V1.6
ASTRON	30.5 (13%)	22.5 (10%)	26.5 (12%)
ATNF	29.5 (13%)	36.0 (16%)	36.0 (16%)
BIMA/NCSA	19.0 (8%)	34.0 (15%)	28.0 (13%)
NRAL	12.0 (5%)	6.0 (3%)	0.0 (0%)
NRAO	142 (61%)	125.5 (56%)	128 (59%)
Target FTE-weeks	233.0 (100%)	224.0 (100%)	218.5 (100%)
Defects	46.6 (+20%)	67.2 (+30%)	65.6 (+30%)
Target + Defects	279.6	291.2	284.1
Total FTEs	13.3	13.9	13.5

CURRENT STATUS

This section provides a more detailed listing of development in AIPS++ in this development cycle. These are sub-divided by project area.

Operations: AIPS++ is now in a regular release cycle of six-month duration, with public releases nominally scheduled for the spring and fall of each year. The fourth public release of AIPS++ (v1.5) was issued in June 2001. Each development cycle proceeds by the following phases: a) planning and the establishment of priorities; b) development of work breakdown structure and targets; c) development; d) release preparation; and e) distribution. Other activities in defect tracking, defect correction and user support occur throughout each development cycle. Target tracking is updated on a weekly basis. The most recent version of the AIPS++ software engineering practices can be found at: http://aips2.nrao.edu/docs/notes/237/237.html; related information on our code test and review GUIdelines can be found at: http://aips2.nrao.edu/docs/html/qag.html. In the cycle we have scheduled approximately 230 FTE-weeks of development work per six-month cycle, split in the approximate ratio of 65:35 % between NRAO and the other AIPS++ consortium partners.

In the cycle AIPS++ has supported the following architectures: a) Linux (Redhat 6.x; SuSE 6.x); b) SUN Solaris 2.5-2.7; c) SGI IRIX 6.5.x; d) HP-UX. We ship releases for Linux and Solaris at present, which together account for over 80% of the user community. We have shipped a large number of distributions ( $\sim$ 600-1000) for each release, in response to requests from a wide range of individuals and sites.

We plan a user survey of this group this year to determine use patterns. The current project compiler is gcc 2.95.3; the SGI C++ compiler is supported on a secondary basis. Local builds are supported at several remote sites outside the consortium; these sites have the discretion to update their systems more frequently than the regular release cycle. We also have the capability to update binary CDROM distributions via patches, which are easy for end users to apply. This has not been deployed in this cycle due to limited resources in the system area. Since we now have a dedicated system person, we expect this to utilize this fully for releases v1.6 and above.

Synthesis development: The priorities in synthesis development over the past cycle have included: a) refining and extending the scientific completeness of AIPS++, with particular emphasis on end-to-end reduction capabilities for connected-element arrays; b) increasing the use of AIPS++ synthesis capabilities in the scientific user community; c) improving the high-level user interface to synthesis applications; and, d) verifying the correctness, robustness and performance of synthesis data reduction in AIPS++. Infrastructure development has been scheduled in areas only in which it is required as part of higher-level scientific development.

Data fillers are an important part of scientific completeness, and all existing fillers (ATCA, WSRT, BIMA and the VLA) were enhanced during the cycle. Fillers for consortium instruments, such as WSRT, ATCA and BIMA, are maintained and developed by the associated AIPS++ consortium partners. The VLA filler was upgraded to support the new data weighting scheme recently adopted for the VLA, shadowing flagging, improved data selection, concatenation to existing data files and real-time operation. Improvements have also been made in the correctness of all coordinate frames in the output data set. The VLA filler has been used extensively at the AOC in the cycle and has undergone numerous improvements based on user group feedback. The BIMA filler has been revised extensively during this cycle, and has also been actively tested by a user group at UIUC/NCSA. A first version of VLBI filler was added to the system, supporting the FITS Interferometry Data Interchange format (FITS-IDI), in use at the VLBA, Mitaka and Penticton VLBI correlators, and the SMA. AIPS++ also has a range of data converters, to read and write AIPS++ datasets to UVFITS, FITS binary table formats, and data formats for other packages such as NEWSTAR. These converters were maintained and improved this year, but were not significantly revised.

A new application was added to allow the concatenation of separate uv-datasets in AIPS++. In a related area, improvements were made in data listing and data summary capabilities.

A significant effort in automated flagging was undertaken in the cycle, led by the AIPS++ group at ASTRON, as part of scheduled development. The current capabilities include optimized median filtering and clipping, as well as spectral editing modes to reject spectra with spikes or anomalous baselines.

Calibration capabilities were tested extensively on connected-element data in the cycle. The speed of the calibration solver was improved significantly by improving the underlying minimization technique. New solution modes allowing pre-averaging were added to support a greater range of polarization calibration methods. The facilities for display of calibration solutions were extended to meet user group requests. An initial VLBI fringe-fitter has also been added to the system. Figure 4 shows the display from a Glish-based prototype of the fringe fitter.

Calibration capabilities were also tested extensively on real data, particularly from the VLA, and compared against solutions from other data reduction packages such as AIPS. New test scripts were added to the system to verify the correctness of the calibration solver against both simulated test data, for which the exact answer is known beforehand, and against datasets calibrated in other packages.

Work in imaging and deconvolution was continued in the cycle, although core capabilities in these areas are now mature in the system. Improved features were added for combining single-dish and interferometer data, and for graphical displays of deconvolution progress. The speed of mosaic gridding has been improved. Imaging tests with real data were continued and expanded this cycle, particularly for VLA data, in keeping with the overall approach outlined above. The wide-field imaging capabilities were thoroughly tested against simulated data, as well as against VLA low-frequency data.

Initial pixon deconvolution capabilities were added to AIPS++, as a research project in science time. Further details can be found at:
http://aips2.nrao.edu/docs/notes/242/242.html. At present these are available for single-dish imaging only. Pixons are proprietary and patented technology, but hold significant promise for state-of-the-art deconvolution. AIPS++ will support an interface to these external IDL libraries, which users will be able to obtain from the private corporation (Pixons LLC).

Source component model capabilities have been expanded to include a more complete error representation. The simulator work has also continued, primarily in the area of supporting pointing errors, and in improving the performance of the simulator tool.

A key focus has been on improving the high-level synthesis applications in AIPS++ and their user interfaces. All AIPS++ applications can be run from the command line or from the automated GUI system, known as the toolmanager. AIPS++ allows different levels of user access to the system, as required in the original specifications for the system. These include a scripting or command-line layer, to allow scientific end-users to do custom reduction or have fine control over their regular data reduction strategies. At this level AIPS++ is a powerful toolkit of radio astronomical data reduction capabilities. A high-level interface is also provided which packages the toolkit capabilities as more integrated applications. We are currently providing fully integrated applications in this area, such as map, which combines lower-level imaging and calibration capabilities, as well as so-called wizards, which provide GUIded reduction in a high-level graphical interface. The wizards offer user choices at decision points in the data reduction path. This entire effort is a vital part of scientific integration, described elsewhere in this report, and has involved extensive interactions with our user groups as well as the AIPS++ user review committee. Scripting is essential for our work in pipeline reduction, in which a heuristic procedure takes decisions about which path to follow in the reduction sequence automatically, as well as in empowering our scientific users to do novel data exploration and experimentation with algorithms. However, high-level integrated applications are essential for an important part of our user base, and we intend to continue this as a priority as part of our scientific integration effort.

As part of the user test efforts described elsewhere in this report, a significant focus has been proving the end-to-end reduction capabilities in AIPS++. For NRAO developers, this effort has been tightly focused on the VLA, and has been based on proving end-to-end reduction for a set of datasets in different observing modes. All associated data files, processing scripts and resulting images are then checked into the system. We also endeavor to use them as examples in our cookbook-level documentation to GUIde scientific end-users. An example of an end-to-end reduction for the VLA is shown in Figure 5, for a VLA mosaic observation of the Orion Nebula (D. Shepherd). The associated processing script can be found in the AIPS++ code distribution in trial/apps/synthesistester/vlaendtoend.g. These data were reduced from VLA archive tape through to final image in AIPS++.

Single dish: Early 2001 saw the acceptance of the GBT from the contractor, and the beginning of commissioning in earnest at Green Bank. The NRAO AIPS++ single-dish group (Garwood, McMullin and Braatz) has supported the commissioning as a priority, but has also continued to develop and expand mainstream single-dish capabilities, such as those available in the dish tool.

AIPS++ is an integral part of the GBT. Commissioning support proceeded primarily in two areas: a) the continued development and maintenance of a suite of scripts for data analysis specific to commissioning, such as DCR testing, holography back-end testing and pointing/focus measurement and correction; and, b) supporting the data interface between AIPS++ and the monitor and control system. The latter work has involved a substantial effort in upgrading the GBT data filler to support FITS files now available for a broader range of back-end devices.

In mainstream single-dish development, the dish tool has been more closely integrated into AIPS++ and the toolmanager GUI interface. This has also improved the scientific scripting capabilities for single-dish reduction. The command-line interface was significantly enhanced to improve ease of use, in both interactive and scripting environments. An interface has recently been developed in collaboration with several staff scientists at Green Bank to integrate dish capabilities at a higher level. This effort is analogous to the high-level integration work, which has been undertaken in synthesis, as described above.

There have been major advances in demonstrating the single-dish imaging capabilities, which already existed in the imager tool. These have been refined and used directly to image single-dish mapping data taken during commissioning, as shown in Figure 6. Work has also continued in unifying single-dish and interferometric calibration. The msplot uv-visualization tool was also augmented to support single-dish data. Test scripts and sample single-dish data files have been checked-in to the system in the same manner as has been done in synthesis, as described above.

Image analysis: AIPS++ has extensive image analysis capabilities that are packaged in the image tool. These capabilities are relatively mature, and development in this area in the cycle has focused primarily on missing high-level scientific features, or improving the ease of use of existing capabilities for scientific end users.

A new polarization image analysis tool has been developed, which allows extensive operations on full polarization images, including all corrections for statistical bias. An application has been added for rotation measure estimation using Fourier techniques. Spectral fitting capabilities have also been added, and the error handling for image-plane component fitting has been improved. A separate tool has been provided to allow easier manipulation of coordinate systems by end users. General infrastructure improvements have included improvements in mask and region handling, convolution, and image algebra capabilities. A basic point source finder has been added, to generate initial estimates for more detailed component fitting.

Visualization: Visualization efforts have focused on developing high-level applications for scientific end users. The AIPS++ Display Library infrastructure, which has been developed over the past several years, has been consolidated, but this has been a secondary priority over applications development in this area.

The ease of user of the viewer display tool has been improved in a number of areas. This work has been based on extensive user interaction and feedback, and specific user interviews to contrast the viewer with comparable display tools in other systems. Image visualization work is contributed primarily by ATNF. This work has included enhancements in sky catalog overlays on images, and most importantly, the development of an integrated 3-D slicer application (see Figure 7). This application tests all aspects of the AIPS++ visualization infrastructure.

Parallelization and high-performance computing: The NRAO AIPS++ project is a participant in the NCSA Alliance; a collection of partners developing high-performance computing solutions for a range of scientific disciplines. Parallelization and high-performance computing in AIPS++ is separately funded through this NSF grant.

The objective of this initiative is to develop solutions within AIPS++ for the most computationally demanding problems in radio astronomy, and make them available in the community through our regular AIPS++ release process. We have developed a specific parallelization interface to allow parallel algorithms to be integrated seamlessly into AIPS++. In the cycle, we have developed a parallelized implementation of wide-field (3-D) imaging at NRAO, which allows key steps in this algorithm to be divided by imaging facet and distributed amongst multiple parallel processors. We have demonstrated this capability on a 74 MHz VLA observation of the Coma cluster by R. Perley and collaborators. A 225-facet image was produced for this dataset using 32 processors on a supercomputer system at the NCSA in approximately 4% of the time it would have taken on a single processor. This image is shown in Figure 9. The parallel wide-field capability holds great promise for dealing with the most demanding VLA 3-D imaging cases.

Documentation: AIPS++ has an extensive documentation system containing information of several different types:

Cookbook-level documentation, indexed by scientific purpose (Getting Results in AIPS++);

A detailed user reference for the scripting or toolkit capabilities (User Reference Manual);

Extensive programmer documentation for the code library. Documentation is a major element in our code review and acceptance process. In addition, AIPS++ is an open project and all our internal documents, reviews and reports are placed on our web site (http://aips2.nrao.edu). We also edit and distribute a user newsletter approximately every quarter.

We have added to our documentation in the cycle, particularly in the area of the cookbook-level GUIde for scientific users, and we intend to continue this work. Not unlike other software projects, we have had mixed success in getting users to generate documentation. We are currently proceeding with an approach in which users edit and revise initial draft documentation produced by the AIPS++ project. Scientific users are not unanimous in their views on documentation, so we try to average over a large ensemble of comments. Quite naturally, users tend to view documentation through the prism of their own scientific interests or work styles and there is a great diversity of opinion on what constitutes good documentation. We have tried to follow industry models in documentation and to be responsive to requests from scientific end users.

User interface: As discussed above, AIPS++ has several layers of user access by design, and in order to meet the design specifications. Our GUI interface has been improved in a number of areas over the cycle. We have added new capabilities for high-level applications and have tried to improve the intuitiveness of the interface wherever possible. This development is strongly influenced by user comments and feedback, and has proceeded iteratively in this manner. A focus of recent work has included improving GUI speed. Completely intuitive GUI's are extremely expensive to develop, and scientific users are increasingly exposed to commercial packages of this quality. We intend to pursue user interfaces in a similar manner to documentation, but soliciting extensive feedback from the user base.

General infrastructure: As noted above, infrastructure work has been carefully targeted in AIPS++ to ensure that the focus remains on scientific integration. Some level of infrastructure development work is always required however, and appropriate as part of coordinated development efforts. In this area, AIPS++ has added support for large data files (> 2.1 GB), improved our compliance with the C++ standard library, and enhanced our least-square fitting and coordinate conversion infrastructure. In addition, the memory handling in Glish has been made more efficient.

Personnel: Mark Holdaway moved to the ALMA project in February 2001, and has been replaced by Kumar Golap, who previously worked in the parallelization project. Sanjay Bhatnagar of NCRA will join the AIPS++ project in Socorro in September 2001 to replace Kumar Golap. Toney Minter transferred to the GB main budget from the AIPS++ visualization grant in November 2000. Vacancies in visualization and parallelization are currently being advertised. Wes Young has moved from the parallelization grant to a new position for AIPS++ system support, which has become an increasing need in the project in this operational phase.

TESTING AND DEFECT HANDLING

For objective information on the health of the AIPS++ package, we have analyzed the defect submitted within the last two years.

On average, the AIPS++ group produces about 10,000 lines of code per person per year, which is an excellent level of productivity. The growth of the package in lines of code is shown in Figure 10. Note the steady growth since mid 1995, with only the hint, at most, of saturation.

To fix a defect, you first have to find it. We have a number of ways to find defects:

C++ unit testing: to pass review, C++ classes must have test code that exercises 80% of the code in that class. We run these tests regularly and use the results as a basic measure of health of the package.

Integration testing: Glish test scripts are present for most of the substantial Glish-based capabilities. These test scripts are run daily and the results summarized in a mail message to all AIPS++ developers.

Pre-release testing: Prior to release, we run through a set of standard interactive tests of the package.

Comparison with other packages: as appropriate, some tools are compared in performance and results to the corresponding applications in other packages such as AIPS and MIRIAD. For example, we have conducted detailed comparison of the calibration solutions found in AIPS++ and AIPS (showing excellent agreement).

Directed user testing: We ask scientists to test certain tools, and to comment on documentation, interfaces, scientific completeness, etc.

Use testing: Scientists use the package for reduction and analysis, and any defects are noted.

The most substantial change over the last year has been the large increase in user testing. The AOC based group of testers has contributed many defect reports. A bulge in the upper envelope around August and September 2000 in Figure 9 is due to the advent of the AOC test group.

Defect handling is a large part of the day-to-day activity of the AIPS++ group. This last year we increased the fraction of time allocated to defect fixing from 20% to 30%. We classify defects in the following way:

Users assign these severities on submitting a bug. Our goal is to address all Severity 1, 2, and 3 defects immediately, and the lower Severity defects on a background basis.

Since March 1999, all defects are handled via a commercial tracking system called ClearDDTS (replacing the public domain program gnats). This provides voluminous feedback to all relevant parties on actions involving a defect report. It provides an excellent mechanism to track many aspects of the defect fixing process. The AIPS++ Project management is responsible for the initial assignment of defect reports and so we see all defects filed. We follow the statistics carefully from week to week. A useful summary graph is shown in Figure 10. This shows the disposition of defects as a function of time for all severity levels. The status means:

Non-Project users verify relatively few defects, so the distinction between Resolved and Verified is not particularly significant. To demonstrate that we are successful in keeping the most serious defects under control, in Figure 11, we show the same plot for Severity 1 and 2 defects.

A detailed analysis of about 10% of the recent defects shows that about 55% of all defects are true coding errors, 30% are caused by requirements incompleteness or drift, and 15% are can be described as being due to documentation inadequacies. These numbers are in rough agreement with those typically quoted in the literature.

We have also analyzed the defects per AIPS++ tool. Here we count the number of defects per tool as a function of time (or actually defect identifier which is roughly proportional to time). In Figure 12, we show the defect submissions as a function of tool, and in Figure 13, we show the cumulative defect count. Some conclusions:

Defect rates for the simpler tools (i.e. catalog, help, bug, logger) obey a sharp-rise, slow-fall curve (often described as Rayleigh in the literature), and have dropped close to zero now.

The ten most ``defect''ive tools (i.e. imager, msplot, toolmanager, Glish, viewer, image, ms, calibrater, table, and GUIentry) account for roughly 50% of the defects. All of these tools have been widely used in the last two years, and have also undergone continuing development during that time to complete the scientific capabilities of the package.

imager is one of our flagship applications. It exercises nearly the entire core AIPS++ library, and is very highly used, being the prime tool for synthesis and single dish imaging. New functionality has been continuously added over the last two years. It also has an extensive Glish-based regression test suite that is part of the daily tests, so we find changes in behavior quite quickly.

msplot, toolmanager and viewer are some of our largest Glish-based applications, being about 5000 lines. The apparent defect rate is about 10 times higher than it should be for a Glish function of this size (compare to tablebrowser which is the same size). We believe that the defect rate is high principally because of the extensive development that has gone on.

The completeness of submission of defects (i.e. the fraction of errors that result in a filed defect report) is unknown, but the defect submission rates are known to vary tremendously from person to person, and from site to site. About 60% of our defect reports have been submitted by 5 of the core developers. Our most committed scientific tester (Debra Shepherd) has submitted about 4% of the total defect reports. We believe that unfortunately a remarkably large fraction of the user community do not expect defects in astronomical software to be fixed, especially in a large package like AIPS++. This presents a substantial obstacle to debugging AIPS++. In public presentations we continually reinforce the point that we like and expect to receive defect reports.

In summary, we see that the package continues to mature. We can expect that a substantial number of defects still remain in the package, of which some will probably remain for a number of years. Those parts of the package that have seen few changes in the last two years are quite stable, but those areas under continuing development will require more debugging.

RELEASE 1.6

Planning for the next development cycle, version 1.6 to be released October 2001, is now complete. Feedback from the various user groups has been very influential in determining the priorities.

Scientific completeness Demonstrated end-to-end reduction capabilities for targeted instruments in all scientifically important observing and reduction modes.

Usability by the astronomical community Quality of the user interface, applications presentation and user documentation.

Robustness, correctness and accuracy Prove the correctness and robustness of existing capabilities and ensure comparable performance to other disk-based packages.

Continued deployment to an expanded user base Continue to increase the scientific user base for AIPS++.

Performance Continue to measure and improve the performance of AIPS++ applications.

Detailed information may be found at:
http://aips2.nrao.edu/docs/notes/246/246.html.

SUMMARY

The AIPS++ project has released its fourth release. All four major releases have been issued within 1-2 months of the scheduled date.

The infrastructure of the package is well established and has been used in a variety of demanding applications.

The operating practices and ethos of the project are set, and the project team works well together. New members have been able to join and typically become productive within 3-6 months of training.

Although the package is getting closer to meeting the original specifications there remains work to be done. In the package itself, the emphasis must remain on scientific completeness. This means completing connected element and then VLB interferometry support. The former is getting closer for NRAO and NCSA, but serious deployment and testing is yet to occur at ASTRON or ATCA.

Knowledgeable, effective and engaged user groups have been set up at the project and site level. These have been helpful in (a) bringing users to the package, (b) acquiring feedback, and (c) testing the existing capabilities. Similar groups should be set up at all consortium sites.

The main user demands are for improved documentation, more intuitive interfaces, greater robustness, and rectification of some areas of poor performance. All of these areas are being addressed in the developments for release 1.6.

Other organizations have expressed interest in joining the consortium. It seems likely that AIPS++ may be adopted as the reduction package for ALMA. Outreach to other organizations such as LOFAR will become easier as the package spreads.

APPENDIX A: ASTRON REPORT

The upgraded WSRT (and the JIVE correlator operations) continue to rely heavily on AIPS++. The on-line system (TMS) uses AIPS++ Tables, Measures and Glish, and MS2uvfits for data export. Data inspection is done with (at least) three different uv-data visualization tools, which have been written locally with Glish and AIPS++ modules. The next step is a quality-monitoring pipeline, which might evolve into a calibration pipeline. The latter is seen as an important way to demonstrate what AIPS++ modules are capable of, which should help to convince local astronomers to start doing their own data reduction with AIPS++ rather than with one of the older packages.

Ger van Diepen and Arthur Coolen have contributed mostly to the AIPS++ infrastructure, and between them they keep the local site going. Ger continues maintaining and (modestly) upgrading the various data converters. Oleg Smirnov has spent most of his time implementing synthesis applications, notably the flagger. Tom Oosterloo uses AIPS++ (modules) for WSRT off-line processing, and Jan Noordam does the same for LOFAR/SKA design and simulation. Between them they promote and assist local astronomers in the use of AIPS++.

It means that on Solaris and IRIX files > 2 Gb can be used. On Linux it will also be possible if the kernel supports large files.

The Complex class has been changed to use the complex class supplied with the new C++ standard. It makes the AIPS++ code more reusable.

A class has been developed for Neil Killeen to make it possible to access an image in a FITS file (in fact, in any file) directly as an AIPS++ image. It makes it possible to view a FITS image in AIPS++ without the need to convert it first.

Added the ability to remove storage managers

Local AIPS++ work at ASTRON: ms2uvfits has been improved on request of Robert Braun. Some assistance has been given to various people.

Oleg's efforts this development cycle were entirely dedicated to the AIPS++ flagger. He has completed a new flagger tool for this release of AIPS++ (see autoflag.g, and AIPS++ User Reference Manual, Synthesis package). The tool is relatively feature-complete, with only a few minor improvements expected in the near future. Autoflag has a number of critical advantages over previously available flagging facilities. It is implemented entirely in C++, and designed to scale to large measurement sets, and to limit memory usage in order to avoid degrading performance. As a result, speed improves 2x-5x over small data sets, and by orders of magnitude over large data sets. Autoflag supports the new structures of MS2 (spectral window and data description IDs) in a transparent way, which had made the older tools unsuitable for some consortium sites. Autoflag has a modular architecture, so new flagging algorithms can be implemented and added in a consistent way. Within this cycle, Oleg has implemented the following flagging algorithms: median filters (in time and frequency), spectral rejection, UV binning, and direct selection (a.k.a. ``command-based'' flagging). This allows autoflag to supersede, in terms of features, the old flagger.g script, and the old MSFlagger which was available via the ms tool.

APPENDIX B: ATNF REPORT

We migrated to the new GNU compiler this cycle. Our systems have been stable except for the usual running around after the documentation system.

Changed all UVW- measures conversions to make them fully compatible with observed directions and VLA definitions, incorporating solar gravitational bending a.o. in proper direction

Made all direction related conversions based on the same set of basic routines (like baseline, UVW, direction calculations).

Made all glish measures and related functions work with multi-valued quantities to improve the efficiency of transferring data between Glish and C++ code. In addition arrays of quantities became possible as well.

Changed polyfitter to enable solutions when supporting vectors are all but parallel

Wrote multi-Gaussian and other shapes fitting class for profile like data, including the use of masks

Rewrote the higher level fitting classes to cater fro problems with the existing Functional classes (which will be reviewed next cycle)

Made the administrative template support correct for the new Complex classes based on the standard complex<> (reident, used, unused, duplicates)

Made the template handling mkinst work with the standard include files (which have no .h extension)

Removed many unused String functionality, to enable easier integration with standard string class

Derived the standard AIPS++ String class from the ANSI standard string class, incorporating the AIPS++ extensions in a transparent way (including the use of regular expressions), and installed it in system

Added proxy includes classes for many system include files to cater for SGI compilation library omissions.

Fixed some problems with the Linux I/O library (which made alphabetic code read as numbers)

Many problems with the use of const removing casts in older code found and either isolated or corrected.

Added documentation and examples for the use of the set of `UP'-scripts that handle the administration/test of overall system changes

Coordinates: fix obscure FITS handling problems, redid Stokes Coordinate handling, implemented relative coordinates, added pointing center to ObsInfo, add concept of preferred world axis units and preferred doppler type (SpectralCoordinate) to be used in formatting and elsewhere in DisplayLibrary, new constructors in SpectralCoordinate taking a list of velocities (for Dish), more functions in SpectralCoordinate for converting to and from velocity, add functionality to support world ranges used in mixed (world/pixel) coordinate conversions, world coordinate conversion now offers user many formats, optimized speed of bulk coordinate conversions.

Images: more modular structuring of image viewer support code, added spatial averaging to image.{getchunk,getregion}. Improved image.view profiler in many ways. Created FITSImage class to give native access to FITS images and bound to Image tool, support relative coordinates in region classes, added functions image.addnoise, image.maxfit, image.adddegaxes

ProfileFitter: created new tool to interactively fit profiles from images and reworked code from Imagefitter so that they both can use it.

Polarimetry: added multiple RM components to test image, fixed Stokes labeling, improved plotting in RM method,

DisplayLibrary: created display data to display vector maps, added support for on-the-fly mask to Lattice display datas, supported complex data in Lattice contour display data, improved position tracking to have its own rollup in the adjust GUI and to support absolute or relative positions, choice of velocity or frequency units, eradicated many compiler warnings from DL code, reworked the Attribute classes, added to axis labelling relative or absolute, world or pixel, selection of units.

Defect work: huge amounts mainly in the display library regarding axis coordinate labelling for mixed (world/pixel) cases.

Presentations at AstroFests, ComputerFests, Steering Committee, AIPS++ User Group

Various reports (CSIRO board, ATNF annual report, AIPS++ cycle reports, ATUC, future developments)

Distributed CDs

Viewer support for channel maps. Substitution of WorldCanvasHolder with MultiWCHolder. This includes infrastructure changes and redesign of parts of the viewer.

Started improving SkyCatalog overlay. Added on-the-fly coordinate reference code transformations.

Mark's time (15%) was spent on defect fixing, most of it related to the MeasurementSet DO.

APPENDIX C: BIMA REPORT

These efforts included contributions at NCSA from Ray Plante (local manager, 50%), Dave Mehringer (site installation manager, 50%), Harold Ravlin (50%), Anuj Sarma (50%), and Paulo Cortes. In addition, Peter Teuben of the University of Maryland also contributed part time to development and testing.

A special effort was made to build up greater local expertise in the AIPS++ system. Plante, Mehringer, and Teuben traveled to Socorro for ``high-bandwidth'' discussions with experts at NRAO. The establishment of a local users group has also helped build our experience using the AIPS++ package.

Efforts continue to enable end-to-end calibration and imaging of BIMA data. The bulk of the effort this cycle has been in filler development by Plante and Teuben. The previously existing filler, bimafiller, developed and maintained by Teuben, is now designated as the development filler: it is tool to develop and test new capabilities needed in the short term, such as MS-2 compliance and system temperature-based error weighting. It also includes a workaround to the incomplete support for heterogeneous spectral windows in the AIPS++ core library. This filler was used this cycle to support imaging of the BIMA SONG galaxy survey. Plante developed the production filler, mirfiller, based on the experimental development by Teuben. This filler includes full MS-2 compliance and support for heterogeneous spectral windows, multiple sources, and multiple arrays.

Plante and Mehringer outlined the basic calibration strategy for BIMA data, and Mehringer began implementing high-level tools that apply the strategy using the standard calibration infrastructure. Added to the system in this cycle was the bimams tool, which understands the BIMA MS conventions used by the BIMA calibration strategy. This strategy uses wideband averages of data from multiple windows from the same sideband, and bimams supports the recalculation of these averages.

The bulk of the effort this cycle has been spent on porting AIPS++ to experimental 32-bit and 64-bit clusters at NCSA. The 32-bit builds were complete with out change to the basic AIPS++ system or code, and initial timing tests were run using the parallel spectral line imaging and wide-field imaging algorithms. The 64-bit port was incomplete due to the use of immature compilers. Some changes to code were necessary and documented; however the basic 64-bit support infrastructure (see Note 227) appeared to be sufficient to support the new 64-bit clusters at NCSA. Primarily Mehringer and Cortes carried out porting work, with assistance from Plante and Wes Young of NRAO.

Sarma performed general timing tests on the NCSA Origin 2000 and cluster platforms using the wide field imaging algorithm.

Ravlin began his work exploring OpenGL-based visualization with Aipsview. A current motivator for this work is to support large-pixel, multipanel displays at NCSA for displaying large images. Once he gained the basic experience using OpenGL to display astronomical images, he began transferring that expertise to the AIPS++ Display Library with the creation of an OpenGL DisplayCanvas class. This work is on-going.

To help prepare BIMA astronomers for processing data with AIPS++, a working group was established at UIUC called the LAI AIPS++ Users Group and Help (LAUGH). It included of AIPS++ developers (Plante, Mehringer, and Sarma) along with two astronomers from the local BIMA group, Lanie Dickel (research associate) and Dave Fong (graduate student). The main activity of this group has been to write the BIMA calibration cookbook for AIPS++ (i.e. a Getting Results chapter). A basic draft is nearly complete.

This committee also sponsored two outreach activities. The first was a series of short tutorials given by Plante at the biweekly UIUC BIMA meetings to introduce users to basic AIPS++ concepts; see
http://monet.astro.uiuc.edu/AIPS++/gentle. The second was a hands-on workshop. The topic of the workshop was multi-scale cleaning; however, this topic was used as a forum for exercising basic skills needed for running AIPS++. This workshop is described in AIPS++ Note 245.

Systems

The computer systems at NCSA include Linux, Solaris, and SGI IRIX machines. AIPS++ is primarily run on Solaris platforms for both development and end-user use. An SGI 4-processor Linux box is used for the BIMA Image Pipeline, now under development. The builds on SGI IRIX platforms continue to be stable, including the NCSA Origin system. AIPS++ was also built on 32-bit 128-node Linux cluster at NCSA for testing purposes; a build on a 64-bit Linux cluster was attempted but not completed.

APPENDIX D: NRAO REPORT

Jim Braatz

Jim Braatz' primary responsibilities are to manage and support the Green Bank installation of AIPS++, support GBT operations, develop software for analysis of GBT commissioning and continuum observations, and assist with single-dish spectroscopic applications. Specifically, Jim worked on the following tasks during this cycle:

Commissioning of the GBT using the prime focus receivers and the S-band Gregorian receiver began this cycle.

Jim participated heavily in the commissioning work by providing software for analysis and participating in the observing and planning.

Development of automated routines (GOpoint) to handle pointing updates were written and incorporated into the online GBT control system. Early results show that the software is performing well and pointing of better than 6 arcsec was being achieved with prime focus

Development of the DCR tool continues. As this is the primary software tool for analysis of GBT commissioning data, its development this cycle has been mostly in response to the needs of that process. The CLI interface was made to operate seamlessly with the (more limited) GUI, allowing for easy scripting. Functions for analyzing, recording, and displaying results of pointing observations were stressed.

Functions to handle focusing scans were written as well. A number of specialized analysis routines were incorporated to simplify common procedures.

Soon after commissioning began, the need to optimize the DCR tool and GOpoint software became evident. To streamline the procedures, it was necessary to separate DISH from the DCR tool and optimize the data mining from the measurement set. The tool now runs faster by a factor of a few.

Assisted in early processing and analysis of Spectral Processor commissioning data.

Began managing the weekly AIPS++/GBT meetings, at which the relevant staff in GB, CV, and the AOC discusses the requirements and priorities of AIPS++ single dish software development, particularly as it relates to GBT commissioning and early science.

The AIPS++ system continues to be widely used in Green Bank, especially in support of GBT operations and analysis but also in general science applications. Jim has prepared scripts, personal instruction, and given other assistance to engineering and science staff. While the documentation in several areas remains under development, this personal instruction requires a considerable effort.

Jim continues in his responsibility for correcting defects and making enhancements to the pgplotter routines. Several improvements relating to the interactive editing feature and line masking capabilities were checked in this cycle.

Reviewed and tested DISH and proposed a new CLI to aid in the processing of online data and to simplify scripting. This has been adopted as the ssa package (written by Joe McMullin).

Bob Garwood

Bob Garwood's primary responsibility is to oversee and contribute towards the single dish work in AIPS++. This work remains focused on the DISH environment and the support of the GBT. His contribution in support of the GBT is primarily through the GBT fillers that convert the GBT FITS data files to an AIPS++ MeasurementSet. His DISH work involves the maintenance and enhancement of the data access tool (sditerator), the data averaging too (sdaverager) and the data conversion tools (sdfitstoms and ms.tosdfits). In addition to these duties, he is responsible for the maintenance and enhancement of the FITS classes. He is also the chief code cop. Over the past development cycle (November 2000 through May 2001) he has done the following:

Continued work to develop the GBT filler. The filler was used for the first time during commissioning of the GBT. Several defects were discovered and fixed. The filler was modified to handle several new FITS files produced by the GBT. Work was nearly completed on reading the IF and LO FITS files. The handling of antenna telescope pointing information was changed to use a new FITS file now produced by the GBT. The filler tool interface was modified so that it now runs asynchronously so that the Glish session invoking it continues to be usable while the filler is running.

The speed of the sditerator tool was greatly improved when the underlying data is in a MeasurementSet. Speeds for browsing and summarizing such an sditerator went from several minutes to a few seconds.

The SDFITS conversion tool was fixed to decrease the amount of data bloat. Several addition defects involving this tool were fixed. Participated in many discussions regarding the tasks required of the gbtmsfiller when filling data from the GBT spectrometer.

Tim Cornwell

Tim Cornwell's time devoted to AIPS++ has dropped to about 50% as a result of his new responsibilities as Associate Director for Data Management at NRAO. Many of those new duties affect AIPS++ either directly or indirectly. Many of Tim's maintenance duties have been shifted to others. He remains responsible for assigning and tracking defect reports, which keeps him in close connection with all activities within the project.

Single dish imaging using the imager tool as applied to the GBT commissioning observations. Tim visited GB three times during this cycle to work on this and the visualization of single dish data using the msplot tool.

Tim and Amos Yahil (SUNY) worked on the incorporation of Pixon image deconvolution into AIPS++. The implementation calls IDL from inside AIPS++ routines. Both speed and performance are excellent. At the moment, only single dish data and image plane convolution can be thus treated. Tim continues to work on synthesis data reduction using Pixons.

In addition Tim was deeply involved in many of the management activities within the Project.

Kumar Golap

In cycle 1.5 the major piece of work has been the completion of the wide-field parallelization problem. The process of parallelization was slowed a bit with finding associated problems with the file locking mechanism. One of the major time consumers was that the user-locking mechanism was getting lost and i had to understand what was really happening. But the reasonable speed-ups that we have got show that the model of parallelization we chose is a practical one (at least on shared memory machines). The model we chose is to have the different workers CPU's access the same data sets independently instead of the master CPU passing the data around.

In the last few weeks of the cycle I have shifted into my new position and have been more involved in fixing bugs and user support. One of the major the satisfaction was the solving of the `position bug' that I found when I was checking the working of the wide-field imaging process long ago. On the science side I have been processing some P-band data and I think it should be reasonably easy (so I think right now!) to solve the VLA beam asymmetry.

Athol Kemball

During this development cycle I have fulfilled several coordination roles as Deputy Project Manager. These have included the coordination of development in synthesis, parallelization and NRAO visualization efforts. I have also contributed to the coordination of development planning activities for v1.5 and v1.6, and the drafting of the development plan for each of these cycles. I have continued as technical leader for AIPS++ during this period, and have contributed to systems issues as part of that role. This has included direct involvement in release preparation and associated patch application and testing. I have maintained a direct involvement in code generation, defect correction and user support, particularly in the area of MS infrastructure, calibration capabilities and parallelization infrastructure. Activities in these areas have included enhanced polarization calibration facilities, the creation of higher-level synthesis applications and the unification of existing MS access code. I have been involved in a number of public outreach activities for AIPS++, including working directly with user groups, and have prepared and given presentations on AIPS++.

I have been working on DisplayData classes to be used in things like msplot for display and editing of MS data. To that end, I have been studying the workings and interface of many parts of the Display and general aips++ library: canvases, DDs, glish interface, GUI elements, CoordSystems, Images, MSs, Tables, access/iteration classes, and basic data structures (probably over 150 major classes in all).

I am also trying to learn the science as fast as I can (it is really essential to understanding the form of the problem). I have a lot of catching up to do, but feel I am making good progress, and am enjoying the challenge.

I am starting with the Raster (TVFLG-like) DD, and have some basic user interface mapped out and working through to the viewer. The basic structure of the DD is also clear: it will retrieve data from a single MS using VisIter and related access classes, and present it to the user as a five-axis hypercube (Time, Baseline, Sp. Window, channel, and polarization). Usually time, along with baseline or channel, will be the current 2D slice viewed, other axes being controlled through sliders/animators. In principle, any of the axes may be set as display axes; sky Images currently have this flexibility in the viewer. I need to share as much of that structure as possible.

Very recently I have created a test program for iterating MSs, and am gathering useful information on access efficiencies in different modes, as well as learning how to operate the iterators. I am currently working on extracting ranges, creating Coordinates and an Image, to test how quickly such a lattice can be built and accessed. These should comprise the major data structures for display in the DD; it remains to be seen how much of the data (between the Matrix currently displayed and the entire MS selection) should be cached in an Image.

Other details I'm thinking about: time and channel averaging, editing interface elements, data structures to go back from GUI editing events to the flag data, combining, ordering and removing degenerate axes in the ui, coordinate labeling in various cases and forms. Fortunately, I believe a simple framework that actually displays some uv data can be created fairly soon, and will provide concrete terms for addressing these issues afterward. I am staying in touch with Ralph to integrate the DDs into msplot, and to make use of his experience.

Ralph Marson

For the v1.5 development cycle, Ralph Marson's contribution focused largely on three areas: improvements to the component handling structure, improvements to the VLA filler & improvements to msplot. Aside from this time was also spent on maintenance issues like the linux build or helping users & programmers. During this cycle Ralph moved from the Australia, where he had been working the ATNF group, back to Socorro. The details are:

Improved the VLA filler so that it writes weights that are dependent on the observing bandwidth and system temperature (as well as the integration time).

Improved the VLA filler so that it automatically sets the tape drive to variable block size

Joe McMullin

Joseph McMullin's main responsibilities are work on single dish applications within AIPS++ and help with GBT commissioning duties. He is also currently serving as Chief Tester and also handles the aips2-request list, logging and shipping AIPS++ release CDs.

Updated DISH to deal with a revision in the fundamental data structure (SDRecord).

Wrote a DISH test script that runs through a sequence of reduction operations on demo data to verify the operational state of the package.

Worked (with Athol Kemball) on single dish calibration issues. Several notes have been generated which detail the procedures required for continuum calibration at the GBT and a sketch of the implementation for spectral line analysis which will integrate into the synthesis calibrate tool. Coordinated with the ATNF group on use of spectralcoordinates in DISH. A test case has been implemented which demonstrates the possible code sharing and integration. On to-do list. Developed a general zoom function outside of DISH as a plugin to users of the pgplotwidget.

Worked with Tim Cornwell and Ralph Marson on MSPLOT requirements testing and functionality for single-dish use.

Replaced sdimager with an interface (quasi-wizard) to the imager module in AIPS++ as part of improving integration between single-dish and synthesis tools.

Developed a new command line interface (CLI) based on user response, called SSA (Scan-based Single-dish Analysis). It builds on the previous revision, constructing simple macro commands from the base language.

Help in commissioning data taking and reduction. First beam map observations produced (mainly Tim Cornwell's contribution)!

Reviewed GBT proposals and summarized backend, observation-type, etc. for each project to derive a list of first science requirements on AIPS++.

Generated a template tool for specific GB fitting needs. Not implemented as a general operation yet.

Investigated speed issues in GB and ultimately found (with Bob G.) disappointing startup performance when cache and state files are on a non-local disk. There is an implemented (band-aid) solution to this.

Continued to mail out weekly reports on the unit testing and (quasi) daily reports on builds and assay.

Perform weekly shippings from aips2-requests and maintain customer database. Release V1.4 had a distribution of about 750 CDs out, going to at least 34 nations (CDs handed out at meetings are not logged).

Worked with Arecibo staff to produce initial AO reduction scripts. Helped with their continuing installation problems.

Hosted and organized a small single-dish working group (Karen O'Neil (NAIC), Ellen Howell (NAIC), Dana Balser (NRAO-GB) and Jim Braatz (NRAO-GB/AIPS++). This group spent 3 days working with DISH and the new SSA CLI to learn about the software and help identify deficiencies or needed enhancements.

George Moellenbrock

For the v1.5 development cycle, George Moellenbrock contributed as follows. In the area of synthesis (and related) development:

Developed fringe-fitting prototype in Glish. The prototype includes standard fringe-fitting operations (including gridding/convolution of data samples and the Fourier transform, Alef-Porcas, and Cotton-Schwab fringe solutions) and provides a platform for modeling new fringing approaches and visualization of them. The current version includes use of the viewer to display animations of the fringe search. The current release contains a limited C++ conversion of the prototype that includes the transform and Alef-Porcas solutions. The next cycle will see the fringe-fitter mature to include the Cotton-Schwab solution and flexible fringe search-range selection, among other things.

Enhanced (in the course of fixing defects) the graphical chooser GUIs used to select antennas and baselines in msplot. These choosers are very effective mechanisms for improving ease of use, and should be further improved and uniformly extended to other tools (e.g., imager, calibrater).

Enhanced operation of calibration solution plotter (calibrater.plotcal) including production of multiple plots per page and spectral window selection.

Continued study of the overall operation and structure of AIPS++, with some emphasis on the calibration and MS definition and access classes.

Spent considerable time assisting local AIPS++ testers with use of AIPS++. My own knowledge of the system and its operation improved sufficiently during this cycle to allow me to be a more effective tutor to the testing group. Generally speaking, ease of use and bulletproof-ness continue to be the principle concerns of the testing group with respect to promoting AIPS++ with new users. Ease of use will improve with improved ``cookbook-type'' documentation and establishing better consistency of behavior and operation across the different tools (e.g., consistent message logging, consistent appearance and style of parameter entry). The package will become more bulletproof as new users explore new and different permutations of operations.

Assisted A. Kemball in conducting a weeklong AIPS++ user tutorial at NRAO in Charlottesville, VA in December 2000. Attended the AIPS++ User Group meeting at NCSA in January, 2000, and presented a demonstration of scripting in Glish using the generic tools (fitting, fftserver, quanta, measures, table, etc.).

Used Glish and the generic scripting tools to perform a detailed and direct comparison of the output of the three correlators used for the VSOP mission (Mitaka, Penticton, and VLBA). This comparison illuminated significant differences in the amplitude scale realized at these correlators and demonstrated remarkable consistency in their phase response. The LBA correlator (in Epping, Australia) and the EVN correlator will be added to this comparison in the near future.

Darrell Schiebel

My primary responsibilities include maintenance and development of the Glish scripting language, system level release preparation, and minimal maintenance (fixing serious defects) of the code development and distribution system.

During this development cycle most of my effort has been invested in exploring Glish problems without (unfortunately) resolving any major problems. These problems fell into two categories memory leaks and performance problems. Many of the problems with memory leaks were fixed during past development cycles, but a report from JIVE revealed that some memory leaks still remain. A fair amount of time was invested in this without resolution, and then work stopped on this path in order to investigate GUI performance problems with Glish/Tk in advance the developers meeting. Understanding the Glish/Tk performance problems was complicated by the fact that the GUI is created and updated via events between two separate processes. These events are sent through a Unix pipe, and surprisingly there do not seem to be any tools available for analyzing the dynamic behavior of multiple processes loosely coupled via pipes. It was clear that the raw throughput of the pipes was sufficient and comparable to other IPC mechanisms. These tests implied that the problem was delays both in sending and receiving the events, i.e. packing, unpacking, and context switching on the interpreter side. Threading of the Glish interpreter is part of the solution to this problem, and it will be the first fix to be implemented. It should improve the speed of context switching within the interpreter and open other avenues for addressing this problem, e.g. shared memory and clients as a thread within the interpreter. It will also make it possible for Glish clients to be multi-threaded. Significant progress toward threading the interpreter is expected during the next development cycle. Some defects were also fixed during the cycle, but none were significant defects.

I also did the initial work with putting the release together. Sadly, this will be my last release preparation as the honor now falls to Wes Young.

Helped track down many SGI/C++ issues in the switch to using the standard C++ library string, complex, and iostream classes.

Solved a nasty page allocation/freeing problem Ray Plante encountered on the SGI.

General support, i.e. finding and fixing bad links and sorting out LaTeX build problems.

Appendix E: Summary of Staff

We list below the names of people in the various AIPS++ groups and the nominal fraction of time allocated to AIPS++.

Neil Killeen (60%), Wim Brouw (65%), Malte Marquarding (75%), and Mark Wieringa (15%).

Dave Mehringer (50%), Ray Plante (50%), Harold Ravlin (50%), and Anuj Sarma (50%)

Braatz (75%), Cornwell (50%), Garwood (75%), Golap (75%), Kemball(75%), King (100%), Marson (75%), McMullin (75%), Moellenbrock(75%), Schiebel (100%), and Young (100%).