Getting Started Documentation Glish Learn More Programming Contact Us
Version 1.9 Build 000 News

FAQ

Search

Home

AIPS++ Quarterly Report: 1996 Q1

Tim Cornwell, AIPS++ Project Manager

April 15, 1996

Introduction

This report summarizes the status of the AIPS++ Project at the end of the first quarter of 1996. It describes the achievements during that quarter and gives a list of planned developments for the second quarter of 1996. It also describes long-term plans for the Project. Reports from each active AIPS++ site are included in appendix A.

Developments in 1996 Q1

In Single Dish, a development plan was formulated and fixed for the time being. This plan includes work needed for both the GBT development and the Parkes HI survey. Work has continued in a number of areas:

David Barnes of the ATNF Parkes HI survey team spent 8 weeks working with the SD group in Charlottesville.

In Measures, Wim Brouw froze the design and worked on implementation. Completion of all major features except high precision VLBI support is expected in May. Parts of the code are now being reviewed.

In Synthesis processing, a demonstration is available at:

http://www.cv.nrao.edu/aips++/docs/uvdemo/101/101.html

This shows the use of the imager program to self-calibrate VLA data solving for both normal antenna-based gains, and polarization leakage effects (the "D-terms"). A substantial amount of effort was expended on optimization of this program, with results that should benefit general applications development in AIPS++. Revised specifications for various telescopes were incorporated into a development plan. The design for the MeasurementEquation was refined, splitting the ME into two parts, a SkyEquation and a VisEquation, thus easing the eventual application of this formalism to single dish processing. In coordination with the single dish group, the MeasurementSet was redesigned, moving away from one big AIPS++ table for all data to a more conventional main table / sub tables scheme. This redesign was easily implemented and is now in use in synthesis processing. Calibration information was made persistent by writing to AIPS++ tables, in which form, it can be viewed by tools in Glish/Glishtk such as the tablebrowser and the plotting utilities. All these are steps on the way to a full calibration package. This work has been performed in collaboration with Mark Wieringa, who is spending six months working at the Center with the synthesis group (this visit is funded by ATNF and NFRA).

In Visualization and Image Analysis, the NCSA group made various improvements to the aipsview visualization tool with the goal of a beta release by April 15. The major area of improvement is the inclusion of contour plotting via the PGPLOT library.

In AIPS++ Infrastructure, a large number of improvements were made in response to the requirements arising from applications development:

In Documentation, we decided on an overall approach to programmer documentation, and began to implement it. An initial version of a revised programmers manual is now in place and being improved.

In the System area, we worked on the implementation of shared libraries under the Sun native compiler, and changed to rely upon native exception handling. In addition, we have made ports to various compilers:

In Management, we continued the implementation of a more formal scheme for proposing and accepting changes to AIPS++.

Long-term plans

The long term plans for the AIPS++ project were first presented in the previous Quarterly report. Here we present an updated list of AIPS++ goals in the next table:

AIPS++ Goals
Date
Milestone
Reliability
January 96First synthesis application - polarization self-calibration and imaging High
April 96Graphics system independent of Xrt/Graph Medium
May 96Measures system largely complete High
Spring 96First version of tasking system and user environment High
Mid 96Initial synthesis calibration package Medium
July 96Support "friendly" observer at 140' High
Mid 96Table system largely complete High
Mid 96Framework for user and programmer documentation complete High
Q3 96First image analysis application Medium
August 96Support multi-beam observing at Parkes High
August 96Support use of TMS at WSRT - simple calibration and editing of synthesis data present High
September 96First meeting of AIPS++ Scientific and Technical Advisory Group High
October 96Beta release of OpenInventor Aipsview High
October 96Measures support for VLBI Medium
October 96Wide-field imaging for VLA data Medium
December 96First mosaicing applications Medium
January 97First beta release of AIPS++ Medium
July 97First full release of AIPS++ Medium
Mid 97Support GBT commissioning - fully capable Single Dish package present High
Early 98Complete calibration and imaging for VLBI Medium
Late 99Complete migration of AIPS, Miriad, NewStar capabilities to AIPS++ Medium

As before, the reliability gives an assessment of the probability of meeting that goal. This is based upon a number of factors: how well specified the goal is, the difficulty of the goal, and the intrinsic priority of the goal to AIPS++.

Only one major milestone has been passed since the last report: the first synthesis application (denoted by the mark).

Plans for 1996 Q2

The previous section gave long-term plans. Here we summarize the expected developments over the next quarter.

In Single Dish support, we will continue with the work required to support the friendly observer test at the 140' telescope in June 1996. This includes writing a number of Glish clients for analysis, and putting together all the key elements in the AIPS++ GUI.

In Synthesis support, we will continue the development of the polarization deconvolution and selfcalibration task, adding a simple user interface based upon the Tasking design, and a GUI based upon Glishtk. We will continue development of simulation software for testing and verification. We will write a simple mosaicing task to test the applicability of the concepts in the MeasurementEquation formalism. We will finish the WSRT filler into the MeasurementSet and perform some trial reductions of WSRT data. We will write new tools for gridding and fast Fourier transformation of visibility data. We will begin an investigation of the adaptation of the synthesis code to parallel processing.

In Measures, we will complete the implementation, save for the highest precision modes needed for VLBI and pulsar observations. We will write a modified Image Coordinate System incorporating the Measures system.

In Glish support, we will work towards improving the stability and robustness of Glish and Glishtk. We will revise the documentation of Glish to bring it up to date. We will develop a test suite for Glish capabilities. We will issue a new release of Glish.

In AIPS++ Infrastructure, we will develop a Table data manager for sparse data, allow the use of Record (the AIPS++ heterogeneous container) in Tables. We will make a number of improvements to the tablebrowser, for both functionality and efficiency.

In Visualization and Image Analysis, we will issue a beta release of aipsview complete with contouring and other improvements. We will study the possibility of incorporating into aipsview support for AIPS++ Images and full non-linear coordinates. We will issue a development plan for the ATNF/NCSA collaboration on Visualization and Image Analysis. We will continue work targeted towards a beta release of Inventor AIPSView in July 1996.

In the System area, we will investigate possible schemes for reducing the deleterious effects of the wide-spread use of templates in AIPS++.

In Documentation, we will continue the revision of the programmers manual.

Project-wide concerns

Inadequate staffing

In the last report, it was noted that the Project was inadequately staffed. In particular, the following positions were deemed to be particularly important:

None of these have come about. The danger to the Project is obvious: a possibility that we will miss some of the major milestones.

Transition to an operational system

According to the timetable presented above, AIPS++ will begin to be available for end-users in early 1997. It is vital that we start planning for this now. The most pressing concerns are to:

  1. Ensure that interesting functionality is present for some users,
  2. Improve user and programmer documentation,
  3. Improve end-user installation procedures
  4. Ensure adequate testing prior to any release.

Addressing some of these concerns will require additional staff. 1 is implicitly addressed in the various development plans. The functionality targeted in the beta release of AIPS++ will be specified in June 1996. The user side of 2 is to be addressed by Alan Bridle, who plans to resume work inside the Project in May. The one person currently dedicated to programmer documentation is soon to be removed from the AIPS++ group. The inadequacy of the end-user installation procedures is well demonstrated (see e.g. the ATNF and BIMA/NCSA site reports). Inside NRAO, we expect to draw on the experience of the AIPS group in end-user installations. 4 could usefully be addressed by scientists from the various consortium members. An example of this type of work is current testing of the Glish system where Harvey Liszt and Darrell Schiebel are scheduled to work together for a few months with the specific goal of finding and fixing bugs. Similar efforts should be started with other parts of the system later in the year.

Standard hardware configuration for AIPS++

We have attempted to define a standard configuration for a machine that is to run AIPS++. The basic formulation is that AIPS++ should perform well for problems of a normal size on a Unix-based workstation costing $5,000. An exemplar of such a machine is a Sun Sparcstation 4, with 64 Mbyte of main memory, and a few gigabytes of disk storage. Our main concern is that the memory will be too small for interesting problems. We will soon have to fix a policy for the use of memory inside AIPS++ programs. We plan to continue to track this issue closely over the next 6-12 months.

Scheduling

To better quantify our ability to meet targets, we have reviewed the success of the target dates scheme that we use to track progress. In the 9 months since we instituted the target dates in May 1995, we have completed 111 targets. Of these, 3 were in advance, and 48 were more than 10 (calendar) days late (where the average duration of a target is about 20 days). The 48 late targets can be categorized in the following way:

19 Underestimated difficulty of target (e.g. complex design issues)

19 Person obviously overloaded (usually due to unforeseen problems or, more rarely, other targets in conflict)

4 Obviously bad choices for target dates (e.g. inappropriate dates for some external event such as ADASS)

3 External factors (e.g. waiting for someone outside the project to do something)

3 Person culpable

The division is highly subjective, and the analysis is clearly limited. However, the basic conclusions are that we need to pay more attention to the first two types of problem. Fixing the first (underestimating the difficulty) will require more intermediate targets as checks. The second (overloading of time) is harder to deal with since it tends to be interrupt-driven, as key people are called upon to help diagnose or fix urgent problems. Perhaps the best approach here is to ensure that everyone is aware of this latter effect.

Reliance on the Internet

AIPS++ is suffering from the side-effects of the vast increase in interest in and use of the Internet. We rely upon the Internet for a number of purposes:

Both the latter uses are being adversely affected by the general increased load on the Internet. One answer inside NRAO may lie in the tentative proposal for an NRAO Intranet. This will do little to address difficulties for external sites and so we anticipate having to do some work make our code management and distribution system more useable over a poorly responsive network. Note that a decision on the possible move of the code master to Socorro has been deferred until the question of an NRAO Intranet has been settled.

Scientific and Technical Advisory Group

The member of the advisory group is now mostly settled, with a few remaining possible additions. See Appendix D for the list of members. The group is expected to meet first in September or October. The purpose of the group is to advise the Project Manager on scientific and technical matters relevant to the Project.

Appendix A: Site reports

ATNF

The ATNF currently has 5 people working in AIPS++. These are Neil Killeen (local manager), Mark Calabretta, Wim Brouw, Mark Wieringa and Tom Oosterloo. Additionally, there is a collaboration between the AIPS++ project and the ATNF headed Parkes 21cm multi-beam receiver project, which includes Taisheng Ye working primarily in AIPS++.

In Epping we have acquired a new Sun Ultra 140 server specifically for AIPS++. It is configured with 192Mb of memory (essential for AIPS++ C++ compilations to run at a reasonable speed) and a fast-wide 9Gb disk.

Mark Calabretta's main responsibility is to the code distribution system. His time in the last quarter has been spent on:

Mark has spent approximately 50% of his time on AIPS++ related work. His nominal AIPS++ allocation is 50%. At present, this level will be sustained indefinitely.

Wim Brouw's responsibility is mainly to designing and implementing the measure and related classes. His time has been spent on:

However in trying to finalize the full measure class and its conversions, Wim encountered an obscure cfront compiler error. With help from NRAO, this was eventually bypassed. This plus some other pressing work has delayed delivery for the full system (due 22/mar/96).

Wim notes that his productivity was badly damaged by spending about 40% of his AIPS++ time on system/compiler related issues. He has spent 22% of his time on AIPS++ related issues in this quarter.

Mark Wieringa is a part of the team working on the synthesis related classes. He is currently in Socorro on his second 3-month stay there working with Tim Cornwell. He has spent his AIPS++ time on

Mark has spent 75% of his time on AIPS++ in this quarter. His nominal AIPS++ allocation is 50%.

Mark's involvement in AIPS++ is negotiated yearly with Narrabri since he has been seconded to the project from the Narrabri computer group. Currently the agreement is until mid-1996.

Tom Oosterloo is working on visualization and image analysis software. This work involves collaboration with the NCSA AIPS++ group. His time in the last quarter has been spent on

Tom's nominal AIPS++ allocation is 75%. He has spent about this amount on AIPS++ in the last quarter.

Neil Killeen spends most of his AIPS++ time attending to local ATNF AIPS++ management issues but is also beginning to plan some simple image analysis software. Neil writes that he has spent his AIPS++ time on

I have no nominal AIPS++ time allocation, and the load varies. I would guess that averaged over the last quarter, I have spent some 30% of my time on AIPS++.

The multi-beam received project is using AIPS++ as its software base. Their main experience comes from David Barnes who spent a few months in Charlottesville starting in early 1996. They report:

BIMA/NCSA

The BIMA/NCSA effort the past quarter continued to concentrate on visualization development. New features and bug fixes for aipsview, continued development of Inventor aipsview, AIPS++ graphics, and installation of a full AIPS++ system at NCSA were the major efforts. Individual efforts are listed below:

Dick Crutcher (25% time): Time was spent on AIPS++ management, testing aipsview, and writing aipsview user documentation.

Polly Baker and George Baxter (50% time each): In the area of 3D visualization, they continued work toward a July beta release of the Inventor aipsview program. They did considerable cleanup to the code supporting Inventor aipsview, started a review of the in-code documentation, built a tool for reading the VRML files produced by Inventor aipsview into NCSA's virtual environments: the CAVE and ImmersaDesk. In part, this served as a warm-up exercise to building a tool that can support Inventor aipsview's data visualization functions in these environments.

Harold Ravlin (80% time): He is the main person responsible for aipsview maintenance and improvements. The major effort has been to add the pgplot vector graphics package to aipsview. Major new features which have been added to aipsview this quarter are the ability to plot contours, to show a box around an image display area with axis ticks and coordinate annotation, to show a colormap wedge when an image is shown as a raster and the axes are displayed, and to produce a postscript file (for printing) of raster images and/or contour representations of images, with or without axis notations. Contour values may be set as the user chooses, contours may be displayed alone, on top of the associated raster image, or on top of another raster image. Contours update just like rasters when animations are run. Axes annotations are controlled by defaults. Various minor features and bug fixes were also added to aipsview.

John Pixton (100% time): He has worked mainly on producing a plot1d widget using pgplot that will replace the commercial widget now used in AIPS++ for general vector graphics work. This is expected to be finished by the middle of April. In addition, he worked on a design for an interface between aipsview and external programs, that might be used to interface existing fortran or C analysis codes with aipsview. He also worked to fix some aipsview bugs, and assisted Randy Sharpe in the installation of a full AIPS++ Solaris system at NCSA.

Randy Sharpe (20% time): He installed a full AIPS++ installation at NCSA on a Solaris system and began work on a port of AIPS++ to our SGI Power Challenge Array. This work will be taken over by Doug Roberts, who will become a 50% AIPS++ worker during the second quarter, with a focus on parallel processor applications.

Peter Teuben (10% time): He worked on the specification of BIMA requirements for calibration of BIMA data. This should be finished by the end of the quarter.

NFRA

Local project members: Ger van Diepen (GVD), Jan Noordam (JEN, local manager), Friso Olnon (FMO), Jayaram Chengalur (JNC, local Project Scientist)

General

It has not been possible to hire an AIPS++ application programmer at NFRA in time to play a significant role in the commissioning of the first production version of the new WSRT Telescope Management System (TMS) in august 1996. In view of the financial situation of NFRA, it is also increasingly doubtful that such a person can be hired in 1997, despite general agreement that it is highly desirable.

Jan Noordam has checked in version 2.0 of AIPS++ Note 185.

AIPS++ applications

Friso Olnon has delivered a working prototype of the WSRT data filler. The work has been delayed somewhat by the revision of the MeasurementSet.

AIPS++ Site in Dwingeloo

End of February the regular maintenance of NFRA's AIPS++ site was transfered from the local AIPS++ project group to the general R&D maintenance group (FAM), consisting of Henk Vosmeijer and Friso Olnon. As part of this transfer activity, Olnon wrote the "NFRA/AIPS++ Configuration Manual" conform the NFRA standard, which is essentially an introduction to the AIPS++ System Manual by Mark Calabretta.

At the same time the NFRA installation was extended to include an HP Unix host, which is required for the new WSRT on-line system TMS. The environment for AIPS++ work now exists, but not all standard AIPS++ code compiles and links on the HP. As long as the Center does not have an HP itself, the NFRA group will continue to assist the Center in supporting the HP platform in the future.

The AIPS++ installation on the Sun/Solaris host keeps working without much trouble. That is the platform on which all current local AIPS++ development is done. At the time of writing (March 5) we have problems building Glish clients, but the FAM group is working on that with the help of Darrell Schiebel.

Ger van Diepen has conducted two C++ workshops at NFRA. This included advanced subjects like "counted reference", "copy-on-write" and "expression tree".

Infrastructure software.

Ger van Diepen has put considerable work into documenting the new Table software. He also implemented some small changes in the Table system at the request of some users.

He has developed a framework for data format conversion (e.g. VAX<->IEEE). This will be used in the MeasurementSet filler, and perhaps by TMS. The student Anco Boersma has developed an IO framework, which will allow transparent IO in which the conversion classes can be easily integrated.

The development of the new Record classes is almost finished. With these classes, arbitrary recursive structures can be specified, instantiated and made persistent. The Table system supports these structures.

Miscellaneous

NFRA shares with ATNF the cost of sending Mark Wieringa to the Centre.

NRAO

Personnel

Wes Young has been reassigned back to the AOC Computer Division, effective May 1, probably for a period of about two months, but only as a backup for system administration.

Appendix B: Summary of AIPS++ Personnel

In this section, I give the names of people in the various AIPS++ groups and the nominal fraction of time allocated to AIPS++.

The ATNF group is: Mark Calabretta (50%), Wim Brouw (50%), Mark Wieringa (50%) and Tom Oosterloo (75%), with Neil Killeen as the local Manager.

The BIMA/NCSA group currently has 4 active workers, as well as Dick Crutcher, the local manager. These are Polly Baker (50%), John Pixton (100%), Harold Ravlin (80%), and Doug Roberts (50%); In addition, Jim Morgan and Peter Teuben at Maryland follow AIPS++ developments and expect to become directly involved in the future.

The NFRA group is: Ger van Diepen (100%), Jan Noordam (25%), Friso Olnon (50%), and Jayaram Chengalur (25%)

The NRAO group is: Tim Cornwell (100%), Bob Garwood (90%), Brian Glendenning (100%), Ralph Marson (100%), Tim Roberts (100%), Darrell Schiebel (100%), Paul Shannon (100%), Shelby Yang (100%), and Wes Young (90%). In addition, a number of scientists participate at various levels: Alan Bridle, Rick Fisher, Bob Hjellming, Harvey Liszt.

Appendix C: Review of developments promised for 1995 Q4

This section has been written to mirror the section in the last quarterly report on plans for 1995 Q4.

In Single Dish support, we did implement an improved tablebrowser, based upon the Glishtk GUI machinery. We did not yet write a first GUI with simple astronomical task, but we did complete approximately 50% of the Glish C++ clients needed for the observations. David Barnes did visit Charlottesville to work with the group for 8 weeks on support of the Parkes Multi-beam observations.

In Synthesis support, we did finish a task to do polarization leakage self-calibration and imaging, we did improve gridding and the use of FFTs, we did ensure persistence for calibration objects, we did develop capabilities for simulation of synthesis observations, but we did not write a first GUI-based imaging task. In preparation for the commissioning of the WSRT TMS, we did write a prototype filler to load WSRT data into a Synthesis MeasurementSet. We did not yet start a collaboration between the Synthesis group and NCSA to investigate and possibly implement selected synthesis algorithms on parallel processing machines.

In Measures, we did complete a re-design of the classes. A completed Measures implementation will not be available by the end of the quarter but we expect it to be finished mid-April.

In AIPS++ Infrastructure, we did continue work on the Table system, finishing the Tiled storage manager. We deferred improving the existing Run-Length-Encoded storage manager (called the "Miriad" storage manager). We did complete a re-write of the File and I/O classes. We did implement a system allowing time-changing data such as IERS data to be available inside AIPS++ via the Tables system. The VLA calibrator list is one of the first sources of data included. The implementation of the tasking system (http://www.cv.nrao.edu/aips++/docs/notes/185.ps) did continue, first with the control hub and C++ run-time system, and then with a sample application to the Table system.

In Visualization and Image Analysis, we have make progress but did not yet finish re-implementing the gplot1d Glish client on top of the PGPLOT graphics widget. We did investigate and proceed with putting PGPLOT graphics capabilities directly into Aipsview, and we did add hardcopy output to Aipsview.

In the System area, we did partially implement shared libraries. We did not add automatic registration for the AIPS++ e-mail exploders (a "majordomo"), nor transfer the AIPS++ master code repository to Socorro. We did attempt ports to the g++ compiler, and to Dec Alphas, SGI and HP/UX configurations. We did implement the now-standard compiler exception mechanism.

In Documentation, we did revise the programmer's section of the on-line documentation, identify and remove out-of-date documents. We started on but did not complete a revision of the format of source code documentation. We did put programming entries in the AIPS++ Glossary. We did fully implement and start to use the gnats bug reporting system.

In Management, we did form and schedule a meeting of a Scientific Advisory Group for AIPS++. Tim Cornwell did not visit Pune for discussions with GMRT staff.

In addition, Allen Farris of STScI did re-design and re-implement the FITS classes used by AIPS++. We will benefit directly from this work.

Appendix D: Membership of AIPS++ Scientific and Technical Advisory Group

The following have agreed to serve:

Robert Braun NFRA

Jay Chengalur NCRA/NFRA

Roger Foster NRL

Walter Jaffe Univ. Leiden

Harvey Liszt NRAO

Lee Mundy U. Maryland

Bob Sault ATNF

Lister Stavely-Smith ATNF

Dave Shone NRAL

Huib van Langevelde JIVE

Tony Willis DRAO