Getting Started Documentation Glish Learn More Programming Contact Us
Version 1.9 Build 000 News

FAQ

Search

Home

AIPS++ Quarterly Report: 1995 Q4

Tim Cornwell, AIPS++ Project Manager

January 11, 1995

Introduction

This report summarizes the status of the AIPS++ Project at the end of the fourth quarter of 1995. It describes the achievements during that quarter and gives a list of planned developments for the first quarter of 1996. It also describes long-term plans for the Project. Reports from each active AIPS++ site are included in appendix A.

Developments in 1995 Q4

In Single Dish processing, the Charlottesville-based group (Bob Garwood, Darrel Schiebel, and Paul Shannon) continues to work closely with the GBT on providing AIPS++ software for single dish processing. The most important goal is to allow use of AIPS++ single dish data analysis processing by a first user on the NRAO 140' telescope in June 1996. This is a key milestone in the overall goal of providing a complete SD analysis package for the commissioning of the GBT, now expected in the summer of 1997.

In addition, the ATNF has decided to use AIPS++ to support an HI survey using a 13-feed multi-beam system on the Parkes telescope, scheduled for August 1996. The goals of the multi-beam project and a development plan for the software are available on the WWW. To support this work, David Barnes will travel to CV for 8 weeks early next year to work with the SD group.

Darrell Schiebel has developed an extension to Glish, called Glishtk, that allows construction of a Tk-widget based GUI from inside Glish. This was originally developed for the GBT work, but is expected to be used in a wide range of different applications. It gives the ability to design and implement a GUI from the Glish command line. Paul Shannon has developed a number of tools based upon Glishtk, the major being a system for providing help for users of Glish, and a tool for overseeing and controlling the filling of GBT Monitor and Control data into AIPS++. Furthermore, this work and the Tasking work by Brian Glendenning have led to a design for the AIPS++ GUI. This design will be implemented over the next few months using Glishtk and the Tasking classes.

A single dish development plan has been produced by Bob Garwood taking into account the needs of GBT and the Parkes Multi-beam projects. This plan is currently found on Bob's home page on the WWW and soon will be checked into the AIPS++ system.

In Measures, Wim Brouw has made excellent progress on designing and implementing the Measures Classes. Working from an original design by Mark Calabretta, he expects to have all of the system, apart from high-precision VLBI support, available by April 1996. Much of the coding has been done now but he is now redesigning the interface partly in response to comments by others and partly as a result of his own experience in implementing the classes.

In Synthesis processing, Mark Wieringa and Tim Cornwell have produced a design and implementation for synthesis calibration and imaging based upon the Measurement Equation for a Generic Interferometer developed by Hamaker, Bregman, Sault and others. The short term goal is to produce a self-calibrated and cleaned image from VLA data, incorporating corrections for polarization leakage, by January 1996. Development will then proceed on various fronts. One important goal will be to allow support of the commissioning of the new WSRT on-line control system, TMS, expected to commence in August 1996. Another is an early test of the AIPS++ GUI mentioned above by implementing a general imaging and self-calibration task. This will be made available to users for testing and use. A draft development plan for the synthesis area is available from Tim Cornwell's home page. This will be completed after receiving additional advice on VLBI and BIMA processing.

In Visualization and Image Analysis, Tom Osterloo has joined the AIPS++ Project at ATNF to work in collaboration with the NCSA group headed by Dick Crutcher. This was agreed in late September. Unfortunately, ATNF has since had to reallocate some of Tom's time in the next year to support the Parkes Multibeam Project with the result that the Image Analysis work that Tom was slated to do will be delayed by about 6 months. This will delay Image Analysis work at a time when some rudimentary capabilities will be sorely needed. Negotiations are underway to find a solution.

The NCSA group has worked on the graphics capabilities in AIPS++. Currently for the GBT tests, we use a Glish client, gplot1d, that calls a commercial widget, Xrt/Graph. A tentative plan was to replace this at some point with classes developed at Fermilab. A better option is to instead use a Motif-based widget for the Caltech PGPLOT library that is widely used in Astronomy. John Pixton of NCSA is currently investigating this possibility. In a related but separate line of development, Harold Ravlin is investigating the possibility of including PGPLOT graphics directly in AIPSView.

In Infrastructure, the Table system continues to evolve and grow towards completion. Ger van Diepen of NFRA has recently worked on a Tiled Storage Manager that allows efficient access to multi-dimensional arrays in different directions. This will be vital in many different application areas. In addition, the core group working on the related classes Table, Lattice and Image (Ger van Diepen, Brian Glendenning and Tim Roberts) have worked to act in response to applications needs. For example, the synthesis work by Cornwell and Wieringa produced a requirement for the Image class to allow images of objects of fixed form. The case in point was an image of StokesVector: a class representing a vector in I,Q,U,V space. A modification of the Table system was agreed to, designed and implemented, all within a few weeks. Similar changes to the AIPS++ library are expected to occur repeatedly in the future as applications needs come to drive the Project.

Another area of infrastructure improvement is the porting of the Newstar Linear Least Squares facilities (developed by Wim Brouw) to AIPS++. This is still in progress.

In Documentation, we have resolved our staffing problem mentioned in the last Quarterly report: Wes Young of the NRAO/Socorro computing division is now allocated almost 100% to AIPS++ and is working primarily on programmer documentation. A large number of changes are now in progress. AIPS++ has adopted the Free Software Foundation's gnats program to track bug reports and change requests. The system has a particularly straightforward HTML interface that we plan to link to from every piece of HTML documentation inside AIPS++.

In the System area, we have implemented automated testing whereby the entire test suite is run weekly and the results monitored. The goal is to catch problems with the Library early on. We are now working towards rectifying existing problems. We have also implemented registration of template specializations, and are in the process of implementing shared libraries, something that will be vital to reduce the typical size of executables.

The Project was well-represented at ADASS this year. Presentations were made by Brian Glendenning (Creating an Object-Oriented system: the AIPS++ experience), Dick Crutcher (The AIPSView Astronomy Visualization Tool), and Bob Garwood (AIPS++ and the GBT), and, in addition, posters were presented by Darrell Schiebel (Programming in Glish) and Paul Shannon (GUI Tools in AIPS++).

In Management, a major push was made during the last quarter to formulate development plans for various key areas of the Project. A development plan is essentially a detailed time-line with deliverables, responsibilities and requirements on other parts of the Project all laid out clearly. This is a vital discipline for a project as diverse and complex as AIPS++, serving not only to aid in management but also in ensuring that all members of the Project are aware of goals and activities. We also implemented a formal scheme for managing substantial changes to the system, involving a proposal, request for comments, and decision date. A description is available on the Web.

Long-term plans

In the two previous quarterly reports, we have presented only timelines for the next quarter. The AIPS++ development plan put in place in the summer of 1995 purposefully had a look ahead time of about a year. Now that the Project is running more smoothly and achieving goals with some regularity, it is possible to lay down longer-term plans. A list of AIPS++ goals is given in the next table:

AIPS++ Goals

     Date                            What                       Reliability

    January 96 First synthesis application - polarization       High       
               self-calibration and imaging                                

      March 96 Graphics system independent of Xrt/Graph         Medium     

      March 96 Measures system largely complete                 High       

     Spring 96 First version of tasking system and user         High       
               environment                                                 

      April 96 First mosaicing applications                     Medium     

       July 96 Support "friendly" observer at 140'              High       

     Summer 96 Table system largely complete                    High       

     Summer 96 Framework for user and programmer documentation  High       
               complete                                                    

       Fall 96 First image analysis application                 Medium     

     August 96 Support multi-beam observing at Parkes           High       

     August 96 Support use of TMS at WSRT - simple calibration  High       
               and editing of synthesis data present                       

  September 96 First meeting of AIPS++ Scientific and           High       
               Technical Advisory Group                                    

    October 96 Beta release of OpenInventor Aipsview            High       

    October 96 Measures support for VLBI                        Medium     

    October 96 Wide-field imaging for VLA data                  Medium     

   December 96 Initial VLA calibration package                  Medium     

    January 97 First beta release of AIPS++                     Medium     

       July 97 First full release of AIPS++                     Medium     

     Summer 97 Support GBT commissioning - fully capable        High       
               Single Dish package present                                 

      Early 98 Complete calibration and imaging for VLBI        Medium     

       Late 99 Complete migration of AIPS, Miriad, NewStar      Medium     
               capabilities to AIPS++

When reading this timetable, there are a number of points to be borne in mind.

Plans for 1996 Q1

The previous section gave long-term plans. Here we summarize the expected developments over the next quarter. The overview is that applications development is heating up and key elements of the AIPS++ library are approaching completion. Three time-critical milestones occur later in the year (support at GBT, Parkes, and WSRT), and so the work in this next quarter is largely in the necessary preparation.

In Single Dish support, we will work towards the goal of supporting the use of AIPS++ by a friendly observer on the 140' telescope by June 1996. We will write a filler to load GBT data to a single dish MeasurementSet. We will implement an improved tablebrowser, based upon the Glishtk GUI machinery. We will write a first GUI with simple astronomical task, and we will complete approximately 50% of the Glish C++ clients needed for the observations. David Barnes will visit Charlottesville to work with the group for 8 weeks on support of the Parkes Multibeam observations.

In Synthesis support, we will finish a task to do polarization leakage self-calibration and imaging, we will improve gridding and the use of FFTs, we will ensure persistence for calibration objects, we will develop capabilities for simulation of synthesis observations, and will write a first GUI-based imaging task. In preparation for the commissioning of the WSRT TMS, we will write a filler to load WSRT data into a Synthesis MeasurementSet. We will start a collaboration between the Synthesis group and NCSA to investigate and possibly implement selected synthesis algorithms on parallel processing machines.

In Measures, we will complete a re-design of the classes. A completed Measures implementation will be available by the end of the quarter with only one exception, support for high precision VLBI, which is expected later in the year.

In AIPS++ Infrastructure, we will continue work on the Table system, finishing the Tiled storage manager and improving the existing Run-Length-Encoded storage manager (called the "Miriad" storage manager). We will complete a re-write of the File and I/O classes. We will implement a system allowing time-changing data such as IERS data to be available inside AIPS++ via the Tables system. The VLA calibrator list will be one of the first sources of data included. The implementation of the tasking system will continue, first with the control hub and C++ run-time system, and then with a sample application to the Table system.

In Visualization and Image Analysis, we will re-implement the gplot1d Glish client on top of the PGPLOT graphics widget. We will also investigate putting PGPLOT graphics capabilities directly into AIPSView. We will add hardcopy output to AIPSView.

In the System area, we will fully implement shared libraries, add automatic registration for the AIPS++ e-mail exploders (a "majordomo"), and transfer the AIPS++ master code repository to Socorro. We will attempt ports to the g++ compiler, and to Dec Alphas, SGI and HP/UX configurations. We will implement the now-standard compiler exception mechanism.

In Documentation, we will revise the programmer's section of the on-line documentation, identify and remove out-of-date documents. We will revise the format of source code documentation. We will put programming entries in the AIPS++ Glossary. We will fully implement and start to use the gnats bug reporting system.

In Management, we will form and schedule a meeting of a Scientific Advisory Group for AIPS++. Tim Cornwell will visit Pune for discussions with GMRT staff.

In addition, Alan Ferris of STScI is re-designing and re-implementing the FITS classes used by AIPS++. We will benefit directly from this work.

Requests for action or approval

  1. Increased staffing: For the last year, the staffing of the Project was well-matched to the activities planned. Now that we are moving towards considerable applications development, the Project is becoming short on staff in various areas. We should anticipate adding or reallocating staff sometime this year to a number of areas:

Executive committee response: NFRA is actively searching. Others took note.

  1. AIPS++ Scientific and Technical Advisory Group: The role of this group is to provide advice on scientific and technical matters to the AIPS++ Project Manager. We would like to aim for a small group of people initially, probably no more than 8. We expect that the group will grow in size as AIPS++ expands. This group would meet for a few days and then write a report to be given to the Project Manager and then passed on to the Executive Committee. It would meet first in Fall 1996 and thereafter yearly or more frequently, as desired by the Project Manager. To fill the membership, we would like 3 or 4 nominations from each Director on the Executive Committee. We prefer astronomers with a good background in computing issues. We propose that each organization pay the travel costs of its nominees and that NRAO will pay for the costs of the meeting.
Executive committee response: approved, nominations will be submitted shortly.

Appendix A: Site reports

ATNF

The ATNF currently has 4 active workers, as well as Neil Killeen, the local manager. These are Mark Calabretta, Wim Brouw, Mark Wieringa and Tom Oosterloo. Tom has just joined the project in early December. Bob Sault acts in a consultant mode.

The 21cm Parkes multibeam project (joint ATNF and several Universities about the world) has chosen to use AIPS++ as its software platform. Multibeam project members who are writing software will thus collaborate closely with AIPS++ project staff. In particular, David Barnes from the University of Melbourne will visit Charlottesville in early 1996 for 2 months.

Mark Calabretta's main responsibility is to the code distribution system. His time in the last quarter has been spent on:

Mark has spent approximately 60% of his time on AIPS++ related work (including WCSLIB). His nominal AIPS++ allocation is 50% At present, this level will be sustained indefinitely.

Wim Brouw's responsiblity is mainly to designing and implementing the Measure and related Classes. His time has been spent on

All routines have entries for further corrections (IERS etc).

Wim has spent 17% (October), 40% (November) and 12% (December) on AIPS++.

Mark Wieringa is a part of the team working on the interferometry related classes. He has just returned from 3 months in Socorro working with Tim Cornwell. He has spent his AIPS++ time on the basic design and implementation of full polarization calibration and imaging for "generic" interferometers. Good progress on the basic framework is now in place. Mark spent substantial time on efficiency aspects, and writing fast square matrix and stokes vector classes used in the computations.

Mark has spent 100% of his time on AIPS++ in this quarter. His nominal AIPS++ allocation is 50%. Mark will return to Socorro for a further 3 months in February. Mark's involvement in AIPS++ is negotiated yearly with Narrabri since he has been seconded to the project from the Narrabri Computer Group. Currently the agreement is until mid-1996.

Tom Oosterloo joined the AIPS++ project at the ATNF in December 1995; his appointment comers through the IISE demonstrator funding mentioned in the last quarterly report. He is working on visualization and image analysis software. This work involves collaboration with the NCSA AIPS++ group. The demonstrator positions have a joint divisional component. In this case, Tom will spend some time collaborating with the DRP ALLSITE project (a virtual work bench for sophisticated exploration of data sets). Tom is currently working on

Tom's nominal AIPS++ allocation is 75%. However, he is also currently winding up some work from his previous ATNF appointment, so he will be not be available for his full AIPS++ allocation until about mid February.

Neil Killeen currently does no actual code work and spends his AIPS++ time attending to local ATNF AIPS++ management issues. In the last quarter this largely revolved around

filling the IISE demonstrator position (Tom Oosterloo). This included planning and negotiating with Tim Cornwell and Dick Crutcher on the area of work and its structure for the next year or so. It also involved negotiating with the CSIRO Division of Radiophysics on the joint divisional aspects. Neil was also involved in discussing AIPS++ with Parkes 21cm multibeam project members and helping to facilitate the collaboration between them and the AIPS++ project

Neil has no nominal AIPS++ time allocation, and the load varies. A guess at the load is that averaged over the last quarter, he has spent some 15% of his time on AIPS++.

BIMA

Personnel activity:

AIPS++ activity has been in the area of visualization and graphics. Polly Baker, John Pixton, and George Baxter worked mainly on the OpenInventor 3D AIPSview code. The ability of the user to control the transfer functions used in mapping data to graphics has been extended. We also experimented with saving the current state of a visualization in a file format (SVML) that can be read by a CAVE viewer program that we developed. Eventually, this will provide a unified environment for data exploration across both desktop and virtual reality platforms. Other new functionality in OpenInventor AIPSview includes the ability to save a current visualization as a VRML file for sharing on the WWW. VRML files have been generated and are accessible for a number of data sets in the NCSA Astronomy Digital Image Library. This collection of VRML files has also been made accessible from the NCSA VRML home page. We expect to release OpenInventor AIPSview to the astronomy community sometime next year.

John Pixton also spent time on Glish development within AIPSview and investigation of vector graphics options. He is now working essentially 100% time on AIPS++ vector graphics. Harold Ravlin responded to user queries (help in installation and minor bug fixes) generated by our release of the 2D AIPSview code to the community at the end of September. Currently, over 100 people have registered themselves (through our AIPSview Web page) as AIPSview users. Harold also began work on adding vector graphics capability to AIPSview; this included learning about the PGPLOT library and using it to draw onto the AIPSview canvas for coordinate annotation, contours, and xy plotting. These features are expected to be in the released code by Feb. 15, followed by postscript output capability.

Other activity:

Conferences and presentations:

Dick Crutcher gave an invited talk on AIPSview at the ADASS V conference in Tucson and presented an on-going demonstration of AIPSview during the conference. His ADASS paper on AIPSview is checked into the AIPS++ system.

Dick Crutcher, Polly Baker, John Pixton, and Doug Roberts attended the Supercomputing 95 conference in San Diego and gave a demonstration of radio synthesis imaging data processing on remote parallel CPUs and of AIPSview visualization. Baker was also a member of the Technical Program Committee and chaired a paper session.

Polly Baker and George Baxter attended the annual IEEE Visualization '95 conference in Atlanta, GA. Baker was a member of the Conference Committee and organized the Birds-of-a-Feather sessions. Baxter (along with Pixton) was a co-author of the paper "Crumbs: A virtual environment tracking tool for biological imaging", which was presented by NCSA staff member Rachel Brady at the conference's symposium on Biomedical Imaging.

NFRA

Local project members: Ger van Diepen (GVD), Jan Noordam (JEN, local manager), Friso Olnon (FMO), Jayaram Chengalur (JNC, local Project Scientist)

General

NFRA is pleased with the level of consultation. The items from the "NFRA Requirements for AIPS++" find their way into the designs and plans. Noordam and Chengalur are actively following the design and implementation of the first version of uv-data calibration and imaging based on the M.E. Special concerns about tied arrays, the interface to the SkyModel, and the generality of the M.E. matrix expression have been met. Redundant Spacing Calibration will be addressed in a later stage.

TMS

Together with Marco de Vos, the project leader of the new WSRT on-line system TMS, a list of requirements was compiled for AIPS++ to support the first version of TMS in august 1996. TMS designs and plans can be accessed via the web.

AIPS++ applications

Olnon is on schedule to deliver a working prototype of the WSRT uv-data `filler' by the end of December.

AIPS++ Site in Dwingeloo

The local AIPS++ installation in Dwingeloo causes very few problems. Olnon checks the weekly `inhales' and makes small changes in the configuration where necessary.

Infrastructure

Van Diepen continues to work on the Table System, under the guidance of Brian Glendenning at the Centre. The tiled hypercube storage manager has been finished. It is now possible to dynamically add new columns to an existing or new data manager. A new data manager has been created which can store arrays of data objects directly into a table. Finally, Van Diepen has polished various aspect of the table system, including documentation and test programs.

Van Diepen's student Anco Boersma has finished the classes Path and File. Van Diepen has contributed to the debate about improvement of cxx2html.

Miscellaneous

NFRA shares with ATNF the cost of sending Mark Wieringa to the Centre, and contributes to the salary of one of the new Centre programmers.

It turns out to be difficult to find suitable candidates for a joint NFRA/JIVE position for an AIPS++ application programmer in Dwingeloo. We have dropped the requirement of experience in aperture synthesis, and concentrate on proven productivity and some experience in OOP/C++.

All three papers on Radio Polarimetry by Hamaker, Bregmam and Sault have been accepted by A+A without modifications. Hamaker will give a talk on the subject at the URSI General Assembly in 1996. Noordam has updated Note 185, the `official' description of the M.E. in AIPS++.

The Software Engineering Group (SEG) at NFRA, of which Olnon and Van Diepen are members, has produced an "C++ Coding Standard for NFRA" and a "Configuration Standard for NFRA". An internal workshop was held about the last subject. All SEG documents are available on the web: (URL: http://www.nfra.nl/~olnon/seg).

NRAO

Personnel

We are pleased to have Shelby Yang join the Socorro AIPS++ group. He arrived in November. Ralph Marson is due to arrive in early January. Wes Young is now working 90% of his time inside AIPS++, principally on Programmer Documentation.

Support

All members of the Charlottesville group continue to make extensive visits to Green Bank to support the GBT work. A two-day meeting with the GBT Monitor and Control group was held in Green Bank in early December to discuss a number of issues relating to the ongoing AIPS++/GBT activities.

Network problems

A major problem with the code management and distribution system occured when a hacker break-in at Charlottesville caused that site to severely curtail temporarily its Internet accessibility. Since this is the site of the AIPS++ code repository, this broke the code management and distribution system temporarily and caused many sorts of unexpected obstacles for a few weeks following, impairing the productivity of many people in the Project. The Charlottesville computer staff acted quickly and efficiently to first stop the break-in and then restore essential services such as the AIPS++ code distribution system. Although a firewall is now in place in Charlottesville, we may still be open to such attacks in the future.

Meetings

The group in Socorro has bi-monthly meetings on AIPS++-related subjects.

Appendix B: Summary of AIPS++ Personnel

In this section, I give the names of people in the various AIPS++ groups and the nominal fraction of tiem allocated to AIPS++.

The ATNF group is: Mark Calabretta (50%), Wim Brouw (50%), Mark Wieringa (50%) and Tom Oosterloo (75%), with Neil Killeen as the local Manager.

The BIMA/NCSA group currently has 4 active workers, as well as Dick Crutcher, the local manager. These are Polly Baker (50%), John Pixton (100%), Harold Ravlin (80%), and George Baxter (50%); Doug Roberts is expected to become an active AIPS++ worker at the 50% level in 1996 (as soon as SGI Array porting and development work becomes appropriate). In addition, Jim Morgan and Peter Teuben at Maryland follow AIPS++ developments and expect to become directly involved in the future.

The NFRA group is: Ger van Diepen (100%), Jan Noordam (25%), Friso Olnon (50%), and Jayaram Chengalur (25%)

The NRAO group is: Tim Cornwell (100%), Bob Garwood (90%), Brian Glendenning (100%), Tim Roberts (100%), Darrell Schiebel (100%), Paul Shannon (100%), Shelby Yang(100%), and Wes Young (90%). Ralph Marson will join the group in January 1996 at 100% AIPS++ activities. In addition, a number of scientists participate at various levels: Alan Bridle, Rick Fisher, Bob Hjellming, Harvey Liszt.

Appendix C: Review of developments promised for 1995 Q3

This section has been written to mirror the section in the last quarterly report on plans for 1995 Q3.

In Single Dish support, we did continue support of the GBT tests. A development plan for Single Dish targeted for mid November was delivered in early December.

In Synthesis support, we did continue development of the measurement equation for a general interferometer. We were able to make images, deconvolve and self-calibrate polarization measurements, but not for production. The initial goals for continuing development will be a Clean algorithm for simultaneous deconvolution of Stokes I,Q,U and V, and self-calibration of the antenna-based polarization leakage terms.

In AIPS++ Infrastructure, we continued but did not finish work on the tiled storage manager for the Table system (needed for efficient access of multidimensional data). We did finish design and start on implementation of classes for Task control and interaction. We did continue with the development of Coordinate classes now proceeding under the supervision of Wim Brouw. We started but did not finish work on improved linear algebra classes (needed for the Coordinate classes, and other areas).

In Visualization and Image Analysis, the collaboration between ATNF and NCSA has been slow to start. The ATNF visualization work will now be substantially delayed due to a re-allocation of staff.

In the System area, we are in the process of implementing shared libraries (thus reducing executable size). We did start to use by default the new Sun native code compiler (speeding compilations substantially), and implement a bug reporting system. We did not implement release versions. The move of the AIPS++ master code repository to Socorro has been deferred until January 1996.

In Documentation, we did resolve our staffing problem and start progress.

In Management, we did not closely evaluate the NFRA Software Engineering Group activities.