At the December 2005 LISA International Science Team meeting in Pasadena the Data Analysis Working Group was more formally constituted, with co-chairs Neil Cornish and Bernard Schutz. The working group has, among other things, the task of organizing mock data challenges for the LISA data analysis development efforts.

The mock data challenges are the principal means we will have during the next couple of years to validate proposed algorithms and ensure that they can indeed meet the requirements of the LISA data analysis system. We envision that there will be a number of such challenges, starting with relatively simple ones and culminating (hopefully by the end of 2007) in a fairly realistic one. Initially there could be separate challenges for different kinds of sources, but the final challenge should test algorithms on a data set that contains at least the minimum mission goals of galactic binaries and supermassive black hole mergers

To this end we are setting up a Mock Data Challenge Task Group under the following charter:

(1) Decide on a minimum set of physical conventions for the mock data sets (e.g. coordinate systems, LISA orbital parameters, definitions of barycenter h+ and hx, choice of TDI variables).

(2) Identify a minimum set of file descriptors that will appear in the headers of the input barycenter waveforms and the simulated mock data streams (e.g. time stamp, data cadence, file length, orbital parameters, noise spectral densities, TDI channel labels, source class, source parameters)

(3) Decide on a lightweight, flexible, extensible and well supported data format for the input waveforms and challenge data sets (e.g. FITS, XML, HDF)

(4) Develop a plan for conducting a series of progressively more difficult mock data challenges. (e.g. should the initial data challenges be restricted to a limited bandwidth, what source types should be included in each challenge, whether full TDI data will be used or some kind of synthetic Michelson streams, what is expected to be found: detections; parameter recovery etc)

(5) Distribute training data sets with similar content to the various challenge data sets.

(6) Decide on the metrics that will be used to measure the relative performance of different algorithms.

(7) Administer the mock data challenges. This will consist of announcing (well in advance) the general characteristics of the mock data set to be generated; generating one or more random instances of such a set and ensuring that the “contestants” do not see them before the challenge begins (so that the challenge is blind); evaluating the results of applying the algorithms to the data set(s)

These are meant to be blind tests, but not really a contest. We are supposed to learn from the tests how to improve each prototype system. So the construction, administration, and evaluation of the challenge must be open. The only part of the process that is kept hidden is the actual instance of the data being used in the challenge. We anticipate that tasks (1) - (6) will be handled by the full task force, while the actual administration of the challenges will be handled by a sub-group made up of people who will not be entering the challenges. Our goal is to have tasks (1) - (3) completed by early April 2006, and tasks (4) and (5) completed by June 2006.

The task group will be lead by Michele Vallisneri and Alberto Vecchio, with John Baker serving as coordinating secretary in charge of setting up telecons, sending out e-mails and circulating drafts of the planning documents. Confirmed members of the task group as of January 15th, 2006 are: Keith Arnaud, Stanislav Babak, Matt Benacquista, Joan Centrella, Neil Cornish, Curt Cutler, Shane Larson and Jean-Yves Vinet. There is a great deal of work to be done in a very short amount of time, so we encourage anyone who is interesting in committing a significant work effort to sign up for the task force. The first monthly telecon will be held in the first week of February 2006. Draft documents concerning the physical conventions and data descriptors have been prepared by the Mock LISA Data Archive [http://astrogravs.gsfc.nasa.gov/docs/mlda/] steering committee (effectively the predecessor to the new task force), and will be distributed to all task force members for prior to the first telecon.

With Best Regards, Neil Cornish & Bernard Schutz