Back home

Notes on the challenge-2 generation pipeline

Prerequisites

  • A Unix platform (including OS X and Cygwin). However, note that Neil has provided Galaxy generation codes only for Linux i386 and OS X PowerPC (these work also on Linux 64-bit and OS X Intel).
  • GCC 3.3 or greater (note that 4.0.0 is a buggy release that breaks Synthetic LISA, but 4.0.1 is OK). If you don’t have it, you can get it from http://gcc.gnu.org; on Mac OS X, get the XCode tools from http://developer.apple.com/tools.
  • Subversion (SVN). Try “which svn”. If you don’t have it, you can get it at http://subversion.tigris.org (on Mac OS X, use the installer package found at http://www.codingmonkeys.de/mbo).
  • Python 2.4 or greater (if greater, all the python2.4 references below should be changed to the right version). If you don’t have it, get it from http://www.python.org.

A note on Cygwin

LISAtools works under Cygwin (at least, under version 1.5.24-2 on Windows XP). You must include the following additional packages in the Cygwin setup (I include the versions I tried, but anything newer would probably still work):

  • Python (2.4.3-1)
  • Subversion (1.4.2-1)
  • openssh (4.5p1-1)
  • SWIG (1.3.29-2)
  • gcc (3.4.4-3, this may not be needed since mingw is the standard compiler)
  • patchutils (0.2.31-1)
  • make (3.81-1).

Note that currently we don’t have Neil’s Galaxy generation code. Hopefully that will be forthcoming. Please e-mail vallis@caltech.edu with any problems compiling or running LISAtools under cygwin.

SVN installation and a quick SVN tutorial

You will need to identify a working directory on your workstation, on a disk large enough to contain all working files (probably a few gigabytes if you want to reconstruct the full Galaxy). Supposing this directory is /home/vallis, the subdirectory /home/vallis/lisatools will become a working copy of the Google Code SVN archive “lisatools”, while /home/vallis/lib, /home/vallis/bin (and so on) will become the installation directories for the lisatools libraries.

To get the SVN archive, cd into /home/vallis and run

svn checkout http://lisatools.googlecode.com/svn/ lisatools

“co” stands for “checkout”. This will recreate the SVN directory structure in /home/vallis/lisatools. Inside this directory, you can use the SVN commands described at http://svnbook.red-bean.com/nightly/en/index.html. The most important svn commands are:

  • “svn update”, which will update all files in the directory to the latest repository version; the command will print a list of updated files prefixed by “U” (or added files prefixed by “A”); if you see “G”, the changes in the repository version were successfully merged with the changes you had made to the file in the meantime; if you see “C”, the changes were incompatible, and more work is needed to harmonize the versions (see http://svnbook.red-bean.com/nightly/en/index.html)
  • “svn status -u”, which will show you which files have newer versions in the repository, and which you have modified locally (”M”); files prefixed by “?” are local files that are not in the repository (and perhaps should not be, such as .xml working files)
  • “svn commit -m “messagefilename“, which will commit your modified version of file filename to the repository, making a note of comment message
  • “svn add filename” or “svn add dirname“, to add a file or directory to the repository; the addition is not final until you issue a “svn commit” for the same filename or dirname
  • “svn del filename” or “svn del dirname“, to delete a file or directory; again, you need to follow this up with “svn commit”

More installation

If you’ve successfully checked out this archive from the googlecode SVN, you’re almost ready to run the master installer script in lisatools. Before you do that, you will need to decide where the compiled lisatools libraries should be installed (for instance, /usr/local, or /home/yourname), and set some environment variables. For instance,

export LISATOOLS=/home/yourname
export PYTHONPATH=$LISATOOLS/lib/python2.X/site-packages:$PYTHONPATH
export PATH=$LISATOOLS/bin:$PATH

on bash, or

setenv LISATOOLS /home/yourname
setenv PYTHONPATH $LISATOOLS/lib/python2.X/site-packages:$PYTHONPATH
setenv PATH $LISATOOLS/bin:$PATH

on tcsh. In these commands, replace “python2.X” with the version of Python on your system (just run “python” to see what you have; exit with CTRL-D). These paths will need to be set whenever you begin a lisatools session, so you can work the commands above into your .profile or .tcshrc.

Great! We’re all set. Cd into lisatools, and run

python master-install.py --prefix=$LISATOOLS

This will install:

  • numpy 1.0 (a Python array library)
  • pyRXP 1.11 (a Python XML library)
  • SWIG 1.3.31 (a C/Python interface generator)
  • Michele’s lisaxml (a lisaXML parsing library)
  • Stas’s BBH and EMRI signal generation packages (C++ codes with Python interfaces)
  • Neil’s Fast Galaxy code (C codes and precompiled executables)
  • Michele’s Synthetic LISA
  • Neil’s LISA Simulator (two separate versions, compiled for one-year and two-year datasets, will be installed in $LISATOOLS/lisasimulator-1year and $LISATOOLS/lisasimulator-2year respectively)

In fact, the master-install.py script will check if all these packages are already available in the system, and install them only if they are not. There are two exceptions, however:

  • SWIG will be installed if an older version is detected
  • the script will look for GSL (the GNU Scientific Library) under $LISATOOLS. If it does not find it, it will ask you to specify its location with a command line argument —-gsl=GSLDIR (for instance, —-gsl=/usr/local, if the GSL libraries are in /usr/local/lib). If you don’t have it or wish to reinstall it under $LISATOOLS, the script will install v1.8 if you give it the —-installgsl option.
  • the script will look for FFTW3 (the fastest Fourier transform in the west) under $LISATOOLS. If it does not find it, it will ask you to specify its location with a command line argument —-fftw=FFTWDIR (for instance, —-fftw=/usr/local, if the FFTW libraries are in /usr/local/lib). If you don’t have it or wish to reinstall it under $LISATOOLS, the script will download and install v3.1.2 if you give it the —-installfftw option.

You should be all set! If something went wrong, the script will complain in a way that suggests a solution. If not, you should complain to Michele.

Usage

Go to the directory lisatools/MLDCpipelines2. You will be working here. The primary script that you will be calling is bin/challenge2.py. It takes one argument and several possible options. The argument is the name of the challenge you want to build, for instance challenge2.1 or challenge2.2. For each such challenge, there must be a corresponding script (e.g., bin/challenge2.2.py), which has the task of choosing source parameters randomly for all the sources included in the challenge.

As for options, the only required option is the “master” seed for the random number generators used to make noise (option “-s” or “—-seed”). Other possible options are

options:
--version             show program's version number and exit
-h, --help            show this help message and exit
-t, --training        include source information in output file [off by
                      default]
-T DURATION, --duration=DURATION
                      total time for TDI observables (s) [default 62914560 =
                      2 years = 2^22 * 15 s]
-d TIMESTEP, --timeStep=TIMESTEP
                      timestep for TDI observable sampling (s) [default 15]
-s SEED, --seed=SEED  seed for random number generator (int) [required]
-n SEEDNOISE, --seedNoise=SEENOISE
                      noise-specific seed (int) [will use the global seed
                                                 (-s/--seed) if not given]
-m, --make            will try to use partial results of an interrupted run
                                                    [will not redo sources]
-S, --synthlisa       run only Synthetic LISA [by default both sims are in]
-L, --lisasim         run only the LISA Simulator [by default both sims are in]

Note that the LISA Simulator will be called only if the DURATION is one or two years (31457280 or 62914560).

Thus, if bin/challenge2.2.py does source parameter generation for the Challenge 2.2 sources, the challenge dataset will be built with both Synthetic LISA and the LISA Simulator by running (for instance)

bin/challenge2.py --seed=1234 challenge2.2

while to get a training set you’ll do

bin/challenge2.py --training --seed=1234 challenge2.2

Secondary scripts

  • The primary script relies on several secondary scripts
  • The most important are bin/challenge2.1.py, bin/challenge2.2.py, etc., which must take a single parameter (the seed, without any “-s” or “—-seed”), and place one or more lisaXML source parameter files in MLDCpipelines2/Source. Only one seed is passed, so this script has the responsibility of modifying to generate multiple sources.
  • makesource-BBH.py is called to generate a single-BBH source parameter file with randomized parameters. The specifics of the randomizing are hard-coded in the script. A similar script makesource-EMRI.py takes care of EMRIs.
usage: makesource-BBH.py [options] OUTPUT.xml
options:
--version             show program's version number and exit
-h, --help            show this help message and exit
-s SEED, --seed=SEED  seed for random number generator (int) [required]
-d D, --distance=D    distance to source (Pc) [required]
-t TC, --coalescTime=TC
                      time of coalescence from t = 0 (days) [required]
-v, --verbose         display parameter values [off by default]
  • makebarycentric.py is called to generate a single-source hp-hc strain file from a single-source parameter file
usage: makebarycentric.py [options] SOURCEDEF.xml BARYCENTRIC.xml
options:
--version             show program's version number and exit
-h, --help            show this help message and exit
-b PREBUFFER, --preBuffer=PREBUFFER
                      length of data computed before initial time (s)
                      [default 900.0]
-p POSTBUFFER, --postBuffer=POSTBUFFER
                      length of data computed after final time (s) [default
                      900.0]
-t INITTIME, --initialTime=INITTIME
                      initial time for waveform, not counting prebuffer (s)
                      [default 0.0]
-T DURATION, --duration=DURATION
                      total time for waveform, not counting pre- and
                      postbuffer (s) [default 31457280 = 2^21 * 15]
-d TIMESTEP, --timeStep=TIMESTEP
                      timestep for waveform sampling (s) [default 15]
-v, --verbose         display parameter values [off by default]
  • makeTDIsignal-synthlisa.py is called to generate single-source fractional-frequency TDI X,Y,Z. makeTDIsignal-lisasimulator.py, which takes the same parameters, does the same job for equivalent-strain observables
usage: makeTDIsignal-synthlisa.py [options] BARYCENTRIC.xml OUTPUT.xml
options:
--version             show program's version number and exit
-h, --help            show this help message and exit
-t INITTIME, --initialTime=INITTIME
                      initial time for TDI observables (s) [default 0.0]
-T DURATION, --duration=DURATION
                      total time for TDI observables (s) [default 31457280 =
                      2^21 * 15]
-d TIMESTEP, --timeStep=TIMESTEP
                      timestep for TDI observable sampling (s) [default 15]
-v, --verbose         display parameter values [off by default]
  • makeTDInoise-synthlisa.py generated MLDC-compliant noise, working from a seed. Same for makeTDInoise-lisasimulator.py
usage: makeTDInoise-synthlisa.py [options] OUTPUT.xml
options:
--version             show program's version number and exit
-h, --help            show this help message and exit
-t INITTIME, --initialTime=INITTIME
                      initial time for TDI observables (s) [default 0.0]
-T DURATION, --duration=DURATION
                      total time for TDI observables (s) [default 31457280 =
                      2^21 * 15]
-d TIMESTEP, --timeStep=TIMESTEP
                      timestep for TDI observable sampling (s) [default 15]
-s SEED, --seed=SEED  seed for random number generator (int) [required]
-v, --verbose         display parameter values [off by default]
  • mergeXML.py will merge two TDI files, keeping only the source parameter information (to make key files), or only the TDI observables, which are summed (to make challenge files), or both (to make training files)
usage: mergeXML.py [options] MERGED.xml TDIFILE-1.xml TDIFILE-2.xml ...
options:
--version      show program's version number and exit
-h, --help     show this help message and exit
-k, --keyOnly  output only SourceData information [off by default]
-n, --noKey    do not include SourceData information [off by default]
  • Two additional scripts will handle the creation of binary-background parameter files and of binary-background TDI files

Correction of the amplitude in accord with requested SNR

  • To adjust the distance according to the requested SNR we use the analytic fit in the “Noise + galaxy” spectrum:

Galaxy without noise: :listwg1b:galaxyxno_noise.png; Noise only :listwg1b:xyznoise.png; and all together: :listwg1b:galaxyxyznoise.png

Preliminary results

  • Here is the preliminary result :listwg1b:challenge2_2.png
 
/home/web/dokuwiki/data/pages/listwg1b/pipeline2.txt · Last modified: 2007/10/23 18:21 by vallis
 
Recent changes RSS feed Creative Commons License Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki