This page has been superseded.

Please visit Colm Talbot's Introduction to LALInference page instead. That page is loosely based on material found here, but it is better organised and easier to read.

Getting Started with LAL

LAL is C-based software used for a lot of work in the LIGO CBC group. Thanks to Rory Smith for assistance putting this together! For additional information, check out the official LALInference_HowTo.

Running your first lalinference batch

install lalsuite: Use JHBuild following these instructions from Leo Singer (recommended by Rory Smith). If you follow the old instructions below, there are subtleties in compilation that require expert help to fix. The instructions will create a directory called modulesets used in installation, but the lalinference source code will live in ~/src. To start a shell with your newly built packages in the environment, run: jhbuild shell

How do I reinstall lalsuite? Something has gone terribly wrong, e.g., generateIMRPhenomPv2 gives you a different result every time you run it! It's time to rebuild lalsuite. Follow these instructions.

Do I have lalinference?
-bash-4.2$ which lalinference_nest
~/repositories/tmp/opt/lalsuite/bin/lalinference_nest --help

getting started with lalinference:
While lalinference_nest does the actual exploration of parameter space, lalinference_pipe is the code you want to run to get started. It handles pre-processing (locating data and setting up dags), processing (running lalinference_nest), and post-processing (calculating posteriors from nested samples). Rory: I highly recommend _nest over _mcmc as it is ~100x faster.

Once lalinference is installed, the next thing you need is to get an ini file. An ini file is analagous to a stochasic pipeline paramfile. It is human readable and parsed by LAL code. It specifies many things like maximum chirp mass, maximum signal length, the location that lalinference output will be written to, the "engine" (MCMC versus multinest), required memory, GPS start time, strain channel names, which detectors to analyse, whether to use real/simulated data, etc. There are many hard links that need to be replaced with links to your local directory. Of particular interest are the variables baseurl and webdir, which is where your output plots will be created. Note that you don't have to mkdir webdir; it will be made for you if it doesn't already exist.

Here is an example of a working ini file. I copied it from /home/rory.smith/projects/offLine/4sBBHroqpaper/HLVIK/test.ini. As you look through the ini file, you will see that it is divided into sections such as [paths] and [condor]. Rory: once you have a working ini file (probably sent to you from a collaborator), you can modify it to make new ini files to do different tasks.

The next think you need is an xml file to describe the injections you wish to perform. Here is a copy of an xml file. If you want to see what is contained inside the xml file (which is not human readable), you can use the following command: lwtprint -t sim_inspiral -c mchirp ROQ_test_injections_4s_IMRPhenomPv2_smaller_v7_cSL_only_actual_inj.xml. This particular xml file contains 89 injections.

Now you're ready to run _pipe. Here's the syntax:

lalinference_pipe -r ~/rory/run -p /usr1/ethrane/ -I ~/rory/ROQ_test_injections_4s_IMRPhenomPv2_smaller_v7_cSL_only_actual_inj.xml ~/rory/eric.ini

This will execute lalinference according to the parameters in eric.ini, using the injections in the xml file provided, dumping the results in run/, and dag logging in /usr1/ethrane/. Stuart: all dag logging should be sent to /usr1/. You can run lalinference_pipe --help for more options.

This example runs with 21 injections, creating posteriors for each one. However, it is also possible to run without any injections (just looking at an actual event. While there are 89 injections in the xml file, we only carry out 21 because of this line in the ini file: events = [57:77]. So, just injections 57 through 77. For each injection you run on, you get a set of posterior PDFs on its own output page (in your case, 21 different output pages each corresponding to a unique injection in the xml table). Each injection has a separate .err and .out file in the log directory. The number 17587809 in the filename lalinference-17587809-0-.err refers to the condor job number.

Once you run _pipe, you should poke around your run/ directory to make sure everything was correctly assemebled. In my case, I see run/lalinferencenest/IMRPhenomPv2pseudoFourPN/, which contains the important directories engine/ (where the nest chains are stored in hdf5 files) and log/ (where you can chart the progress of your chains).

Now it's time to run condor. The _pipe wrapper will tell you to submit run/multidag.dag. However, Rory suggests this instead:
cd run/lalinferencenest/IMRPhenomPv2pseudoFourPN
condor_submit_dag lalinference_966394376-971437662.dag

Now, monitor the progress of your condor submissions. In addition to using condor_q to make sure they're running, look at your new log files:
cd /home/ethrane/rory/run/lalinferencenest/IMRPhenomPv2pseudoFourPN/log
tail -f lalinference-17587817-0-.err
This file wcontains a number of variables, but there are only a couple that are interesting to track. This is what the .out file for a coherent recovery looks like

injected SNR 82.5 in IFO V1
injected SNR 70.8 in IFO H1
injected SNR 31.9 in IFO L1
injected Network SNR 113.3

Once the job has finished, check the summary pages to check out the results, e.g., example. Check the sky map and reconstructed masses to see if they're close to the injected value. Currently, posterior PDF plots are not generated...debugging in progress. The post-processing command to generate the posteriors is given on the bottom of the page, e.g.,

/home/ethrane/local/bin/cbcBayesPostProc.py 
--inj /home/ethrane/rory/ROQ_test_injections_4s_IMRPhenomPv2_smaller_v7_cSL_only_actual_inj.xml 
--snr /home/ethrane/rory/run/lalinferencenest/IMRPhenomPv2pseudoFourPN/engine/lalinferencenest-58-V1H1L1-966842498.125-1.hdf5_snr.txt 
--skyres 0.5 
--outpath /home/ethrane/public_html/files/rory/offLine/4sBBHroqpaper/HLV/lalinferencenest/IMRPhenomPv2pseudoFourPN/966842498.125312328-58/V1H1L1 /home/ethrane/rory/run/lalinferencenest/IMRPhenomPv2pseudoFourPN/posterior_samples/posterior_V1H1L1_966842498.125312328-58.hdf5 
--psdfiles /home/ethrane/rory/run/lalinferencenest/IMRPhenomPv2pseudoFourPN/engine/lalinferencenest-58-V1H1L1-966842498.125-1.hdf5V1-PSD.dat,/home/ethrane/rory/run/lalinferencenest/IMRPhenomPv2pseudoFourPN/engine/lalinferencenest-58-V1H1L1-966842498.125-1.hdf5H1-PSD.dat,/home/ethrane/rory/run/lalinferencenest/IMRPhenomPv2pseudoFourPN/engine/lalinferencenest-58-V1H1L1-966842498.125-1.hdf5L1-PSD.dat 
--eventnum 58

Note: this command has to be run after: jhbuild shell.

To make your own 1D posteriors by hand, open the posterior_samples.dat. Each column is a list of samples for a different variable. Tto make your own PDF, just histogram the list of sampling points for the variable of interest.

The variable delalogLmax is approximately equal to matched filter SNR^2/2. If you are looking at reasonable injections, this should have a maximum value of about 5000. Values much larger than that are usually indications of problems, e.g., templates getting rung up from a glitch. The variable dZ is the convergence of the evidence integral. The _nest code is set up to run until this becomes less than unity. Note that dZ does not necessarily fall linearly. Running without ROM, it can take >8 days to run the exmaple script. The variable Zratio is the (log of the) evidence ratio (Bayes factor). Large positive values of Zratio indicate that the signal model is preferred to Gaussian noise. Initially, Zratio usually takes on (large) negative values, e.g., -3745.068, because Gaussian noise is usually a much better fit than some random point in parameter space. However, as the program runs, Zratio should trend toward increasingly positive numbers.

If you want to compare two different models (e.g., no-memory versus memory), you can run lalinference twice, for two different waveform families, and then take the difference between Zratio. Lalinference has some automatic model selection features comparing coherent versus incoherent evidences, though, I haven't played around with these.

In order to do parameter estimation, the post-processing code harvests information stored in hdf5 files, which are sort of pseudo-directories containing nested data structures. The hdf5 files are not created until after _nest is complete so you cannot watch them evolve in realtime. Where does this hdf5 file live? Explain that you can puruse this hdf5 file with h5ls. In order to see the parameters use verbose: h5ls -v (stuff)/nested_samples/. Use data flag to see the nested samples: h5ls -d (stuff)/nested_samples/time. Nested samples (list of parameter values with ever-increasing likelihood) are not posteriors, but they can be converted into posteriors using lalapps_nest2pos. Other relevant code for post-processing is cbcBayesPostProc.py.

How long will this take to run? Expected wait time: a couple days if ROM is turned off. Update: actually, the code has been running now eight days with 3/21 jobs still running.

How do I speed things up? One option is to narrow the prior range. Making the signal louder doesn't actually help _nest to converge faster, though, it may help _mcmc. ROM can speed things up by a factor of ~100.

How do I run on real data with ROM (Reduced Order Modelling) and RQM (Reduced Order Quadrature)?

Borrow online_pe.ini from Rory:
cp ~rory.smith/projects/tests/EricThranePipelineTest/online_pe.ini .
Compare to the other eric.ini file above to note the differences. Some of the key differences: the coherence test is on, pointing at real data, new channels and types, no Virgo, DMT flags for science mode, no injections.

roq_b_matrix_directory = /home/cbc/ROQ_data/IMRPhenomPv2/
computeroqweights = /home/ethrane/local/bin/lalapps_compute_roq_weights

To run:

jhbuild shell
lalinference_pipe -r ~/rory/online_pe --gid G211117 --daglog-path /usr1/ethrane/ online_pe.ini
condor_submit_dag /home/ethrane/rory/online_pe/multidag.dag

How do I run on a list of trigger times?

Replace the -I file.xml flag with -g TIMES.txt where TIMES.txt is a list of GPS trigger times. This section is under construction: an example ini file, times.txt file, and frame file are on the way.

More advanced tips

How do I read/edit the xml file? try ligolw

turning on/off spin: spin is automatically included if you use a waveform family for which spin is defined. However, it can be turned off by adding a command to the ini file...not sure what that command is yet.

How do I add memory to lalinference? The easiest way to add memory is to create a new approximant that includes memory. This would involve almost no hacking of lalinference, just the creation of a new waveform family. Aside: if we do this for IMRPhenomP_v2, Rory can create a Reduced Order Model (ROM), which will run 100x faster than if we don't. Aside: other expert: Scott Field (UMass Dartmouth). By comparing Zratio for IMRPhenomP_v2 with and without memory, it should be possible to calcualte the memory Bayes factor.

How do I add totally new parameters? This is probably harder than adding new approximants. Check out how the non-GR (e.g., dchi4) parameters are added, e.g., on L1973 of lalinferenceInitCBC.c. Then, you put something in your ini requesting that these non-standard parameters are included. See also: LALInferenceReadData.

option of turning on coherence test (not on by default)
in your ini file: coherence-test = True
Given 21 injections, there will be 21 x 3 = 63 jobs: one coherent analysis and two incoherent analyses. The coherent takes only slightly longer than the incoherent. Almost always do coherent AND incoherent in order to checkR = BF coherent / (BF incoherent H + BF incoherent L) >> 1 for real signals versus different R values for glitches.

Using faster Broadwell processors (rather than Sandy Bridge).
emacs /home/ethrane/rory/run_RealDataYesROM-ROQ/lalinferencenest/IMRPhenomPv2pseudoFourPN/8s/*.sub
add this command after "getenv = true":
Requirements = regexp("E5-2650 v4", TARGET.cpuinfo_model_name)
Not yet set up to be added at the ini file...to be added.
When one specifies masses in the xml injection file, are these in the observer frame or the source frame?
They are in the observer frame.

How do I make a plot of pure signal?
Lalinference outputs results for noise-only and signal+noise. You can work out the pure-signal results by subtracting these two. The whitened data that lalinference outputs lives in locations like this one: ~ethrane/rory/run_RealDataYesROM-ROQ/lalinferencenest/IMRPhenomPv2pseudoFourPN/8s/ROQdata/211117/data-dumpL1-timeData.dat

Where can I find the maximum likelihood parameters?
Check out the summary statistics.

Do not run with three detectors.
this can cause problems with interpolation to calculate the tc time. alternatively, if you have to use three detectors, you can try using - - no-detector-frame

Errors with pragmas openmp
I think the issue is coming from the openMP pragmas in the waveform generator which hasn't been fixed on the master branch of lalsuite but is fixed on the production-run branch.Could you try sourcing the official O2 build with ~cbc/pe/lalinference_o2.sh and change your binaries to the official O2 ones. They should all be in /home/cbc/pe/local/bin/ Assuming that your xml table is fine, I'd bet that this will fix the strange output you got. PUNCHLINE: source the production-run branch. Then you don't even need to muck with jhbuild.

Caution: ROM/RQM are constructed using some set of priors. If you attempt to change the priors, but use ROM/RQM, they will simply override your chosen priors.

Relating python to C code in lal:

In python, the XLAL bit of the function name is removed, so it would be
import lalsimulation
lalsimulation.SimInspiralTransformPrecessingNewInitialConditions

Spin conventions / email about the different variable names for lalinference: rtf file

Getting help

for help on PE problems use "Mattermost"
http://chat.ligo.org
channels > more > join lalinference-devel, lalinference-help, parameter estimation
posting a code snippet:
```
code snippet here
```

Frequently (and not-so-frequently) asked questions

What assumptions does lalinference make about the coalescence time?
By default, lalinference searches ±0.1 s around the trigger time.

I am getting different waveforms every time I run lalsim_IMRPhenomPv2 with identical parameters. Add the following line to your script (there's an openmp bug):
export OMP_NUM_THREADS=1

Does lalinference calculate the evidence for the null hypothesis (no signal present)? Yes. It is trivially calculated as log(Z0) = -1/2 (h, h).

My condor jobs have died. What do I do to fix the following error in my .err file?

  ==> log/lalinference-88739631-0-.err <==
  condor_exec.exe: error while loading shared libraries: liblal.so.12: cannot open shared object file: No such file or directory

Solution

  I've observed this before and I was able to fix it by cleaning the build and then rebuilding: 
  jhbuild cleanone $(jhbuild list lalsuite) && jhbuild uninstall $(jhbuild list lalsuite) && jhbuild build lalsuite

Debugging with the command line:
/home/pablo.rosado/projects/tbs/lalinference/online_pe/lalinferencenest/IMRPhenomPv2pseudoFourPN/8s/lalinference_1135136072-1135136369.sh

Debugging with gdb: IN PROGRESS

Why is my condor submission finishing immediately?
The reason that the jobs are failing is that the rescue dag files are still in run directory which link to job IDs which are overwritten when you rerun the lalinference pipe command. Try removing the online_pe/lalinferencenest directory, rerunning lalinference_pipe and then resubmitting the dag.

Pablo's tips: pablo.txt

obsolete instructions

Back to Resources