This exercise was created for the 2011 CERN CERN School of Computing, hosted by the Niels Bohr Institute, University of Copenhagen.

Credits: physics content of this exercise by Jørgen Beck Hansen from the Niels Bohr Institute.

LHC Monte Carlo event generation and analysis

Assignment

Generate 500’000 LHC t-tbar events with PYTHIA and analyze the data: find bosons and top hadrons, run jet-­?algorithm: cone R<0.7

Prerequisites

• Download and install VirtualBox.
• Download and fire up a CernVM. It’s probably easiest to use the installer.
• Configure your CernVM to use the repositories hepsoft, grid and sft and setup a user account.
• Login on your new CernVM.
• Setup GridFactory:

  export PATH=$PATH:
  /cvmfs/sft.cern.ch/lcg/external/experimental/gridfactory.org/gridfactory_ui:
  /cvmfs/sft.cern.ch/lcg/external/experimental/gridfactory.org/gridworker:
  /cvmfs/sft.cern.ch/lcg/external/Java/JDK/1.6.0/ia32/bin
  

• Start a gridworker:
  
  in a separate shell:

  gridworker.sh [-n] gridfactory.nbi.dk/db/
  

  Hint: If you have X enabled, you can run in graphical mode by leaving out the -n

• Setup Root:
  
  In a 32-bit VM:

  source /cvmfs/sft.cern.ch/lcg/external/gcc/4.3.2/i686-slc5-gcc43-opt/setup.sh
  export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:
  /cvmfs/sft.cern.ch/lcg/external/ROOT/5.28.00/slc4_ia32_gcc34/root/lib
  export PATH=$PATH:
  /cvmfs/sft.cern.ch/lcg/external/ROOT/5.28.00/slc4_ia32_gcc34/root/bin
  

  In a 64-bit VM:

  source /cvmfs/sft.cern.ch/lcg/external/gcc/4.3.2/x86_64-slc5-gcc43-opt/setup.sh
  export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:
  /cvmfs/sft.cern.ch/lcg/external/ROOT/5.28.00/x86_64-slc5-gcc43-opt/root/lib
  export PATH=$PATH:
  /cvmfs/sft.cern.ch/lcg/external/ROOT/5.28.00/x86_64-slc5-gcc43-opt/root/bin
  

• Get and unpack the exercise tarball TTBarExercise.tar.gz.

Help

Compile PYTHIA + analysis code

cd TTBarExercise/siscone-2.0.0 
      
./configure --prefix=$PWD/../siscone
make clean     
make
make install       
cd ..

rm -f *.o
make

g77 -o pythia pythia_exercise.f pythia-6.4.25.f

Tests

• Run PYTHIA: ./pythia
• Analyze PYTHIA output and save result to root histogram: ./AsciiReader Eventsgen.ascii

Create batch jobs

In a 32-bit VM:

mkdir jobs
for n in {1..25}; do
cat > jobs/job$n.sh <<EOF
#!/bin/bash
source /cvmfs/sft.cern.ch/lcg/external/gcc/4.3.2/i686-slc5-gcc43-opt/setup.sh
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:
/cvmfs/sft.cern.ch/lcg/external/ROOT/5.28.00/slc4_ia32_gcc34/root/lib
export PATH=$PATH:
/cvmfs/sft.cern.ch/lcg/external/ROOT/5.28.00/slc4_ia32_gcc34/root/bin
./pythia $((19780503 + $n)) 20000
./AsciiReader Eventsgen.ascii
mv AsciiReader.root  ttbar_analysis_$n.root
EOF
sed -i "s|$n|$n|" jobs/job$n.sh
done

In a 64-bit VM:

mkdir jobs
for n in {1..25}; do
cat > jobs/job$n.sh <<EOF
#!/bin/bash
source /cvmfs/sft.cern.ch/lcg/external/gcc/4.3.2/x86_64-slc5-gcc43-opt/setup.sh
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:
/cvmfs/sft.cern.ch/lcg/external/ROOT/5.28.00/x86_64-slc5-gcc43-opt/root/lib
export PATH=$PATH:
/cvmfs/sft.cern.ch/lcg/external/ROOT/5.28.00/x86_64-slc5-gcc43-opt/root/bin
./pythia $((19780503 + $n)) 20000
./AsciiReader Eventsgen.ascii
mv AsciiReader.root  ttbar_analysis_$n.root
EOF
sed -i "s|$n|$n|" jobs/job$n.sh
done

Submit batch jobs

First check that GridFactory is setup correctly:

psub -h

If all is well, continue with submitting all the jobs:

rm jobs.txt
for n in {1..25}; do
 psub -b gridfactory.nbi.dk jobs/job$n.sh -i pythia -i AsciiReader -e pythia -e AsciiReader -o ttbar_analysis_$n.root | grep -v submitted >> jobs.txt
 echo "submitted job $n"
done

Hint: if you have problems with the above, try running a single job:

psub -b gridfactory.nbi.dk jobs/job1.sh -i pythia -i AsciiReader -e pythia -e AsciiReader -o ttbar_analysis_1.root

Monitor jobs

pstat `cat jobs.txt`

Get and merge output histograms

mkdir results
for n in `cat jobs.txt`; do
 dir=`echo $n | awk -F / '{print $NF}'`
 mkdir results/$dir
 pget -o results/$dir $n
done

hadd ttbar_analysis.root results/*/*.root

Open the final histogram with Root

root ttbar-analysis.root
.
.
.
root [1] TBrowser b;

Clean up

pclean `cat jobs.txt`
rm -rf jobs results