• Note, minor changes for production. Mostly using a new branch. *

We describe the steps to make neutrino interaction pictures from the very beginning. To keep things focused, this tutorial will cover how to make MC neutrino + EXT-unbiased overlay events.

We will first make 8 events, using test files. One should run the chain and confirm through the LArCV event display that things are working.

Once the images look good, we'll run a grid job capable of producing order of 10k events we need.

• replace occurrences of my username, tmw, with your FNAL username

First, determine the tagged version of uboonecode required.

Go the redmine repository browser here

Click on the view tab. In the source, find the version of uboonecode. Example below is as of 1/10/2017.

# This @product_deps@ file defines dependencies for this package. 
# The *parent* line must the first non-commented line and defines this product and version	
# The version must be of the form vxx_yy_zz (e.g. v01_02_03)	
parent uboonecode v06_26_01_10	
defaultqual e10	

Here, the version of uboonecode we will need to setup is v06_26_01_10. We also make note of the qualify e10. We will use this version and qualifier for what follows, but these tags can change.

Using the version and qualifier, follow these instructions. We summarize them here. Replace tmw with your username.

kinit
source /grid/fermiapp/products/uboone/setup_uboone.sh
cd /uboone/app/users/tmw
mkdir develop_dl
cd develop_dl
setup uboonecode v06_26_01_10 -q e10:prof
mrb newDev
source localProducts_larsoft_v06_26_01_09_e10_prof/setup
cd srcs
mrb g uboonecode
cd uboonecode
git checkout DL_br
mrbsetenv
mrb i -j4

To re-setup a new shell to use this code

source /grid/fermiapp/products/uboone/setup_uboone.sh
cd [where your code lives]
source localProducts_larsoft_v06_26_01_09_e10_prof/setup
mrbsetenv

The default fcl file is in uboonecode/fcl/ana and is standard_larcv_uboone_data.fcl. When in doubt use this.

Other options for making larcv (and larlite) files needed for the downstream DL analysis, run the fcl files in uboonecode/fcl/deeplearn

Note that, currently, the downstream analysis requires a larcv file and a opreco larlite file. The latter stores vital trigger information.

These should be in the FHICL_SEARCH_PATH and can be run our of the box via

lar -c [fcl above] -s [input larsoft file]

If one just wants to make larcv (and larlite) MC files needed for the downstream DL analysis, run the fcl files in uboonecode/fcl/deeplearn

Note that, currently, the downstream analysis requires a larcv file and a opreco larlite file. The latter stores vital trigger information. For full MC files, we also save truth information used to study the output of the analysis. The default is standard_supera_mc_wpmtprecuts.fcl. For slim MC files, use standardslim_supera_mcwire_wpmtprecuts.fcl and standardslim_supera_mctruth_wpmtprecuts.fcl.

For overlays, one wants to make cosmic images that pass the precuts and neutrino images that pass the precuts.

For data, use standard_supera_data_wpmtprecuts_noreco2d.fcl. For MC, use standard_supera_mc_wpmtprecuts_noreco2d.fcl

BELOW IS LIKELY OUTDATED

These fcl files are for generating events from scratch and then processing the output to larcv and larlite files.

[More to come]

It's often good to check that things are still working on a small sample before running a grid job.

Copy the example fcl files, those in example_drivers, to some working directory. You can also run the program in the example_drivers folder.

To run the cosmic, use the following command which will process an example larsot cosmic ray file

lar -c standard_supera_data_wpmtprecuts.fcl -s /uboone/data/users/tmw/dl_test_files/testfile_mcc8_r6307_extunbiased.root

Run

lar -c dlmc_bnb_nu_larlite_supera.fcl -n N

On your laptop, you should build the DL LEE code, if you haven't done so already. For instructions on this go here.

If you have, go to the dllee_unified/LArCV folder, and copy the larcv_[data/mc]_[timestamp].root files you made in the previous steps. To run the viewer type

python mac/view_rgb.py [your_larcv_file.root]

Note, remember to run configure.sh in your dllee_unified folder to setup the proper environment variables in your shell.

Another note: you can run this viewer from any folder in your file system, just use the full path

python $LARCV_BASEDIR/mac/view_rgb.py [your_larcv_file.root]

This also requires a copy of the dllee_unified repo. Go to the LArCV, again. Then go to

production/v06

You can run the merger program by

python merger.py simple_merger.cfg [larcv_mc_file.root] [larcv_data_file.root]

Now, before you run, we have to change/check the configuration file, simpler_merger.cfg.

We use the configuration file to tell the program what the name of the TTree objects inside the file the program should use.

The configuration file is in the fihcl format, a format invented by Fermilab and implemented in larcv/larlite by Kazu. It consists as of a nested set of parameter blocks or sets with each block having a name. The blocks are marked with {}.

InputStream2: {
  Verbosity:    2
  EnableFilter: true
  RandomAccess: false
  ProcessType:  ["DataStream"]
  ProcessName:  ["CosmicImage"]
  ...
  ProcessList: {
    CosmicImage: {
      Verbosity:        2
      Profile:          true
      TPCImageProducer: "tpc"
      PMTImageProducer: "pmt"
      ChStatusProducer: "tpc"
      ROIProducer : ""
      SegmentImageProducer : ""
    }
  }
 }

This the parameter set for Input stream 2, which is the second file in the argument list and is meant for the cosmic file. Inside the block there is the ProcessList, this holds configuration parameter sets for the operations performed on the image. For CosmicImage, this process is responsible for reading in the data. What we need to do is tell it the name of the TTree's in the cosmic file.

What should these names be? We can find it by looking inside the file. Do this by

twongjirad@tmw-Blade:~/working/larbys/dllee_unified/LArCV/production/v06$ root -l  larcv_data_20170608_130328_703691.root 
root [0] 
Attaching file larcv_data_20170608_130328_703691.root as _file0...
(TFile *) 0x55780623b340
root [1] .ls
TFile**		larcv_data_20170608_130328_703691.root	
 TFile*		larcv_data_20170608_130328_703691.root	
  KEY: TTree	image2d_wire_tree;1	wire tree
  KEY: TTree	chstatus_wire_tree;1	wire tree
  KEY: TTree	image2d_pmt_tree;1	pmt tree
root [2] 

We want to load the images, whose trees are prefixed by image2d. Note there are two such trees. One tree holds the PMT images, the other holds an image of signals from the wire plane. This tree is labeled with wire.

So we want to go back to the fcl file and set the input TPC image tree name to wire.

InputStream2: {
  Verbosity:    2
  EnableFilter: true
  RandomAccess: false
  ProcessType:  ["DataStream"]
  ProcessName:  ["CosmicImage"]
  ...
  ProcessList: {
    CosmicImage: {
      Verbosity:        2
      Profile:          true
      TPCImageProducer: "wire"
      PMTImageProducer: "pmt"
      ChStatusProducer: "wire"
      ROIProducer : ""
      SegmentImageProducer : ""
    }
  }
 }

You'll want to do the same with InputStream1, which is for the MC data.

Now you're ready to run the merger. Give it a try, and check the output image in the LArCV display.

Running on the grid at Fermilab involves configuring the lar program (which you ran above) to work on the worker nodes of the grid. One configures and launches such jobs using the project.py script.

In /uboone/app/users/tmw/develop_dl/srcs/uboonecode/uboonde/LArCVImageMaker/example_drivers/ you'll find two files

• project_supera_extunbiased.xml
• project_supera_bnb_nu_foroverlay.xml

These configure the job for data and the MC respectively.

The file is basically read, except you need to customize it. Please change the following parameters:

Our modified code will be compressed in this tarball. The workers that complete the job on the grid have restricted access to things like the network or what network hard drives it can read or write to. So, the typical thing is to package your code and ship it over to the cluster and to each of the nodes. When the worker node runs, it will unpack your build of the code and run it.

To make this, go to the base folder of your larsoft code folder. (This is where you mrb newDev.)

Then type

make_uboone_tar.sh larsoft.tar

Finally, we need to make sure that output folder exists

mkdir -p path_to_output_dir

Note, if you haven't done so in a day, reset your grid credentials using the setup_grid_proxy.sh I can provide you -- or run the following

setup jobsub_client
kinit [username]
kx509
voms-proxy-init -noregen -rfc -voms fermilab:/fermilab/uboone/Role=Analysis

Finally,

project.py --xml project_supera_extunbiased.xml --stage supera --submit

After a few seconds you can type the following command to check the status of your job

jobsub_q --user [username]

You can keep checking the status in this way. Once the job is done, it will no longer appear in the list.

Again, you can check the output by downloading it to your computer and viewing it. It's going to be in the output directory you specified in the xml file. The files will be in the sub-folder

[output dir]/out/[jobid number]/

If you don't see them, something went wrong. You can look at the error in the log folder

[output dir]/log/[jobid number]/larStage0.err

Setting this up is basically the same as above. The only difference is that

<numevents>1000</numevents>

Matters. For the data, the number of events is dictated by the number of files you process. For the MC, since we are generating events, this number is actually relevant along with the <numjobs> parameter.

The number of events per job will be numevents/numjobs. You want to set this for a max of 100 or so.

Once the jobs are complete, project.py can check them for you. Run

project.py --xml [xml config file] --stage supera --checkana

Sometimes job don't finish properly. You can rerun after checking them using --checkana by

project.py --xml [xml config file] --stage supera --makeup

project.py --xml [xml config file] --stage supera --merge

This is more convenient for running on your laptop. But warning, this is going to be a very large file, depending on the number of events. You are better off running the merger on the FNAL grid.

coming soon