LArLitify for deep learning - twongjirad/LArLiteSoftCookBook GitHub Wiki

This is a special case for lartifying. Some first calibration adjustments are made to the wires. We also produce specific data products.

The fcl files to do this are on a special branch. So first, one needs to setup a uboonecode environment and checkout the special DL branch.

As of the time of this writing, checkout the correct base uboonecode version, go to a folder where you want your code to live -- probably somewhere under /uboone/app/users/[your_user_name].

source /grid/fermiapp/products/uboone/setup_uboone.sh
mkdir v06_11_00_dl
cd v06_11_00_dl
setup uboonecode v06_11_00 -q e10:prof
mrb newDev
source localProducts_larsoft_v06_11_00_e10_prof/setup
cd srcs
mrb g uboonecode

Note that v06_11_00_dl is just the name of the folder containing your copy of the code. You can name it whatever you like. Once the environment is setup, and you've grabbed the code, get the official DL branch

cd uboonecode
git checkout -b git checkout -b [username]_v06_11_00_dl_v01 origin/v06_11_00_dl_v01

Note that [username]_v06_11_00_dl_v01 can be anything you want. This is just one type of convention.

Finally, set the environment:

mrbsetenv

You should see no errors. It'll look like this:

> mrbsetenv
> The working build directory is /uboone/app/users/tmw/dev/v06_11_00_dl/build_slf6.x86_64
> The source code directory is /uboone/app/users/tmw/dev/v06_11_00_dl/srcs
> ----------- check this block for errors -----------------------
> ----------------------------------------------------------------

And then build the code.

mrb i -j4

Note that you will probably see errors when you run mrbsetenv. See the 'tweak' section on how to properly deviate from the standard procedure and fix the slight problem.

Then setup this code to be able to larlitify

Then use the fcl files in uboonecode/fcl/deeplearn

For example,

 lar -c data_to_larlite.fcl -s [source larsoft file]

Tweak needed

If you setup the code as described above and try to build, you might see the following error

----------- check this block for errors -----------------------
ERROR: Version conflict -- dependency tree requires versions conflicting with current setup of product ubutil: version v01_39_01 vs v01_39_05
ERROR: setup -B ubutil v01_39_01 -q +e9:+prof failed
ERROR: For more information, type "ups depend ubutil v01_39_01 -q +e9:+prof"
                             or "ups list -aK+ ubutil v01_39_01"
ERROR: setup of required products has failed

It seems that the base uboonecode version (v05_08_00_02) needs a newer version of ubutil than specified. You can tweek the setup by editing uboonecode/ups/product_deps

Look for

...
ubutil v01_39_01
...

and change it to

...
ubutil v01_39_05
....

then re-setup the environment with

mrbsetenv

build with

mrb i -j4

Running on the grid

See this.

Standard production grid XML files

There are a number of example xml files for different types of data or MC files. You can use this as the basis of production. For core samples to be shared with others (EXTBNB, EXT-Unbiased, MC BNB, etc.), only minimally change these xml files as they've been vetted to some degree. Only change the user name and paths as required.

A partial list is here:

fcl_deeplearn/dl_production_v00/data_bnb_to_larlite_v00_p00.xml
fcl_deeplearn/dl_production_v00/data_extbnb_to_larlite_v00_p00.xml
fcl_deeplearn/dl_production_v00/data_extunbiased_to_larlite_v00_p00.xml
fcl_deeplearn/dl_production_v00/mcc7_bnb_cosmic_detsim_to_larlite_v00_p00.xml
fcl_deeplearn/dl_production_v00/mcc7_bnb_cosmic_detsim_to_larlite_v00_p01.xml
...
fcl_deeplearn/dl_production_v00/mcc7_bnb_detsim_to_larlite_v00_p00.xml
fcl_deeplearn/dl_production_v00/mcc7_bnb_detsim_to_larlite_v00_p01.xml
...
fcl_deeplearn/dl_production_v00/mcc7_cosmic_extbnb_detsim_to_larlite_v00_p00.xml
fcl_deeplearn/dl_production_v00/mcc7_cosmic_extbnb_detsim_to_larlite_v00_p01.xml
...
fcl_deeplearn/dl_production_v00/data_bnb_to_larlite_v00_p01.xml
fcl_deeplearn/dl_production_v00/data_bnb_to_larlite_v00_p02.xml
...
fcl_deeplearn/dl_production_v00/mcsinglep_eminus_v00_p00.xml
fcl_deeplearn/dl_production_v00/mcsinglep_eminus_v00_test.xml
fcl_deeplearn/dl_production_v00/mc_bnb_nue_appearance_larlite.xml
fcl_deeplearn/dl_production_v00/mcmultip_muminus_v00_test.xml
...

Comments

Note that the UPS dependency configuration file (uboonecode/ups/product_deps) sets up a copy of larlite for you automatically.

...
# Add the dependent product and version

product          version
swtrigger       v02_02_01
larlite         v05_09_00
...

Do not setup your own larlite and then build the code. To make sure the right larlite is being pointed to, try:

> echo $LARLITE_BASEDIR
/grid/fermiapp/products/uboone/larlite/v05_09_00/Linux64bit+2.6-2.12

You can see the larlite copy lives in the ups product area (/grid/fermiapp/products/uboone).