Upload Data for Viewing Via GitHub Pages - CrossR/HepEVD GitHub Wiki

Public Data

To view data on the public website, you need first to have everything you want to see running in HepEVD. Follow the other instructions for that, and come back here once you have it running with the state or states you wish to show publicly.

Exporting the Data

First, you want to navigate to http://localhost:5555/writeOutAllStates, replacing 5555 with whatever port your EVD is currently running on. That should give you a text confirmation when it finishes as to where your data files have been dumped out.

You can also go to /stateToJson to dump out a single state, at which point you can skip to the next step, as you only get a single file out.

You should have 2 sorts of files:

  • Numbered state files: For example 0_Input_2D_Hits.json which contains all the state data for that state. This includes the name, hits, MC Hits, MC Truth string, particles and any markers.
  • Top Level File: There is also a top level eventDisplayInfo.json file, that brings together all of these other files. This means that it contains the event level info, such at the geometry and config, and then links to the other files.

Check your eventDisplayInfo.json file and remove any states you don't care about.

Storing the Files

The easiest way to store these files is in a GitHub Gist.

You will likely want to make a new gist and put in the file name and some placeholder contents. This is because the files we have are probably very big, and a pain to directly copy and paste in. So make a placeholder for each of your files and create a gist.

Once made, you can go to the top of the page and where it says Embed with a link, swap that to Clone via HTTPS or SSH. Gists are basically just Git repos, so can now proceed to clone your gist locally.

You will likely want to clone as follows:

git clone $GIST_CLONE_URL myNiceOutputName

as otherwise you will have just a random set of numbers, which is hard to understand.

Once cloned locally, replace your placeholder files with the real files.

If you've only produced one state, you are basically done, commit and push the change, then check the Gist website again.

If you've produced many files, there is one extra step. Copy over and commit all your state files and the top level config. Push it up and refresh the gist page. You now need to update your config file to actually point to the state files.

The page should now have all your states and config file in. You will want to open and then copy the link from the Raw button for each of your states, so you have a direct link to the raw data in that state. With that link on your clipboard, add it to your eventDisplayInfo.json file at the bottom, in the same part as the relevant state name.

For example:

    "numberOfStates": 2,
    "states": [
            "name": "Input 2D Hits",
            "url": "https://gist.githubusercontent.com/CrossR/2edd3622d13987d37ef3a4c02286207c/raw/eecbcf39fdb4aa51ba5f56733179ce4a1463ffda/0_Input_2D_Hits.json"
            "name": "Final 3D Reconstruction",
            "url": "https://gist.githubusercontent.com/CrossR/2edd3622d13987d37ef3a4c02286207c/raw/eecbcf39fdb4aa51ba5f56733179ce4a1463ffda/29_Final_3D_Reconstruction.json"

Note that the numberOfStates should also be correct, if you changed anything.

You can also add a citation to this file, if useful:

    "citation": {
        "text": "Powered By MicroBooNE Open Data",
        "url": "https://microboone.fnal.gov/documents-publications/public-datasets/"

Finally, before committing that file...double check its size. The state files are produced with no whitespace / new lines or formatting at all, whereas this file is somewhat human readable. If the file is huge, you may want to strip out all the whitespace, just to make the load a bit faster.

You likely won't need to do that for a LArTPC event, as the geometries are pretty tiny, but when running on an ODD event or similar large detector, the files can get very big.

Once fully edited, commit those changes and push up, and you should have a Gist containing your states and a config state that points to your underlying files.

Viewing the data

Finally, once you have your files all uploaded, you can view them by passing the raw data URL (which is taken from the Raw button next to the file) to the GitHub Pages URL.

If you have one state only, this is simply the link to that state. If you have multiple states, it instead the raw URL of the eventDisplayInfo.json file.

The URL is passed as follows:


i.e. the full regular URL, then /?data=$URL.

Loading the page should then load your data as normal.

Please note that some parts aren't currently included in the public version, such as showing images. This is mostly due to the bloated size of the states when using images.