Home - rg2/xreg GitHub Wiki

Table of Contents

  1. Introduction
  2. Walkthrough
  3. Frequently Asked Questions (FAQ)
  4. File Formats
  5. Code and Programs for Related Publications

Introduction

This wiki is intended to help people use the xReg repository, ranging from basic usage of the compiled executables to more advanced topics, such as using xReg as a library in order to provide new capabilities. A walkthrough is provided in order to quickly indoctrinate users with common xReg programs and the usage conventions. Several other examples of xReg usage on other datasets are listed under the "Other Examples" section of the sidebar. See the FAQ section for pointers to more advanced topics. The file formats section provides a description of the major file formats used by xReg and can help users convert data to and from xReg compatible formats. Additional applications leveraging xReg are listed in the related publications section.

Walkthrough

This section outlines a walkthrough, or tutorial, of the basic functionality of the xReg software and how it may be applied to solve common computer assisted surgery workflow tasks, ranging from preoperative planning to intraoperative navigation. All data used in the workflow is either obtained from freely available online sources, such as The Cancer Imaging Archive (TCIA), derived by processing the TCIA data, simulating new data, or some combination therein. Links to all of the derived and simulated data files are provided here.

A basic understanding of 3D Slicer assumed, since it is used to perform basic annotations of landmarks and visualization of intensity volumes and label maps. Some tutorials on 3D Slicer are found here.

The walkthrough provides examples of calling the executable programs and does not delve into lower-level usage of xReg as a library. Users that wish to use, or modify, the xReg library should also examine the appropriate source code as they proceed through the walkthrough.

Walkthrough Outline:

  1. Obtaining the tools
  2. Preoperative planning
    1. DICOM conversion and resampling
    2. Volume cropping
    3. Bone segmentation
    4. Mesh creation
    5. PAO planning and fragment creation
  3. Simulated data
    1. Random PAO fragment and femur movements
    2. Volumetric modeling of PAO fragment and femur poses
    3. Insertion of screws and K-wires and volumetric modeling
    4. Fluoroscopy
  4. Intraoperative 3D/3D registration
    1. Point cloud to surface registration
  5. Intraoperative 2D/3D (fluoroscopy/CT) registration
    1. Single-view pelvis registration
    2. Multiple-view pelvis/femur/PAO fragment registration
  6. Walkthrough Data

Frequently Asked Questions (FAQ)

Help generate this section by asking some questions via the repository issues page, email, or twitter!

  1. Are there pre-compiled binaries for Windows/Mac/Linux?
    • Yes! There are compiled versions of the binaries for Windows, MacOS 10.14, CentOS 7 and Ubuntu 16.04 (all amd64). Check the releases page.
  2. When running the pre-compiled Windows binaries, I get an error message stating that "The code execution cannot proceed because VCRUNTIME140_1.dll was not found. Reinstalling the program may fix this problem." How do I fix this?
    • You need to install the Visual Studio C++ runtime, which is available here or here (under the "Other Tools and Frameworks" section choose x64).
  3. How do I select a specific GPU (or OpenCL device) to use on a system with several GPUs (OpenCL devices) available?
    • This may be accomplished by setting the desired device ID with the --ocl-id flag or through the use of environment variables (e.g. CUDA_VISIBLE_DEVICES). See this page for further information.
  4. Is it possible to run on a system that does not have a GPU or a less-capable GPU (e.g. a server)?
    • Yes! Select the CPU backend by passing --backend cpu or use the OpenCL backend with a compatible CPU device (e.g. something like --backend ocl --ocl-id "Intel(R)Core(TM)[email protected]". Tools that do not provide a backend flag only perform CPU processing. See this page for further information.
  5. My program crashed with some strange OpenCL error message - what happened?
    • This usually occurs when there are insufficient resources, such as memory, on the OpenCL device. The exact reason depends on the processing involved, but typically adjusting the parameters that effect the amount of memory required may resolve the issue. For example, lowering the population sizes for the CMA-ES or Differential Evolution registration strategies will reduce the number of DRRs created during each optimization iteration, and therefore reduce the memory footprint. This may also occur if your code chains together OpenCL ray casters and similarity metrics, but different contexts or command queues are used for each object.
  6. Why did you choose OpenCL for GPU computations instead of CUDA?
    • Although CUDA kernels are generally more computationally efficient than their OpenCL counterparts, using OpenCL avoids a lock-in to NVIDIA GPUs. Since most NVIDIA, AMD, and Intel GPUs provide OpenCL support, nearly any hardware setup, ranging from entry-level laptops to state-of-the-art compute clusters, may leverage GPU support. This should enable any interested user to run this software. Support for CUDA, Metal, and Vulkan processing backends is planned for future work.
  7. How do I incorporate this library into my software?
    • The xReg library is most easily incorporated using standard CMake practices, such as add_subdirectory() and find_package(). See this page for more details.
  8. I would like to add my own similarity metric (or ray caster, etc.) for 2D/3D registration, what is the best way to do this?
    • The typical way to accomplish this is to create a new class that inherits from an appropriate CPU or OpenCL base class. Additional information for extending the library is provided here.
  9. What's up with the non-trivial extrinsic transformation in most of the example projection data?
    • This matrix represents a transformation to a coordinate frame that has an origin and X-axis set to mimic orbital rotations of a Siemens CIOS Fusion C-arm. This is convenient for modeling the non-isocentric rotations of the CIOS Fusion.
  10. Some published methods require C-arm calibration or knowledge of the orbital rotation axis, can xReg be used to calculate this information?
    • Yes. An example of calculating the orbital rotation axis using intensity-based 2D/3D registrations is found here. Although an optimization over intrinsic parameters is possible using the CamSourceObjPoseOptVars type, a concrete example is currently not provided.
  11. I've used xReg in my own work, are there any restrictions on its use or distribution?
    • We only ask that you cite the most appropriate research papers and adhere to the terms of the MIT license in your software. See this page page for further details.

File Formats

xReg uses several different file formats to store data:

  • Projection data (2D images and metadata) is stored in an HDF5 file (.h5) with a layout specific to the xReg library. Using HDF5 also allows arbitrary metadata to be bundled with the flexibility to add and remove fields. Multiple projections may be stored per file. Details on the layout are provided here. The image data must be extracted into another file format in order to load the image data into another application, such as 3D Slicer.
  • Other image data is stored in the NIFTI file format (.nii or .nii.gz). xReg uses NIFTI files for intensity images/volumes, label maps, and deformation fields. NIFTI is not used for storing projection data as it does not have fields for projection metadata and cannot store multiple images each with different metadata. NIFTI has the advantage of being recognized by many other applications, such as 3D Slicer, ITK Snap, and MITK workbench.
  • Rigid transformations are stored in HDF5 files (.h5) using a layout specified by ITK. These are also loadable by 3D Slicer, but may need to be inverted after loading.
  • Landmark annotations are stored in 3D Slicer's FCSV file format. This allows annotations made in 3D Slicer to be easily used by the xReg library and programs. As FCSV files saved by 3D Slicer use the RAS coordinate frame, all xReg programs will perform an RAS to LPS conversion unless specified by the user.

DICOM

Since the most common format used to represent medical images is Digital Imaging and Communications in Medicine (DICOM), xReg provides several methods for converting data from DICOM format into simplified formats usable by xReg and, sometimes, other tools such as 3D Slicer.

The following command line applications are included in the xReg repository:

  • xreg-convert-dicom-vols: This utility is used to convert, and optionally resample, one or more volumetric scans into one or more volume files (e.g. NIFTI, NRRD, MHD, etc.). When a root directory containing many patients, studies, series, etc., the tool will organize the DICOM files and create a new volume file for each series. A multitude of processing options exist are may be listed by passing the --help/-h flags.
  • xreg-convert-dicom-radiograph: This tool will convert a DICOM file representing a 2D radiograph, or multi-frame radiograph/video-fluoroscopy, into a HDF5 projection data file. When metadata such as pixel spacing is not available, as is common with frames stored from an image intensifier, the tool may attempt to estimate the values when other metadata, such as detector size, is available. Pixel spacings and other metadata, such as source-to-detector distance, may also be overridden using command line parameters. The Hip Radiograph Rigid 2D/3D Registration demonstrates the usage of this program in the context of a 2D/3D registration workflow.

The xReg library includes routines and structures for reading and storing DICOM data with interfaces in the following header files:

  • xregDICOMUtils.h :
    • The DICOMFIleBasicFields structure is used to store the metadata found in a DICOM file and is populated from disk using the ReadDICOMFileBasicFields function. boost::optional is used to store fields that optional depending on other values found in the DICOM metadata.
    • The ReorderAndCheckDICOMInfos routine may be used for organizing a collection of DICOM metadata hierarchically by patient, study and series.
    • Various other routines exist for common tasks, such as checking whether a DICOM is a localizer, secondary image, derived image, etc.
  • xregReadProjDataFromDICOM.h: Routines for reading a 2D radiograph (or multi-frame sequence of 2D radiograph/video-fluoroscopy) from a DICOM file and populating a list of ProjData structures.

Code and Programs for Related Publications

The xReg library and supporting executables have been used for research leading to various publications. Some of these publications are listed below, along with links to any available source code. If you have developed software or published work that leverages xReg and would like it listed here, please let us know!

  1. Automatic Annotation of Hip Anatomy in Fluoroscopy for Robust and Efficient 2D/3D Registration
  2. Pose Estimation of Periacetabular Osteotomy Fragments with Intraoperative X-Ray Navigation
  3. Fast and Automatic Periacetabular Osteotomy Fragment Pose Estimation Using Intraoperatively Implanted Fiducials and Single-View Fluoroscopy
  4. Patch-Based Image Similarity for Intraoperative 2D/3D Pelvis Registration During Periacetabular Osteotomy
  5. Fiducial-Free 2D/3D Registration for Robot-Assisted Femoroplasty
  6. Fiducial-Free 2D/3D Registration of the Proximal Femur for Robot-Assisted Femoroplasty