Register new interactive tool - inab/openVRE GitHub Wiki

Installing and Instantiating a New Interactive Tool in openVRE

This guide provides the steps to add and configure a new interactive tool in openVRE. An interactive tool allows users to launch a job and interact with the tool through a dedicated session page that opens after the job is launched. This documentation covers the process of creating the necessary tool directory, modifying input/output PHP files to suit the tool’s requirements, updating asset paths (such as images and HTML), and integrating the tool with MongoDB to ensure proper data handling and consistency across the system.

By following these steps, your tool will be available to users in the openVRE as an interactive tool, allowing them to run jobs and interact with the tool in real-time via a web interface. This setup ensures that all necessary configurations are made for both the backend (e.g., MongoDB and PHP configurations) and frontend (e.g., HTML and assets) to provide a seamless interactive experience for users.

The working directory for this tutorial is the dockerized_vre from the previous step of installing the Dockerized version of VRE and Creating a Dockerized VRE adapted tool. In this step, we are gonna take as an example a VTK Viewer, which is a stand-alone instantiation of Paraview Visualizer for OBJ, STL and PTH files.

Step 1: Create Tool Directory and Initial Files

  1. Create Tool Directory:

Navigate to volumes/openVRE/tools/ and create a new directory for your tool. For example, if you're adding a tool called vtk_viewer, run the following:

mkdir volumes/openVRE/tools/vtk_viewer

1a. Move the vtk_viewer_template dir

An already existing directory for the seqio_tool, rename the directory for the Tool to be available on the platform.

mv vtk_viewer_template vtk_viewer
  1. Create Required Files: Inside the new tool directory, ensure you have the following structure:
volumes/openVRE/tools/vtk_viewer/
├── input.php
├── output.php
└── assets/
    └── home/
        └── index.html

You can use the tool_skeleton directory as a template, but ensure that the content of these files matches the specific needs of your new tool.

Step 2: Modify _input.php_ to Handle Tool Requirements

In this step, you will modify the input.php file to properly handle the tool’s input based on the MongoDB data structure and collection we are gonna update as last step.

  1. Find the Tool ID Section: Open input.php and locate the line that retrieves the tool details. Modify the $toolId to match your tool’s ID.
$toolId = "vtk_viewer";  // Change this to your tool's ID

The $tool_id would be the _id in the MongoDB Tools collections, so make sure they are matching!

  1. Handle Input Files:

In the section where the input files are handled, update the code to dynamically show and retrieve the file based on file types from MongoDB. Specifically, you’ll need to access the data_type collection to match the file types required by your tool. In this example, the input_file is gonna a single one, since the visualizer is gonna process one file at a time.

Original tool_skeleton code:

<?php if( $_REQUEST["op"] == 0 ) { ?>
    <div class="col-md-12">
        <?php $ff = matchFormat_File($tool['input_files']['your_input_file_name']['file_type'], $inPaths); ?>
        <?php InputTool_printSelectFile($tool['input_files']['your_input_file_name'], $rerunParams['your_input_file_name'], $ff[0], false, true); ?>
    </div>
<?php } ?>

Modified code:

<?php if( $_REQUEST["op"] == 0 ) { ?>
   <div class="col-md-12">
						<?php $ff = matchFormat_File($tool['input_files']['3d_object']['file_type'], $inPaths); ?>
						<?php InputTool_printSelectFile($tool['input_files']['3d_object'], $rerunParams['object_file'], $ff[0], false, true); ?>
					</div>
    </div>
<?php } ?>

The tool will now fetch the _file_type_ and _data_type_ dynamically based on the MongoDB collection. You’ll need to replace your_input_file_name with the actual input file key as defined in your MongoDB tools collection.

Step 3: Modify Assets (e.g., index.html, Images)

Next, modify the assets directory for your tool:

  1. Update index.html:

Open volumes/openVRE/tools/vtk_viewer/assets/home/index.html and replace paths referencing the skeleton tool with your new tool’s paths. For example:

<link rel="stylesheet" type="text/css" href="path_to_vtk_viewer/css/style.css">
<img src="path_to_vtk_viewer/img.png" alt="VTK Viewer">
<img src="path_to_vtk_viewer/logo.png" alt="VTK Viewer Logo">
  1. Change Images (if necessary):
  • If the images (img.png, logo.png) need to be updated for your tool, replace them in the assets folder with the appropriate images.
  • If you don’t need custom images, the default ones will be shown.

Add the Tool to MongoDB Collection

To ensure your tool is integrated properly into the system, you need to add it to the MongoDB tools collection.

  1. Add Tool to MongoDB (via MongoDB Compass or JSON File):
  • Option 1: Add tool dynamically via MongoDB Compass:

Open MongoDB Compass and navigate to the tools collection and a new document for your tool using the following structure:

{
  "_id": "tool_interactive_skeleton",
  "name": "Interactive Tool Name",
  "title": "Interactive Tool Title",
  "short_description": "A brief description of the tool's functionality",
  "long_description": "A detailed description of what the tool does, including its use cases, functionality, and any relevant details.",
  "url": "URL where the tool can be accessed or more details are provided",
  "publication": "Publication details or references",
  "owner": {
    "author": "Author Name",
    "institution": "Institution Name",
    "contact": "Contact Email",
    "url": "Link to the author's or institution's webpage"
  },
  "status": 1,
  "external": true,
  "keywords": [
    "keyword1", "keyword2", "keyword3"
  ],
  "keywords_tool": [
    "tool-specific keyword1", "tool-specific keyword2"
  ],
  "infrastructure": {
    "memory": 12,  // Memory requirements in GB
    "cpus": 4,  // CPU requirements
    "interactive": "yes",  // Whether the tool is interactive
    "executable": "./path/to/executable",  // Path to the executable or script as it is in the Docker container
    **"container_image": "container_image"**, // Make sure to not have the tag specifics, the ":" is not accepted from docker.socket
    **"container_port": "8090"**, // Internal port - HAS TO BE ALWAYS 8090! Make sure to be consistent with its Dockerfile
    **"interactive" : true**, // Boolean 
    "clouds": {
      "local": {
        "launcher": "docker_SGE",  // Type of launcher (e.g., Docker, SGE)
        "default_cloud": false,
        "queue": "testq"  // Name of the queue for task submission
      },
    }
  },
  "input_files": {},
  "input_files_public_dir": [
    {
      "name": "input_file_name",
      "description": "Description of the input file",
      "help": "A help tooltip explaining the input file's purpose and format",
      "file_type": ["file_type"],
      "data_type": ["data_type"],
      "type": "input_type",
      "value": "default_file_value_or_path",
      "required": true,
      "allow_multiple": false
    }
  ],
  "input_files_combinations": [
    {
      "description": "Description of input file combination",
      "input_files": ["input_file_name"]
    }
  ],
  "arguments": [
    // Example: Arguments required for tool execution
    {
      "name": "argument_name",
      "description": "Description of what the argument does",
      "type": "argument_type",
      "required": true
    }
  ],
  "has_custom_viewer": false,  // Whether the tool has a custom viewer for its outputs
  "output_files": [
    {
      "name": "output_file_name",
      "required": true,
      "allow_multiple": false,
      "file": {
        "file_type": "output_file_extension",  // Output file type
        "data_type": "output_data_type",  // Data type of the output
        "meta_data": {
          "visible": true,
          "description": "Short description of the output file",
          "tool": "Tool name"
        }
      }
    }
  ],
  "sites": [
    {
      "site_id": "site_name",
      "status": 0  // Site availability status (e.g., 0 = unavailable, 1 = available)
    }
  ]
}

For example, for the Interactive Visualizer, the MongoDB documents is this one:

 {
    "_id": "vtk_viewer",
    "name": "VTK Viewer",
    "title": "VTK Viewer",
    "short_description": "Interactive viewer",
    "long_description": "",
    "url": "http://gitbub.com/mapoferri/Biobb",
    "publication": "",
    "owner": {
      "author": "Maria Paola Ferri",
      "institution": "Barcelona Supercomputing Center",
      "contact": "[email protected]",
      "url": ""
    },
    "status": 1,
    "external": true,
    "keywords": [
      "viewer",
      "structure"
    ],
    "keywords_tool": [],
    "infrastructure": {
      "memory": 12,
      "cpus": 1,
      "interactive": true,
      "executable": "bin/pvpython -m paraview.apps.visualizer",
      "container_image": "paraview_image2",
      "container_port": "8787",
      "container_env": {
        "host": "172.21.0.3",
        "port": "8090"
      },
      "clouds": {
        "local": {
          "launcher": "docker_SGE",
          "default_cloud": true
        }
      },
      "free_port": "9001"
    },
    "input_files": [
      {
        "name": "object_file",
        "description": "OBJ / PTH 3D file",
        "help": "3D files for visualization",
        "file_type": [
          "OBJ"
        ],
        "data_type": [
          "3d_simulation",
          "vertebral_disc_coordinates"
        ],
        "required": true,
        "allow_multiple": false
      }
    ],
    "input_files_public_dir": []
    ,
    "has_custom_viewer": true,
    "custom_viewer": "3D_Viewer",
    "custom_viewer_description": "A tool for visualizing 3D structures and analyzing spatial data from the generated PPI network.",
    "output_files": [
      {
        "name": "output",
        "required": true,
        "allow_multiple": false,
        "file": {
          "file_type": "TXT",
          "data_type": "variants",
          "meta_data": {
            "visible": true,
            "description": "TXT file containing the variants/genomic ids correlated to the disease.",
            "tool": "vtk_viewer"
          }
        }
      }
    ],
    "sites": [
      {
        "status": 1,
        "site_id": "local"
      }
    ],
    "input_files_combinations": [
      {
        "description": "3D Simulations",
        "input_files": [
          "3d_simulation",
          "vertebral_disc_coordinates"
        ]
      }
    ],
    "input_files_combinations_internal": [
      {
        "3d_simulation": "+",
        "vertebral_disc_coordinates": "+"
      }
    ]
  }
  • Option 2: Add tool manually to tools.json:

Open the front_end/openVRE/install/database/tools.json file and add the tool configuration, and copy and paste there the same format as the above schema.

  1. Update file_type Collection: If the tool uses new file types, add them to the file_type collection to ensure consistency. Example of adding a new file type:
{
    "_id": "OBJ",
    "extension": [
        "OBJ",
        "STL",
        "PTH"
    ]
}
  1. Update data_type Collection: If necessary, add new data types to the data_type collection to match the tool’s input/output requirements. Example of adding a new data type:
{
  "_id": "3d_object",
  "name": "3D Object",
  "file_types": [
    "PTH",
    "OBJ",
    "STL"
  ],
  "assembly": []
}

Step 5: Restart the MongoDB Service

To update the new MongoDB configuration, it is necessary to restart the service:

docker-compose up -d mongo_seed

Step 6: Test and Verify Tool Integration

  1. Verify the Tool in openVRE:
  • After adding the tool to MongoDB, reload openVRE service to load the tools configuration.
  • Test the tool by navigating to its URL in the openVRE interface and check if a new box appeared in the workspace.
  • Test if the input/output forms are displayed correctly, based on the html modifications and MongoDB details.
  • Ensure that the file types are correctly recognized and processed.
  • If any issues arise (e.g., incorrect file handling or missing paths), review the input.php and output.php files to ensure they match the expected tool requirements and MongoDB structure.
⚠️ **GitHub.com Fallback** ⚠️