Home - KCL-BMEIS/xnat_form_maker GitHub Wiki

XNAT Form Maker

Overview

The XNAT Form Maker can convert RedCap Data Dictionaries and Excell spreadsheets describing forms (for example, neuropsychological forms, demographic forms, QA forms) into XNAT Plugins.

The template spreadsheet (./plugin-spreadsheet-template.xlsx) contains a sheet FormDetails which lists the forms in the Spreadsheet book, grouped into a schema. For each form listed, you must fill in the corresponding sheets (Form1, Form2 etc). The forms list the questions and the datatypes stored (integer, string, float etc). The template spreadsheet contains detailed instructions and examples.

Multiple spreadsheets and RedCap dictionairies can be converted into plugins at the same time and spreadsheets reused. I.e., once a spreadsheet has been created, it can be used on multiple XNAT instances. Hence, it is good practice to group only related forms together into one schema, for example all QA forms can be grouped into a QA Schema. This modular design makes it easier to reuse and modify spreadsheets.

RedCap Datadictionaries in csv form can also be converted directly into XNAT plugins. Most fields are converted directly, however the Calc field is not yet fully implemented in the XNAT Form Maker. If it is not able to convert the calc field, it is converted into a text field. Please read more about REDCap conversion here

To install plugins on the server, place all the required spreadsheets into the xlsx folder and data dictionaries into the redcap_dict folder, run the build script and restart tomcat. All the forms from all the spreadsheets and dictionaries will now be available from the XNAT user interface.

If you are building the plugins on a separate server to your XNAT, then you will have to manually copy the generated plugins to your XNAT server plugins directory:

https://wiki.xnat.org/documentation/xnat-administration/deploying-plugins-in-xnat

Test your plugin on a test XNAT server before installing on a live XNAT, to ensure everything is as expected.

Installation

  • Install on the same server as XNAT, if possible.

  • Ensure you have python3 and pip installed and dependencies:

redhat/centos: sudo yum install python3 python3-pip dos2unix && pip3 install openpyxl
ubuntu/debian: sudo apt install python3 python3-pip dos2unix && pip3 install openpyxl
  • Download the scripts:

Download as a zip file and unzip or:

git clone https://github.com/KCL-BMEIS/xnat_form_maker.git // GET THE FORM-MAKER
cd xnat-form-maker
  • The folder contains a configuration file, xnat.cfg. Ensure the xnat.cfg file is filled in correctly.
[xnat]
xnat_name = xnat  // name of the XNAT site
xnat_host = // url of server, eg: https://xnat.kcl.ac.uk
xnat_home = // home directory of the tomcat user on the server - usually '/data/xnat/home' if XNAT is installed directly on the server.
xnat_version = // 1.8, 1.7.6 or 1.7.4

If you are not running on the same server, then xnat_home and xnat_user are not used. You will have to copy the generated plugins manually.

The xnat-form-maker directory contains a template spreadsheet, plugin-spreadhseet-template.xlsx, that you can use to create plugins. There are several subfolders, including xlxs where spreadsheets to build are placed. The plugins are created in the folder plugins.

There are example spreadsheets in the ./examples directory.

To create a new plugin

  • copy and rename the plugin-spreadhseet-template.xlsx and place in xlsx directory, for example:

cp plugin-spreadhseet-template.xlsx ./xlxs/test.xlxs

  • Open the new xlsx file. Select sheet FormDetails. Fill in the schema name at the top (in red). No Spaces, special characters or underscores! In the example below, the schema name is wikidemo:

formdetails_formmaker_demo

  • If you wish the forms to be visible in only a specific project, fill in the Restrict to Project field. This will be a wildcard entry, meaning the forms will be visible on any project with project IDs containing this.

  • For each form, write the title, data name, description etc as example above. Data Names are the names as stored in the Postgresql database. These cannot contain spaces or special characters. You have to select whether this is subject or imaging session specific by typing image or subject in the Extension/Level column. Subforms are a special type explained here.

  • For each form, fill in corresponding sheet (form1, etc). See How to Create Forms

  • Place RedCap data dictionaries in the ./redcap_dict/ directory. The name of the generated XNAT schema will be the name of the file. Rename the file if necessary. Ensure the filename contains no special characters, underscores etc.

  • When all the forms have been created, run:

python3 makeforms.py

  • The script will build the plugins and, if run on the XNAT server, place the plugins in the plugins directory ( default /data/xnat/home/plugins).

If not, you will have to copy the plugin to the XNAT server. The plugin are created in the plugins folder. Each spreadsheet will create a separate plugin. The plugin is a .jar file is named after the schema name specified in the spreadsheet. The source code directory for each plugin is also included, in case you wish to manually modify and rebuild the plugin.

If required, copy the plugin jar file to your xnat plugin directory of your XNAT server. Restart tomcat to finish installing the plugins.

https://wiki.xnat.org/documentation/xnat-administration/deploying-plugins-in-xnat

When you have installed the plugin, you can then upload data in bulk using generated csv files - read Importing data using CSV files

Updating Plugins

Spreadsheets and data dictionaries can be updated and makeforms.py re-run to generate new plugins, which can then be installed on your XNAT.

Once a datatype has been installed on an XNAT instance, it cannot be deleted. You can add questions to a form, however if you wish to remove a question that already exists in an XNAT installation, you can put 'x' in the hide column - it will keep the datatype in the database, but it will not display the question in the edit and report forms.

You can edit xnat_version in the configuration file and re-run the form-maker to update the plugin for newer versions of XNAT.

For custom changes, read the section Modifying Existing Plugin.

Restricting a plugin to specific XNAT projects

The visibility of Forms can be restricted to certain projects. The menu item 'Add XX Assessor' of the plugin will be limited to these projects. For RedCap conversions, build the plugin, then place the script plugins_restrict_project.sh in the root directory of the plugin (./plugins/TESTPLUGIN/ for example) and run it.

To run:

./plugins_restrict_project.sh PROJECT_ID

This is a wildcard filter - any project containing PROJECT_ID will display the menu item.