TeraVR is released Open Source, as part of Vaa3D (https://github.com/Vaa3D/release). References and protocols for installing and using Vaa3D can be found at (Peng, H., Bria, A., Zhou, Z., Iannello, G., & Long, F. (2014). Extensible visualization and analysis for multidimensional images using Vaa3D. Nature Protocols, 9(1), 193–208.). While Vaa3D can be used directly on a common workstation or laptop, using TeraVR requires a virtual reality environment to be properly deployed. Currently, TeraVR is compatible with off-the-shelf VR equipment such as HTC Vive and Vive Pro. The user can go to https://www.vive.com/setup/ to download the software for Vive and follow the on-screen instructions to install it (Figure 1). During installation, the user will also be guided to set up the lighthouses, connect the headset to the computer or laptop, pair the Vive controllers, and initialize the virtual environment. A detailed instruction on Vive installation can be found at https://support.steampowered.com/steamvr/HTC_Vive/ .

Figure 1. Installation of the Vive software.

Sometimes, there might be needs for deploying multiple TeraVR systems in a same workspace. In that case, each computer running TeraVR needs to be independently equipped with a Vive headset and two Vive controllers. However, they can conveniently share a same pair of lighthouses for movement tracking (Figure 2).

Figure 2. A setup of multiple TeraVR systems in a shared workspace. In the scene, there are 6 sets of VR/TeraVR systems, all of which share a single pair of lighthouses (pointed by arrows).

First, the user needs to start Vaa3D software, load an image volume, a surface object, or a whole-brain data, power on the Vive device, and also launch the Steam VR software. Next, TeraVR can be launched from either a normal 3-D viewer or a TeraFly 3-D viewer of Vaa3D by clicking the “See in VR” button, as shown in Figure 3. After that, the user can put on the VR headset to start using TeraVR , and all the contents previously in the 3-D viewer are now observable in TeraVR. In addition, a companion window that shows the replicated view in the headset is displayed on computer monitor (Figure 4), in order for other users to know what is currently being seen in VR.

Figure 3. Entering TeraVR from Vaa3D. (a) A normal 3-D viewer. (b) A TeraFly 3-D viewer. The “See in VR” button is highlighted in both (a) and (b).

Figure 4. The companion window showing left-eye and right-eye views of TeraVR.

To exit TeraVR, the user can either press the exit button on the setting controller (see Figure 5a) or press the ESC button on the keyboard. After exiting, all the latest annotation results in TeraVR are brought back to the 3-D viewer, where the user can continue to edit.

User interactions in TeraVR is mainly carried out with the two handheld controllers. These two controllers can be used either individually or simultaneously. Two corresponding virtual models of the controllers are shown in TeraVR, which have synchronized locations with the physical devices. The annotation controller (AC) is mainly used for annotation purpose (Figure 5a), and the setting controller (SC) is mainly used to adjust various settings (Figure 5b). Each controller has a touchpad and a trigger button (Figure 5a, 5b). Their functions will be described in detail in the following sections. Besides, AC has a Menu button , and SC has a Quit button . The menu button is used to activate/deactivate the menu for various TeraVR functions and settings. When the menu button is pressed, the menu for major TeraVR functions (Figure 5c) appears on top of SC. Pressing the menu button again will hide the menu. In order to select a menu item, the user aims the red-colored shooting ray from AC at the desired menu item, and presses the trigger button of AC.

Figure 5. The controllers and the menu interface in TeraVR. (a) Annotation controller. (b) Setting controller. (c) The TeraVR menu interface.

TeraVR is based on stereo visualization and provides an intuitive tool for observing 3-D imaging data and 3-D objects. In TeraVR, real and virtual worlds are synchronized. When a user wearing the VR headset changes the viewing direction by turning the head or moving around, the contents being visualized are automatically updated according to the new viewing direction, which brings a very similar experience as the real world perception. Also, just like dealing with physical objects, the user can virtually hold 3-D imaging data and objects in hand and rotate around to adjust observation perspective. This can be done by holding the trigger button of SC and rotating the controller.

Current imaging techniques often generate anisotropic data, where the z resolution is not as high as the x and y resolutions. TeraVR supports non-uniform scaling of imaging data, and the user can stretch the data in the z direction by touching the button on the touchpad of AC. If the upper half of the button is touched, the z direction keeps scaling up; if the bottom half is touched, the z direction keeps scaling down.

TeraVR supports hierarchical management and visualization of teravoxel-scale images. To zoom-in/zoom-out across different resolution levels, select the button from the menu, and press the button on SC accordingly. To shift to an adjacent image volume of the same resolution level, select the button from the menu, place SC at a desired position, and press the button on SC. Depending on the relative position between the SC and the center of the image volume, a corresponding adjacent image volume will be fetched and displayed. There are also several settings and functions related to visualization, such as adjusting contrast. Refer to “Settings and tools” for details.

TeraVR provides various tools for 3-D neuron morphology annotation. With TeraVR, the user works in a true 3-D environment and has full degrees of freedom while interacting with 3-D imaging data and objects. The user can precisely pinpoint a 3-D location in the virtual world in an intuitive way (e.g., by posing a controller to a desired location) and carry out various annotation tasks, such as tracing neurites, adding markers, modifying geometry, etc. This section describes the major annotation functions of TeraVR.

To add a neurite, the user opens the menu, selects the icon for the “Add neurite” function, and closes the menu after that. Next, the user finds a suitable viewing perspective, aims at the desired neurite signal with the sphere on AC, and traces the neurite by moving the sphere along it while holding the trigger button of AC.

TeraVR provides two modes for tracing a neurite, i.e. the virtual finger (VF) mode and the non-virtual finger (non-VF) mode. With the VF mode, the drawn segment will automatically align with the signal in the imaging data to achieve high accuracy and consistency in tracing. With the non-VF mode, TeraVR will generate a free-form curve that strictly follow the moving path of AC. In general, we can use the VF mode to trace relatively strong signals and use the non-VF mode to trace extremely weak signals.

When the newly traced neurite is close enough to an existing one, TeraVR will automatically attach them together to form a connected neuronal tree. To differentiate structures that have different biological meanings, the user can assign different colors to each curve and marker. If the white color is used, the current neurite will adopt the color of the neurite which it connects to. For settings of the virtual finger mode and the tracing color, refer to “Settings and tools” for instructions.

To delete a neurite, the user selects the icon in the menu. After the function is selected, the user moves AC near a neurite to be deleted and presses the trigger once. TeraVR will find a nearest neurite around the AC and delete it. If there is no nearby neurite, none will be deleted.

Sometime the user needs to put markers at specify locations of interest. To do that in TeraVR, the user selects the icon in the menu. Then, the user moves AC to the location of interest, aims with the sphere on AC, and presses trigger once. A marker can thus be added at that location in the currently chosen color. To change color for adding markers, refer to “Changing the color for tracing”. To delete a marker, the user aims at the marker to be deleted with the sphere on AC, and press trigger once.

Sometime, the user would like to fine tune the geometry of an existing neurite, in order to further improve the precision of reconstruction. TeraVR provides the node dragging function that allows one to modify the location of individual nodes on neurites. To do that, the user selects the icon in the menu. After the function is selected, the user reaches a local region of a neurite with AC and holds the trigger button. A node on this neurite is thus locked by the AC. Then, without releasing the trigger, the user moves the node to a desired location and releases the trigger. The geometry of the neurite will then be modified accordingly.

To split a neurite into two smaller ones, the user selects the icon in the menu. Next, the user aims at the location to be split on the neurite with the sphere on AC and presses the trigger once. After that, the original neurite is broken into two neurites. The user can continue to edit these new smaller neurites, e.g. deleting one of them.

Sometime, the number of nodes of a neurite is too few to precisely represent the exact signal in the imaging data. In such cases, the user can use the subdivision function to add more nodes to a neurite to increase its representation capability. First, select the icon in the menu. Then, use the sphere on AC to aim at the neurite where more nodes are desired. Next, press the trigger once. As a result, the subdivided neurite will now have more nodes which allows the user to modify the geometry in detail. Normally, there would be slight deformation on the neurite to indicated that the subdivision succeeded.

This section describes several settings and tools in TeraVR to facilitate neuron reconstruction. Most of the settings will be remembered even the user temporarily quits from TeraVR.

TeraVR supports undo and redo operations during neuron reconstruction. First, the user selects the icon, and the button appears on the touchpad of SC. To undo a previous operation, the user presses on the left half of the button; similarly, to redo a revoked operation, the user presses on the right half of the button. TeraVR supports 5 levels of undo and redo.

During neuron reconstruction, the strength of signal varies across the neuronal image, and the user usually needs to adjust the contrast of image from time to time, in order to have the best observation of the signal. In TeraVR, the user can first select the icon, and then use the button on the touchpad of SC to adjust contrast. Press left half of the button to decrease contrast level and the right half to increase contrast level. Press the button one or more times to obtain the ideal effect.

In TeraVR, a user can choose to temporarily hide the annotation in order to inspect the imaging data more clearly, and to show the annotation again later to continue the reconstruction. To do that, the user selects the icon from the menu, and presses the or button on the touchpad of SC to toggle the show/hide status.

The color for tracing is indicated by the sphere on AC. To change the tracing color, open the menu and select the icon. As a result, an button will appear on the touchpad of SC. While the user presses the button, the tracing color will be switched among a set of colors. Once a color is set, all the neurites and markers added later will adopt that color.

To toggle between virtual finger mode and non-virtual finger mode, the user opens the menu can select the icon. Then, there will be a or button on the touchpad to indicate the status of the VF mode. Press the touchpad to toggle the VF mode.

Neuronal imaging data might have uneven distribution of signal strength even in a local region which could cause difficulties for reconstruction. For example, while the dendrites are relatively bright, the axonal neurites around soma area can be quite dim and thus difficult to observe even after adjusting contrast. TeraVR has the brightness suppression function, which can be used to suppress the bright signals and make the dim structure easier to observe. To do that, the user selects the icon from the menu, and presses the or button on the touchpad of SC to enable or disable it.

TeraVR can be used for the visualization of both single-channel and multi-channel imaging data. For multi-channel data, TeraVR lets the user show all the channels simultaneously, or switch among individual channels. To switch channels, the user selects the icon from the menu, and a button will appear on the touchpad of SC. Press either “-” or “+” to switch among red/green/blue/all channels, in a forward or backward order.

For 3-D imaging data, sometimes inspecting data from various angles is preferred in order to accurately understand the underlying structure. In TeraVR, the user can select the icon from the menu for the automatic rotation of 3-D data. To enable automatic rotation, the user presses the button on the touchpad of SC. Then, the data will start to rotate automatically using the position of SC as the rotation origin. The user can press the button to stop it.

Reconstructing full morphology of neurons is a challenging task, and a collaboration mechanism can be helpful for both neuron reconstruction and proofreading. The collaboration mode of TeraVR allows multiple users to work collaboratively on a same piece of data. These users can work together in a same workspace or be located worldwide.

To use the collaboration mode, first a cloud-based server needs to be launched to perform user management, synchronize reconstruction data, and coordinate actions among multiple users. The server program can be found at https://github.com/Vaa3D/vaa3d_tools/tree/master/hackathon/liqi/VR_Server.

The collaboration mode of TeraVR can be initiated both from normal 3-D viewers for regular image data and from TeraFly 3-D viewers for whole-brain image data. Take the latter case here as an example. In the TeraFly 3-D viewer (Figure 3b), the user clicks the “Collaborate in VR” button, and will be prompted to input server address, server port, and login name (Figure 6). After entering the above information, it will connect to the collaboration server, and start TeraVR with collaboration. Any other users that are working on the same whole-brain imaging data can join the collaboration with the same procedure. The number of users currently in collaboration is displayed on the title of the companion window.

Figure 6. Setting up connection to the server for collaborative reconstruction. (a) Input of IP address. (b) Input of port number. (c) Input of login name.

In the collaboration mode of TeraVR, an avatar for each user that represents the user’s instant position is displayed, and its position is updated when the user moves. Therefore, a user is aware of where other people are working at. Also, each user is assigned with a unique color, which is used for both the avatar and the reconstructions done by the user. Basically, annotation operations in collaboration mode do not differ too much with those in standalone mode. Only that during collaboration, reconstruction results are shared in real-time among all the collaborators. This means users not only generate reconstructions on their own, but can also edit the reconstructions created by others as well, e.g. deleting a neurite, changing the local geometry of a neurite, etc. Moreover, the collaboration is flexible in that users can even work at different regions of the whole-brain imaging data at different level-of-details.