Getting Started with the HoloLens 2 - utwente-interaction-lab/interaction-lab GitHub Wiki

Introduction

Hololens 1 and Hololens 2

The tutorial on this wiki page is about the Hololens 2. Our lab also has several older original Hololens 1 which work very similarly. However, since these have been discontinued by Microsoft you can only use versions up until Unity 2018.1.0f2 and Visual Studio 2017. The official documentation is a good starting point for developers:

In the box

How to get started

Part 1. Putting it on

  1. Open the box and the case inside. Here you find: the HoloLens, a charger, an information flyer and a cleaning wipe.
  2. Turn on the HoloLens: press the small round button on the back section of the HoloLens (just above the charging outlet)
  3. Put it on: rotate the nob on the back to adjust the lens so it fits over your head, place it on your head and fasten the nob. Also, tighten the strap that goes over your head. And to finish the setup: carefully tilt the glass up and down to adjust your view.
  4. Turn off the HoloLens: briefly press the button for sleep mode OR press and hold until you see ’goodbye’ and all LED lights turn off.

Part 2. Get used to the Mixed Reality

  1. When pressing the ON button, you should see a windows icon appear. This might be followed by a circling icon. This is the iris scanner.
  2. Okay, let’s start interacting with the holograms! You can interact with the holograms by moving your arms and hands. The HoloLens senses your movements and the gestures that you make with your hands. You press and click buttons by holding out your index finger and shortly pushing it through the hologram button.
  3. The first screen you see is an Iris scanning application, click the small dismiss button underneath. Then, it takes you to the Human media interaction login screen. It asks for either an iris, pin or password. For the password, contact the lab manager.
  4. After entering the password, a home screen will appear. Just like a normal computer or tablet, you have different options, such as opening the internet, mail, tips, etc.
  5. For now, open the ‘Tips’ application. This leads you to a short tutorial. It will teach you the first basic interaction gestures that you need to use the HoloLens.
  6. After finishing, you can also try the Guides app. After clicking on the icon in the menu, a 3D shape appears. To interact with the shape, push your hand through the shape and make a grabbing movement, like you are holding something. Now you can drag and move the shape in the space. To release, open your hands. Then, move your hand closer to the shape until a play button appears, and press the button to open the application. Here, you can try a couple of guides, which also give you a nice impression of the use cases for the HoloLens. You can either log in with an account or without.

Tip. If one of the applications does not work: try to connect to WIFI first.

Part 3. Making your first hologram move

  1. In the menu, click all apps. Here, chose 2D viewer.
  2. This opens up the 3D viewer application, and shows you a library with different 3D figures.
  3. To scroll down, push your finger into the screen and drag it down.
  4. Click a figure you like. This will let a 3D model of your choice appear in front of you. It is recommended to try one of the options with a little icon in the top right: this means that the figure has an amination.
  5. Hold and drag the figure down, so it sits on the floor. And remember from the earlier tutorial: you can also rotate or scale your figure.
  6. When you are happy with how your figure is placed, move your hand closer and press the ‘play’ button that appeared. Depending on the animation, the figure will now move in space. Try walking around, changing your perspective, and looking around if you can follow your hologram.

Tip: when you scaled your character, the animation is also scaled. This means that, for example, the skating penguin will now skate large circles in space. So don’t worry if you cannot find your hologram, just try to follow the sounds it makes and look around.

  1. To close the hologram, find the box that it came out off and press the X button.

Tip: take a look at the lifter hologram, this gives a nice impression of a potential use case of the HoloLens.

Developing your own Mixed Reality application

To build mixed reality applications for both VR and AR, we will use the Mixed Reality Toolkit (MRTK). The toolkit provides you with premade components that you can use to get started quickly. This tutorial is based on the Microsoft learning platform, however, multiple issues are resolved in this tutorial.

Chapter 01: Opening Unity

  1. Open the Unity Hub
  2. Click projects, and New project.
  1. On the left, chose Core. From here, chose the 3D core template. If this is not possible, update your Editor Version: in the Instals tab and click Install Editor. Next, give your new project a name and a location. Click Create Project to open the Unity project. This might take a while.

Chapter 02: Configuring Unity

The standard Unity settings are not suitable for mixed reality, therefore we need to make sure the default is changed to immersive headsets.

  1. In the menu bar, select File > build settings
  1. Select Universal Windows Platform.

PROBLEM 1: If there is no platform build support module loaded, click the button to install this additional module. You might have to restart your unity project if no setting appear.

PROBLEM 2: If you don’t have Windows 10 SDK installed, please download it here Restart your unity project afterwards.

PROBLEM 3: Make sure your visual studio has the ‘Universal Windows Platform Development’ and ‘Desktop Development with C++’ workloads. You can install these by opening your visual studio installer application, and press Modify. Select the two workloads, and press Modify. Change the settings as shown in the image below, then press Switch platform.

Chapter 03: Importing and configuring the resources

  1. We will use the Mixed Reality Feature Tool to import the mixed reality toolkit in our project. Download the tool here
  2. Unzip the folder to your desktop
  3. Click the MixedRealityFeatureTool.exe to launch the application _PROBLEM 4: after launching, it might ask you to download NET. Follow the link in the pop-up and install the ‘Run desktop apps’ NET version. You can also find it here _
  1. After opening the tool, you see the start screen. Select Start.
  1. Select the three dots. Navigate to the Unity project you just created.
  1. Select Discover Features
  2. Select the + sign besides Mixed Reality Toolkit, to unfold all options. Here, chose Mixed Reality Toolkit Foundations
  1. Also click the + sign beside Platform Support, and select Mixed Reality OpenXR Plugin.
  1. Click Get Features. Then first click validate, you should see a pop-up that says ‘No validation issues were detected’. Click Import, then Improve.
  1. Go to your Unity Editor and click somewhere in a blank area. This will start the unpacking of the packages.
  2. After Unity has imported all the packages, it asks to restart. Select Yes.
  3. After this, the MRTK Project Configuration appears. Scale the window larger, to make sure all information is visible. If the window doesn’t open manually open it in the menu: Mixed Reality > Toolkit > Utilities > Configure Project for MRTK
  4. Click on Unity OpenXR plugin (recommended) to add it to your project.
  1. A welcome to MRTK screen opens. Here, select Show XR Plug-In Management Settings.
  1. The MRTK Project Configurator opens. Make sure you are on the XR Plug-in Management page and that the Universal Windows Platform settings tab is opened (check the windows logo tab). Make sure the Initialize XR on Startup is selected
  1. Select the OpenXR plug-in. Two items appear underneath OpenXR. Select the first one: Mircrosoft HoloLens feature group.
  1. Click the yellow warning sign besides OpenXR. This opens the OpenXR Project Validation window. There are multiple issues, select the button Fix All. After this, one issue remains.
  1. To fix this issue, click the edit button beside the issue description. This opens up the project settings tap of the OpenXR plugin. Underneath Interaction Profiles, click the + button. Here you can choose different types of interactions. Chose Eye gaze interaction profile from the dropdown menu to add it. Click the button two more times to also add: Microsoft Hand Interaction Profile, and Microsoft Motion Controller Profile. Then, for each interaction: if a yellow triangle appears beside them, click the triangle to open the validation window and select the Fix button.
  1. In the Project Settings window, make sure that the Microsoft Hololens, Hand Tracking, and Motion Controller Model are selected.
  1. In the same window, click the Depth Submission Mode dropdown to select Depth 16 Bit.
  1. Close the Project Setting window
  2. In the MRTK Project Configurator window, select Apply Settings. Then, click Next and then Apply.
  1. Now, we will configure the project settings. Select Edit > Project Settings
  1. In the Project Settings window, select the Player tab from the left-side column
  2. Click Publishing Settings, and give a good name for the Package name field. Here, we name it “MRTK-Tutorials-Getting-Started”.
  3. Close the Project Settings window.

Chapter 04: Make a unity scene and configure MRTK

  1. In the menu bar, click File > New Scene
  2. In this window, select Basic (Built-in), and Create.
  1. In the menu bar, select Mixed Reality > Toolkit > Add to Scene and Configure…
  1. As you can see in your Hierarchy window on the left, two elements are added: MixedReatlityToolkit (this obtains the toolkit itself), and MixedRealityPlayspace (this handles the headsets or controllers)
  1. Select the MixedRealityToolkit in the Hierarchy. You should now see DefaultMixedRealityToolkitConfigurationProfile in your inspector on the right.
  1. We can edit this profile, however, the ‘default’ is already optimized for the HoloLens 2.
  2. Save your scene: select File > Save ass… then navigate to: Assets > Scenes to name and save your project

Chapter 05: Interaction Models

The toolkit provides different types of interaction, think about: speech, hands, or controllers. Let's set them up!

  1. Let’s add hand interaction! In Unity you can create static objects. When you give them a script (think about it as a manual that tells them what to do) the user can interact with it to move, scale or rotate the object with their hands.
  2. In the Menu bar, select GameObject > 3D Object > Cube
  3. The default size of the cube is a meter by a meter, let’s scale it down to 20 cm for each side. Select the cube and change the Scale values to X=0.2, Y=0.2, Z = 0.2
  1. The cube is placed at the (0,0,0) coordinate, which is the same place as the user’s head. Change the Position to X = -0.2, Y = 0.1, Z = 0.5
  1. Let’s also change the rotation of the cube, so we can see all sides of it. Change the Rotation to X = 9, Y = 14, Z = 0
  1. Now, we will add the script to the cube so that you can grab the cube in the AR environment. Select the cube in the hierarchy (you probably still have it selected)
  2. Click Add Component, which is visible at the bottom of the Inspector tab. Search for Object Manipulator.
  1. Again, scroll to the bottom of the Inspector, click Add component, and search for the Near Interaction Grabbable script to add it to the cube.

Chapter 06: Time to play (almost)

  1. We are going to test our cube. Click the play button .
  1. Click the three-dot button besides the Inspector. Select Maximize.
  2. Press spacebar, this makes a simulated right hand appear in the view.
  3. Play around with the cube: press the left mouse button to grab the cube and drag it around.
  1. Some additional controls:
    1. Change the view in the scene:
      1. To move the camera forward/left/back/right, press the W/A/S/D keys.
      2. To move the camera vertically, press the Q and E keys.
      3. To rotate the camera, press the right mouse button and then drag.
    2. Simulate hand input:
      1. To enable the simulated right hand, press and hold the space bar. To remove the hand, release the space bar.
      2. To enable the left simulated hand, press and hold the left shift key. To remove the hand, release the key.
      3. To move either hand around the scene, move the mouse.
      4. To move the hand forward or backward, rotate the mouse scroll wheel.
      5. To simulate the pinch gesture, click the left mouse button.
      6. To rotate the hand, press and hold down the space bar + CTRL key (right hand) or left shift key + CTRL key (left hand) and then move the mouse.
    3. Persistent hands
      1. To enable a hand and keep it onscreen without continuing to hold down a key, press T (left hand) or Y (right hand). To remove the hands, press those keys again.

Chapter 07: From Unity to HoloLens

  1. Select File > Build settings…
  2. In the build setting windows: click Add Open Scenes, this will add our scene to the Scenes In Build list.
  1. Click the Build button.
  2. Chose or create a new map where you want to store your build. Click Select Folder, to start the build process. This might take a few minutes!
  3. Now, open the folder where your build is placed. Double-click the .sln file to open it in Visual Studio.
  1. Select the Release and ARM64 settings in Visual Studios. (If you are using the HoloLens 1, pick x86)
  1. Now we will turn on Developer Mode on both our laptop and the Hololens
    1. On our laptop
      1. Go to Settings (Instellingen)
      2. Go to Update and Security (Bijwerken en beveiliging)
      3. Go to For developers (Voor ontwikkelaars)
      4. Enable Developer Mode (Ontwikkelaarsmodus)
    2. On the HoloLens2
      1. Turn the device on and put it on your head.
      2. Log in, and launch the main menu
      3. Go to Settings > For developers > and enable developermodus
  2. Connect to the same Wifi on your device and on the HoloLens
  3. In Visual Studios select Remote Machine.

_PROBLEM: If this option is not visible, you might have an error message in your solution explorer on the right. Installing the needed tools might solve the problem. _

  1. Go to Project > Properties. In the new window, go to Configuration Properties > Debugging.
  2. Under Debugger to launch, select Remote Machine.
  3. Under Machine Name, enter the IP address of the HoloLens
    1. On your hololens, go to: settings > Network and Internet > WIFI . Scroll down all the way to the buttom to HardWare Properties. Here, you will find the IP address.
  4. Set the Authentication Mode to Universal (Unencrypted protocol)
  5. In visual studio: select Debug > Start Without Debugging. This might take a while! The final output should show the following:
  1. Check your output, if it says “Deploy: 0 succeeded” complete the remainder of this step (otherwise continue to the next step). Select Build > Deploy Solution. The app will now be deployed to your Hololens. If it is the first time deploying an app to your hololens, visual studios will ask for a pin:
    1. On your Hololens, go to Settings > Updates and Security > For developers
    2. Select Pair. This will prompt a pin code.
    3. Enter the PIN in Visual Studios
  2. Your deployment is finished is you see the following output:** Deploy: 1 succeeded, 0 failed**
  1. On your HoloLens open the start menu. Find the tile for our new app and select it to open your app!

Coming soon: building an interactive scenery for the HoloLens.