Picking allows for finding what the hand is pointing at (the bulk hand, not the index finger)

Basic picking

Picking is supported when hands are bound, starting with an already bound hand picking can be done as follows;

hand.pickBulkHand(node)

This will return a standard CollisionResults that can be iterated through to find the object of interest.

Note that the hand model itself will have user data "noPick" = true, it may be sensible to ignore any picked geometies with this userdata

Lemur clicking

IF lemur is on the classpath then additional functionality is available.

Lemur clicking can be configured as follows

FunctionRegistration functionRegistration = setClickAction_lemurSupport("/actions/main/in/trigger", nodeToPickAgainst)

This is likely to be in response to an action but that is not mandatory. (Note see https://github.com/oneMillionWorlds/Tamarin/wiki/Action-Based-Vr for info on registering "/actions/main/in/trigger")

The node should be a node that contains all the GUI geometries, in simple applications it could just be the root node but better performance will be experienced if a subset of the scene is included.

When the action (in this example "/actions/main/in/trigger") becomes true tamarin will simulate a click on the node for Lemur actions. Some Lemur components (like textfields and drop downs) have special Tamarin support to make them work nicely in VR; e.g. clicking on a textfields will open a hovering keyboard.

Creating a 3d lemur gui

Lemur is a GUI library that supports 3d and 2d GUIs, for a VR environment only 3d GUIs are relevant. Note that as 2D GUIs are more common the default lemur size is sensible for 2D, for 3D they should be scaled down considerably, this can be done as follows

    Container lemurWindow = new Container();
    lemurWindow.setLocalScale(0.005f);

An example lemur menu might be created and positioned as follows

    Container lemurWindow = new Container();
    lemurWindow.setLocalScale(0.005f); //lemur defaults to 1 meter == 1 pixel (because that make sense for 2D, scale it down, so it's not huge in 3d)
    Label label = new Label("Example application using Tamarin & Lemur");
    label.addMouseListener(new MouseListener(){
        @Override
        public void mouseButtonEvent(MouseButtonEvent event, Spatial target, Spatial capture){
            System.out.println("Tamarin supports mouse listener click events (but only the fact they happened, no other details)");
        }
        @Override public void mouseEntered(MouseMotionEvent event, Spatial target, Spatial capture){}
        @Override public void mouseExited(MouseMotionEvent event, Spatial target, Spatial capture){}
        @Override public void mouseMoved(MouseMotionEvent event, Spatial target, Spatial capture){}
    });
    lemurWindow.addChild(label);

    Button movingBlocksExampleButton = lemurWindow.addChild(new Button("Start block moving example"));
    movingBlocksExampleButton.addClickCommands(source -> {
        System.out.println("Tamarin supports clicks on buttons");
    });

    lemurWindow.setLocalTranslation(-0.5f,1,7);

    rootNode.attachChild(lemurWindow);

Pick pointer

When using a lemur gui its a good idea to show the user where they are pointing. There are 2 techniques that can be used for this (often both used together)

Adding a picking line

@Override
public void simpleInitApp(){
    FunctionRegistration functionRegistration = boundHand.attachPickLine(pickLine());
    .....
}

private Spatial pickLine(){
    float length = 1;

    Cylinder pickCylinder = new Cylinder(10,10, 0.002f, length, true);

    Geometry geometry = new Geometry("debugHandLine", pickCylinder);
    Material material = new Material(getApplication().getAssetManager(), "Common/MatDefs/Misc/Unshaded.j3md");
    material.setColor("Color", new ColorRGBA(0.4f,0.4f,0.4f,0.5f));
    geometry.setMaterial(material);

    Quaternion rotate = new Quaternion();
    rotate.fromAngleAxis(FastMath.HALF_PI, Vector3f.UNIT_Y);
    geometry.setLocalRotation(rotate);
    geometry.setLocalTranslation(length/2, 0,0);
    geometry.getMaterial().getAdditionalRenderState().setBlendMode(RenderState.BlendMode.Alpha);
    geometry.setQueueBucket(RenderQueue.Bucket.Transparent);

    return geometry;
}

Or adding a pick marker. A pick marker shows a white sphere where the click would happen if a click were to be performed now

FunctionRegistration functionRegistration = boundHand.setPickMarkerContinuous(node);

The node should be a node that contains all the GUI geometries, in simple applications it could just be the root node but better performance will be experienced if a subset of the scene is included