Creating Great AR Experiences - juniverse1103/ARKitStudy GitHub Wiki

Creating Great AR Experiences

Grant Paul, Human Interface Design

AR(Augmented Reality)

  • Reality: In AR, things happen in real world. They happen in the room around you, in the environment you're in, or in your place on the map.
  • Augmented:
    • It can mean augmenting what you know about the world.
    • It can mean getting information that device can understand about the world.
    • It can mean placing virtual things out into the world, giving the virtual this physical context.
  • AR means taking something that is real, and enhancing it, augmenting it.

AR Interfaces and Interactions

  • Getting into AR

    • ARKit needs to understand the world world by moving the device.
    • Guide users to move their device clearly without the text.
    • Use feedback to guide towards success.
    • Apple uses UI which a surface turns into cube when AR is ready.
    • The Guide UI should look like the part of your app.
    • Balance guidance with efficiency: Skip the guidance when user knows how to use

  • Presenting content

    AR experience can be presented in many forms.

    • 2D form
      • Not every AR experiences has to look like looking through your device, seeing a camera preview of the world, then placing objects out into space.
      • You can actually use the information ARKit gets about the world to present a great AR experiences entirely in 2D.
      • Example: "Rainbrow" - play piano with your eybrows, avoid obstacles moving through the keys
    • VR form
      • Use device as a window to VR.
      • No headset required, peripheral vision prevents user from accidentally walking into the wall.
      • Example: "Enter the room" - VR experience of a girl's room created by Red Cross

    Keep text readable

    • Perspective
      • Simplest way to show text is put it into the world.
      • Drawbacks of when you put text in perspective.
        • When you look text from an angle.
        • When you look text from further away.
    • Screen Space
      • Text is always the same size, it's always facing the user.
      • Dosen't have any angle or distance problems.
      • It's still attached to a position in the world.(object or some physical feature)
      • Using Screen Space text is the solution!
      • Example: Apple's "Measure" App's measurement
    • Keep text as minimum
      • It is hard to read texts in AR experience
      • The text will go away when turning the device if it is on AR space.
      • Show details on the display.
    • Transition in and out of AR
      • It's important to have a transition when user is going in or coming out of AR.
      • Transitions can make it really clear what it is the user is looking at.
      • Example: Apple's "Measure" App's transition between measurement icon and view.
    • Transitions preserve identity
      • Transition makes it feel like ther's one version of the object. It makes it feel like that object has its own identity.
      • Treat AR object same as real world object. You can't just make multiple copies.
      • Example: Apple's "Quiklook" App.
        • When switching Object tab to AR tab, the object always stays visible.
        • It doesn't disappear and then come from somewhere else.
        • It should feel like there's one object moving between different parts of your app.

    Summary

    Presenting content

    • ARKit can Power 2D experiences
    • VR is a powerful way to show environments
    • Use screen space text for readable labels
    • Show details and control directly on the display
    • Transition smoothly in and out of AR
    • Give objects sense of identity ans sense of physicality

  • Interactions in the world

    • Touch
      • Touch enables direct manipulation.
      • AR is physical. Objects feel like they are real
      • Use same gestures with iOS
        • Drag to move
        • Multi-touch(Pinch) to scale.
          • Give feedback when scaling
          • Snap back to 100% size with a haptic
        • Twist to rotate
        • Tap Target
          • Use the center of two finger's touch position as a tap target, because it is not easy to land both fingers.
        • Direct manipulation is not enough. Touch is 2D
    • Movement is 3D
      • Moving the device is the primary interaction in AR
      • 2D:
        • Scroll to see more content.
        • Pinch to zoom
      • 3D
        • Move the device to see more content.
        • Get closer or further to zoom
        • It is possible to build totally custom interactions for AR apps with movement. Which can be very natural
      • Swiftshot:
        • To fire a slingshot in Swiftshot, you should move close to that slingshot, pull back and release
        • Moving device is more precise than touch
    • Combine movement and touch
      • Quicklook
      • Touch down to pick up the object and then turn the device and release in a new place.
      • Providing full 3D control of moving device to pick where user is going to place it and move objects to places that user can't see on screen.
      • Keeping the sense of physical interaction.
    • Indirect controls

      Indirect controls: Controls that are flat on the display, not placed in the world and not attached to anything.

      • In the consistent position on the display. User can learn that position.
      • Example "Zombie Gunship AR": Move the device to aim, rest finger above the fire button, and focus on firing.
    • One-handed AR
      • User always have to use at least one hand to control where it is that user is looking.
      • If a developer wants to build a one-handed AR experience, user is going to need to use really reachable controls that are really easy to access.
      • Example "Measure": Measure uses an indirect control. Add point button is at the bottom of the screen and is really reachable spot.
      • Make it easy to engage AR experiences in one hand with indirect controls.

    Summary:

    • Use direct manipulation to give sense of physicality, and physical interaction.
    • Move the device, which is the primary interaction in AR
    • Use reachable indirect controls to focus on the content, rather than direct manipulations such as controls or buttons.

Total Summary:

  • Getting into AR, guiding people down the right path, giving them direct feedback that they are doing the right thing.
  • The ways to present content in 2D and VR, and how to show text readable with screen space and details on display.
  • Give objects that physical identity transitioning in and out of AR
  • Interactions that users can use in AR experience with each touch, moving and combined interaction.