Interaction - Griffith-ICT/1701ICT-Creative-Coding GitHub Wiki

Computers by their very nature are interactive:

• Input
• Process
• Output

Without input computers provide very little function.

However there are different types of interaction.

Initially computers had very physical types of interaction based around switches, on or off.

Later input devices provided proportional or progressive input like game paddles and the mouse.

These forms of input are still not 'natural' for humans.

In user interface design there is a concept of 'direct interaction'. For example instead of choosing a menu item to close a window, you simply click directly on the close button. Instead of typing a command to move a file from location to another, you click and drag the file to its new location. It could be argued that any form of mouse usage is not as direct as it can be. A touch screen is more direct as the finger is in the same vicinity as the item being interacted with. It is more intuitive and requires less training.

There are various forms of touch screens and they have their advantages and disadvantages. The most common currently is the capacitive glass touch screen which supports multiple touch and doesn't require the screen to pressed (just touched).

The iPhone popularised the use of capacitive touch screens and also multi-touch. Apple's early work on the iPhone introduced new forms of interaction such as pinch-to-zoom and swiping. These forms of interaction are also now available on multi-touch trackpads. Other manufacturers have touch screen desktops and laptops. Apple now has an OLED touchbar above the keyboard on some laptops.

From a programming perspective tracking direct touch is not overly different to tracking mouse movements and clicks. However touch also has the concept of a gesture which can become very complex. For example, when does a two finger pinch become a two finger swipe if the fingers change ever so slightly? There are also algorithms to ignore spurious input like the palm of the hand resting on the display.

In this course we will be primarily focussed on mouse and touch interaction.

Physical Interaction

There are however other forms of interaction which allow the user to break away from the constraints of the screen.

Special USB switches can be connected to more artistic forms of interaction. Changes in these USB devices can be detected and allow for the program to respond.

Alternative forms of output

We've mostly talked about input but there are other forms of output apart from the screen and sound.

An interesting and shocking example is that of a program that causes a real gun to fire (in an enclosed transparent box), every time a murder occurs in the US. The 'input' to the program is an API which provides realtime data on murders.