Cookbook: Using a touchscreen under Linux - Psychtoolbox-3/Psychtoolbox-3 GitHub Wiki
Introduction
PTB contains a set of Touch* functions (e.g. TouchQueueCreate) that can handle many different types of touchscreens on Linux systems. To use a touchscreen takes a bit of setting up, especially if you want to use separate X-screens (e.g. MATLAB command window on screen 0 and subject stimulus on screen 1). More implementation details can be found by typing help TouchInput in MATLAB.
Example: how to setup a touchscreen on a dual-display Ubuntu system
This example is based on a 21.5" touchscreen from a Chinese manufacturer (available overseas via Alibaba), that uses a capacitive-touch ILITEK panel. The GPU on the system is an AMD WX5100 with 4 DP ports. We plugged in the touchscreen using DisplayPort-3 (the main monitor is DisplayPort-0), and connected the USB cable that is used for communication. Next, set up PTB in MATLAB using XOrgConfCreator and XOrgConfselector to make two separate X-screens, one for my MATLAB session and one for the touch screen, here are the xrandr results for each screen:
➜ DISPLAY=:0.0 xrandr
Screen 0: minimum 320 x 200, current 5120 x 2880, maximum 16384 x 16384
DisplayPort-0 connected primary 5120x2880+0+0 (normal left inverted right x axis y axis) 527mm x 296mm
➜ DISPLAY=:0.1 xrandr
Screen 1: minimum 320 x 200, current 1920 x 1080, maximum 16384 x 16384
DisplayPort-3 connected primary 1920x1080+0+0 (normal left inverted right x axis y axis) 377mm x 212mm
Next, we checked whether xinput (the system that manages input devices under x windows), can see the touchscreen:
➜ xinput
⎡ Virtual core pointer id=2 [master pointer (3)]
⎜ ↳ Virtual core XTEST pointer id=4 [slave pointer (2)]
⎜ ↳ M585/M590 Mouse id=17 [slave pointer (2)]
⎜ ↳ ILITEK ILITEK-TP Mouse id=9 [slave pointer (2)]
⎜ ↳ ILITEK ILITEK-TP id=8 [slave pointer (2)]
⎜ ↳ PixArt Dell MS116 USB Optical Mouse id=13 [slave pointer (2)]
OK, the ILITEK ILITEK-TP is present in the list. You can get more information on the device using xinput list-props:
➜ xinput list-props "ILITEK ILITEK-TP"
Device 'ILITEK ILITEK-TP':
Device Enabled (147): 1
Coordinate Transformation Matrix (149): 1.000000, 0.000000, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 1.000000
libinput Calibration Matrix (304): 1.000000, 0.000000, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 1.000000
libinput Calibration Matrix Default (305): 1.000000, 0.000000, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 1.000000
libinput Send Events Modes Available (267): 1, 0
libinput Send Events Mode Enabled (268): 0, 0
libinput Send Events Mode Enabled Default (269): 0, 0
Device Node (270): "/dev/input/event7"
Device Product ID (271): 8746, 1
More information can also be found listing /proc/bus/input/devices
➜ less /proc/bus/input/devices
I: Bus=0003 Vendor=222a Product=0001 Version=0110
N: Name="ILITEK ILITEK-TP Mouse"
P: Phys=usb-0000:00:14.0-7.4/input1
S: Sysfs=/devices/pci0000:00/0000:00:14.0/usb1/1-7/1-7.4/1-7.4:1.1/0003:222A:0001.0005/input/input8
U: Uniq=V06.00.00.00
H: Handlers=mouse2 event8 js0
B: PROP=0
B: EV=1b
B: KEY=1f0000 0 0 0 0
B: ABS=3
B: MSC=10
I: Bus=0003 Vendor=222a Product=0001 Version=0110
N: Name="ILITEK ILITEK-TP"
P: Phys=usb-0000:00:14.0-7.4/input0
S: Sysfs=/devices/pci0000:00/0000:00:14.0/usb1/1-7/1-7.4/1-7.4:1.0/0003:222A:0001.0004/input/input10
U: Uniq=V06.00.00.00
H: Handlers=mouse1 event7
B: PROP=2
B: EV=1b
B: KEY=400 0 0 0 0 0
B: ABS=260800000000003
B: MSC=20
The next step is to calibrate the touchscreen. In Ubuntu we first need to install the calibration tool:
➜ sudo apt-get install xinput-calibrator
Then we need to use the DISPLAY environment variable to identify the correct X-screen within which to run the configurator. Not sure why but we had to disable mis-click detection using --misclick 0 as otherwise calibration of the bottom two positions could not complete.
➜ DISPLAY=:0.1 xinput_calibrator -v --misclick 0 --device "ILITEK ILITEK-TP"
Calibrating standard Xorg driver "ILITEK ILITEK-TP"
current calibration values: min_x=0, max_x=65535 and min_y=0, max_y=65535
If these values are estimated wrong, either supply it manually with the --precalib option, or run the 'get_precalib.sh' script to automatically get it (through HAL).
--> Making the calibration permanent <--
copy the snippet below into '/etc/X11/xorg.conf.d/99-calibration.conf' (/usr/share/X11/xorg.conf.d/ in some distro's)
Section "InputClass"
Identifier "calibration"
MatchProduct "ILITEK ILITEK-TP"
Option "MinX" "6465"
Option "MaxX" "73935"
Option "MinY" "16096"
Option "MaxY" "72528"
Option "SwapXY" "0" # unless it was already set to 1
Option "InvertX" "0" # unless it was already set
Option "InvertY" "0" # unless it was already set
EndSection
Once completed you get an xorg.conf section. I created a new file /etc/X11/xorg.conf.d/99-calibration.conf as recommended with that section (note your calibration values will be different). Note if you want to run the calibration again, you have to make sure you reset the --precalibvalues back to the original ones (e.g.DISPLAY=:0.1 xinput_calibrator -v --misclick 0 --device "ILITEK ILITEK-TP" --precalib 0 65535 0 65535`). Then you must log out and back in or reboot and then start MATLAB.
To test if PTB can see the touchscreen:
>> [idx,name,info]=GetTouchDeviceIndices
idx =
10
name =
1×1 cell array
{'ILITEK ILITEK-TP'}
info =
1×1 cell array
{1×1 struct}
Good, we can get the index to the touchscreen. Then you can run MultiTouchDemo to hopefully see the touchscreen in action, it works fine tracking up to 10 positions.
Confirming latency
The KeyboardLatencyTest script, among various other input devices, can also test touch-input timing if one uses the modality parameter 10. This is via detecting hard taps – and the associated bump noise – via microphone + PsychPortAudio audio onset timestamping. Obviously you’ll get the combined touch timing and audio timing error as an upper bound for timing precision / touch latency.
Touchscreens known to work with Linux
- Zhixianda GC215 / Ilitek 21.5" industrial touchscreen - tested to work with dual-Xscreens.
- iiyama ProLite t2735.
- Dell 24 Touch Monitor - P2418HT.