Overall

• app.py was tested on Debian based distributions.
• We configured Landscape and Travis CI.
• We created a tool (app/tools/matrixanalyser.py) to visual the density of events being recorded by the touchpad.

Components

app/app.py

• The app can launch programs.

The executor module launches x-www-browser which is a default web browser on the user's system.

• The app can recognize symbols drawn on the touchpad.

The symbols must be either hardcoded or previously taught by the user during a learning session.

• One hardcoded symbol was added and is recognised by the app.

The K letter as the hardcoded symbol for both 32-bit and 64-bit architectures.

The user has to start the app with either --system 32 or --system 64 arguments to start the app with the hardcoded symbol.

• The user is able to draw their own symbols and teach them to the app.

They just have to start the app with the -l SIZE and draw the symbol SIZE many times. Then they are able to use the symbol simply running sudo ./app.

src/touchpadlib.h

• Implementation of the automatic detection of that the touchpad parameters.

The interface for this functionality was added to the src/touchpadlib.h. The interface in Python was implemented in the app/touchpadlib/touchpadlib.py file. It is used by the app/tools/matrixanalyser.py.

• Automatic detection of the touchpad device.

The user is no longer prompted to choose the touchpad device for the list of all available devices in the /dev/input.

app/classifier/classifier.py

• This is a tool for learning and classifying drawn symbols.

• Learning mode.

In that case, the tool waits for user to draw the symbol the given number of times. Then it undertakes a learning session.

• Classifying mode.

In this case, it compares the drawn symbol to the defined symbols. It returns the probability which indicats if the drawn symbol might be one of the defined symbols.

• For now, there is only the possibility of comparing to exactly one symbol.

It can be either a hardcoded letter "k" (while running app with the -s argument) or the user-defined symbol which can be provided in the learning mode.

• For learning and similarity measuring we use scikit-learn library, specifically the nearest neighbour methods.

app/classifier/featureextractor.py

• The normalization of the symbol.

Changes the format in which the set of points is saved, so that it could be easily compared with other symbols to determine if they are similar or not the similarity

It is done in 3 steps:

• Firstly, the program extracts the values of interest from the given set of points.

The values are:

• The center of mass;
• The length of the line;
• The colors of the points.

Those values determine if the point represents a real point consisting of the x and y coordinates or if it is a special point which signalizes that the user raised a finger off the touchpad.

• Subsequently, the program creates a new set of points.

Those points are equidistant from each other and scaled properly with respect to both the point 0,0 in the center of mass and the constant value of the difference between max/min coordinates.

• In the end it calculates other necessary features from the new set of points.

The current implementation involves:

• Calculating the angles between every line and the x axis
• Creating a new list of points.

The feature list used in the machine learning algorithm is a joint list of coordinates, angles and colors.

Tests

Tests were added to all important functionalities:

• Test for learning and classifying for the classifier module.
• Test for all calculation done in the featureextractor module.
• Test for connecting and loading of the touchpadlib module.
• Test for the signalcollection module.
• Test for touchpadsignal structure and its functionalities.