2. Introduction - GachonCapstoneHCI/pingpong_score GitHub Wiki

As the interest in healthcare, many manufacturers have produced various kind of wearable devices, such as Samsung Gear Fit Pro2, Apple Watch3, Garmin Vivoactive, etc. However, the abandonment rate of smartwatch and fitness tracker is 29 and 30 percent, respectively [Susan Moore, 2016]. The users mostly said those products are not useful enough. In addition, those trackers are mostly focused on personal activities and don’t support group activities. Eric O.Boyer and his team proposed low-cost motion sensing of table tennis players [Eric Boyer, 2013]. They attached sensors on the racket and used Gaussian low pass filter to analyze data in real time. However, they had to tag manually in the graph to classify types of swings. This is not an appropriate way to classify swing types in real time. Other researchers used Kinect to detect a user’s swing action [Geon Heo, 2015, Liming Zhang et al., 2016]. They read 3D information of a user’s action and converted the information into spherical coordinates. Then, through LDA coordinates reduction and k-means clustering, they extracted feature points of the action and classified the swings into 2 types by using the HMM. The study showed 80.2% of accuracy. However, that study was hard to be applied in the real game because all players had to stand in front of Kinect. To enhance usefulness of wearable device and reduce the cost for exercise, we propose a system, Automatic Table Tennis Scoring System(ATTSS).

The proposed ATTSS consists of three steps: swing recognition based on smartwatches, ball sound detection based on smartphones, and scoring algorithm based on table tennis rules.

First, for swing detection, we collect accelerometer and gravity sensor signal from the smartwatch and develop a pattern recognition algorithm to detect whether a user performs a swing or not. From the preliminary study, we find out that x and y represent an event of swing action. To see the integrated force of accelerometer, Signal Vector Magnitude (SVM) was applied on x and y value. The reason z value was not included is that z value not only represented little difference but also affected accuracy of swing detection algorithm. After SVM, we used thresholding 150 and made all value below the threshold to zero. We also used peak detection algorithm to extract only one peak from a cluster of peaks in the graph: other peaks in a certain window size (e.g. 10) were ignored before a next swing. In addition, to differentiate a swing with other actions, gravity sensor z value was used. Figure 1 below is a result of 19 swings. The left in Figure 1 shows that raw x, y and z value of accelerometer: red is y, purple is x and green is z. The right in Figure 1 shows that the result of swing recognition algorithm.

Secondly, system detects sound of ball dropping on the table based on sound signal

with the microphone of a smartphone device. The microphone in the smartphone was used to collect sound data to determine whether a ball was dropped on the table. The original sound signal collected is time domain data in 22,050Hz. To see the range of frequency of a ball sound, Fast Fourier Transform algorithm was applied and converted time domain into frequency domain with a total block size 512. Amplitude from block No. 180 ~ block No.270 (7,751Hz ~ 11,627Hz) became higher than other sound frequency. We determined if a ball was dropped on the table by using this algorithm: check whether amplitude from block No.180 ~ block No.270 exceeds a specific threshold (e.g. |13|), and check whether amplitude between 80 and 180 exceeds a specific threshold (e.g. |3|).

Finally, a referee smartphone device collect all the analyzed data from each player, and determines a winner or a loser based on table tennis scoring algorithm. The winner could be determined by checking an event occurring while playing game. The event is swing event and ball-dropping event. In a general case, there are a sequence of events: Player A swings the ball  Ball drops on Player B table  Player B swings the ball  Ball drops on Player A table  Player A swings the ball  and so on. Based on a sequence of events, each player has an array of expected events. Each player compares ‘now-event’ with ‘expecting- event’. If it is equal, they keep playing, otherwise, each player checks whose event it was in the previous event. If it was A, B is winner and vice versa. To see if the algorithm works, we watched the video clip of a real game, Ryu (South Korea) vs Wang Liqin (China), on YouTube and recorded the sequence of events for an input data. The scoring algorithm produced a correct output,11:6, for the input data. Figure 2 above is implementation of each function.