(Left) The wrist and fingertips are moving in the same trajectory, (right) The wrist and fingertips are moving different
The AIRTEXT: One-Handed Text Entry In The Air For Cots Smartwatches, developed by a research team at Zhejiang University in China, is a smart watch that does not require a surface and can write letters by moving fingers (index finger) in the air. [Image] Display four prediction conversions, select a gesture that tilts the wrist. Only the IMU measured value obtained from the built -in acceleration sensor or gyroscope attached to the smartwatch attached to the wrist, text written with your fingertips in the air. Estimates with a framework using deep learning. Research has been conducted to detect hands with devices attached to the wrist. It was possible to detect the wrist in the same trajectory for the wrist writing with the finger, but it was difficult to accurately capture the movement of the fingertips in a situation where the movement of the wrist and fingers was different. In the study, even in situations where the movement of the wrist and fingers is different, we propose a method of inputting only the IMU measurement value from the smartwatch, and using a deep learning to classify the fingertips and classify the text. This method consists of three stages, a network that estimates the predation and the fingertip trajectory, and a network that classifies characters. In the first stage precaution, the user uses the fact that the acceleration distributed value differs greatly between the use of the letters and not, and is extracted only when handwriting. In the second stage, the three -dimensional position of the three tracking points of the index finger, the second middle finger joint, and the wrist are output to estimate the trajectory of the fingertips from the IMU data after the pre -treatment. It incorporates an algorithm to remove excess stroke interference from the previous character to the current character. In the third stage, the character is classified by another deep learning model, with the estimated fingertips and pre -processed IMU data. It takes time to enter words and sentences to classify one character at a time. Therefore, we adopt a method of presenting several words candidates to users that can be inferred from the classified characters. Four words are displayed up, down, left and right on the smartwatch screen, and users operate and select the wrist lightly from these words candidates. In the experiment, it was implemented in five commercially available smartwatches (such as LG Watch Urbane and Huawei Watch2pro) and evaluated its performance widely. Eight subjects participated in the evaluation and handwritten more than 25,000 characters in the air. As a result, the average typing speed of 8.1 words per minute was achieved, and the average word error rate was 3.6 % to 11.2 %. Exceeding the two baselines, the character input speed of the same character equivalent to the approach using both hands has been achieved. As a limit, it is necessary to make a short pause in the spell of individual characters, and when the body is moving while walking, the noise is large and poor accuracy, so in a state where the body is still stationary. It is used. Source and Image Credits: Y. Gao, S. Zeng, J. Zhao, W. Liu and W. Dong, “AirText: One-Handed Text Entry in the Air for COTS Smartwatches,” in IEEE Transactions on Mobile Computing, doi: 10.1109/tmc.2021.3130036. Mr. Yamashita picks up and explains a highly new nature paper.