Surface Electromyography-based Gesture Recognition by Multi-view Deep Learning.

Gesture recognition using sparse multichannel Surface Electromyography (sEMG) is a challenging problem, and the solutions are far from optimal from the point of view of Muscle-Computer Interface (MCI). In this work, we address this problem from the context of multi-view deep learning. A novel multi-view Convolutional Neural Network (CNN) framework is proposed by combining classical sEMG feature sets with a CNN-based deep learning model. The framework consists of two parts. In the first part, multi-view representations of sEMG are modeled in parallel by a multistream CNN, and a performance-based view construction strategy is proposed to choose the most discriminative views from classical feature sets for sEMG-based gesture recognition. In the second part, the learned multi-view deep features are fused through a view aggregation network composed of early and late fusion subnetworks, taking advantage of both early and late fusion of learned multi-view deep features. Evaluations on 11 sparse multichannel sEMG databases as well as 5 databases with both sEMG and Inertial Measurement Unit(IMU) data demonstrate that our multi-view framework outperforms singleview methods on both unimodal and multimodal sEMG data streams.