A classification method based on binary convolutional neural networks has been proposed in view of some problems in current gesture classification. Based on the characteristics of neural networks, which can keep a relatively high degree of accuracy and robustness in classification even under a low precision, a proposal has been made of a new network structure with the traditional high-precision classification method of convolutional networks and the binary classification method combined together. In the process of the experiment, a research has been conducted on the effect of hidden layer parameters on the hand gesture classification, followed by a comparison between the classification performance and the operational efficiency of the conventional classification methods. The experimental results show that the proposed method has the best performance when N=512. Compared with other methods, its computational efficiency has been significantly improved, with its error rate close to the best result.