In view of the fact that the orbit state is still judged by artificial experience and low recognition efficiency, a track recognition method based on image processing and multi vision feature fusion is proposed. Firstly, the rail contact area image is segmented from the collected rail surface images under different conditions, and the noise interference is removed by preprocessing the contact area; then, the gray mean value and variance of the contact area are calculated to describe the color features of the rail surface image, with the gray level co-occurrence matrix used to extract the texture features of the contact area; then, the two features are fused as the basis for judging the rail surface state SVM is used to identify the state of rail surface. Finally, the simulation and validation of the proposed method are carried out, and the results verify the effectiveness of the proposed method.