Visual detection of surface defects is one of the important aspects of product quality inspection. However, in practical industrial applications, in view of such flaws as few surface defect samples and the need for a large number of labeled samples, attention mechanism is introduced into Resnet50 network, with a positive sample-based visual inspection method proposed for product surface defects. First, the pre-training network Resnet50_CBAM is adopted to acquire the embedding vectors containing information from different semantic layers and resolutions, with the multivariate Gaussian parameters used for a representation of the normal features of the images. Next, the defective images are input to the pre-training network Resnet50_CBAM, thus obtaining the corresponding embedding vectors and multivariate Gaussian parameters. Finally, the Markov distance is used to work out the defective scores of all pixels in the whole defective image, so as to realize the defect area location based on pixel level. The experimental data set verification results show that compared with the existing methods, this proposed method is characterized with a low requirement of normal samples yet with a higher detection accuracy, which can effectively solve the problem of visual inspection of product defects with fewer samples.