卷积神经网络的结构也会对其性能造成影响,设计卷积神经网络更多的是依靠经验和强大的算力,如何设计出性能更好的卷积神经网络目前缺少有效的理论支撑。为了解决这一问题,在分析典型卷积神经网络拓扑复杂性的基础上,为快速实现满足给定复杂性特征的卷积神经网络,给出了由复杂网络拓扑到卷积神经网络的生成算法,通过建立系列不同拓扑特征的卷积神经网络,采用Cifar10和Cifar100数据集分析了平均聚集系数、平均路径长度、图密度、模块度等拓扑性质对卷积神经网络识别有效性的影响关系。实验表明在神经网络的参数数量基本相等的情况下,平均聚类系数会对卷积神经网络的性能产生影响。最终得到结论在统计意义上,平均聚集系数小的网络结构会有更好的性能表现,这为进一步设计出更好的卷积神经网络提供了理论依据。
The structure of the convolutional neural network will also affect its performance. The design of the convolutional neural network relies more on experience and powerful computing power. How to design a neural network with better performance lacks effective theoretical support. In order to solve this problem, based on the analysis of the complexity of the typical convolutional neural network topology, in order to quickly realize the convolutional neural network that meets the given complexity characteristics, the generation from complex network topology to convolutional neural network is given. The algorithm, through the establishment of a series of convolutional neural networks with different topological features, uses the Cifar10 and Cifar100 data sets to analyze the relationship between the average clustering coefficient, average path length, graph density, modularity and other topological properties on the recognition effectiveness of the convolutional neural network. Experiments show that when the number of parameters of the neural network is basically equal, the average clustering coefficient will affect the performance of the convolutional neural network. The final conclusion is that in a statistical sense, a network structure with a small average clustering coefficient will have better performance, which provides a theoretical basis for further designing a better convolutional neural network.
[1]Simard D, Nadeau L, Kroger H.Faster learning in small-world neural networks[J].Physics Letters A, 2005, 336(1): 8-15.
[2]Xie S, Kirillov A, Girshick R, et al. Exploring randomly wired neural networks for image recognition[DB/OL].[2020-03-20]. https://arxiv.org/pdf/1904.01569.pdf.
[3]Lécun Y, Bottou L. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11):2278-2324.
[4]Alom M Z, Taha T M, Yakopcic C, et al. The history began from alexnet: a comprehensive survey on deep learning approaches[DB/OL].[2020-03-20].https://arxiv.org/ftp/arxiv/papers/1803/1803.01164.pdf.
[5]Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[DB/OL].[2020-03-20]. https://arxiv.org/pdf/1409.1556v6.pdf.
[6]Szegedy C, Ioffe S, Vanhoucke V, et al. Inception-v4, inception-resnet and the impact of residual connections on learning[DB/OL].[2020-03-20].https://arxiv.org/pdf/1602.07261v1.pdf.
[7]He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[DB/OL].[2020-03-20]. https://arxiv.org/pdf/1512.03385.pdf.
[8]Huang G, Liu Z, Van Der Maaten L, et al. Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA,2017: 4700-4708.
[9]Zoph B, Vasudevan V, Shlens J, et al. Learning transferable architectures for scalable image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA, 2018: 8697-8710.
[10] Eggemann N, Noble S D. The clustering coefficient of a scale-free random graph[J]. Discrete Applied Mathematics, 2011, 159(10): 953-965.
[11] Newman M E. The structure and function of complex networks[J].Siam Review,2003,45(2): 167-256.
[12] Newman M E J, Girvan M. Finding and evaluating community structure in networks[J]. Physical Review E, 2004, 69(2): 026113.
[13] Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]//International Conference on Machine Learning. Miami, Florida, USA, 2015: 448-456.
[14] Xu L, Choy C, Li Y W. Deep sparse rectifier neural networks for speech denoising[C]//2016 IEEE International Workshop on Acoustic Signal Enhancement. Xi'an,China, 2016: 1-5.
[15] Kingma D P, Ba L J. Adam: a method for stochastic optimization[J]. [DB/OL].[2020-03-20]. https://arxiv.org/pdf/1412.6980v9.pdf.
