Sparse Connectivity and Activity Using Sequential Feature Selection in Supervised Learning

Generally, in neural networks the sparseness is a suitable regularizer in a lot of applications. In this paper, sparse connectivity and sparse representation are used to enhance solutions to the problem of classification. Sequential feature selection is then leveraged to remove redundant features and select relevant ones. Sparseness-enforcing projection operator is used to discovering the most similar vector with a predefined sparseness degree for any input vector as well. As it will be argued, the mentioned operator is approximately differentiable at every point. From the facts it is clear that the sparseness enforcing projection would be appropriate for use as a transfer function in the proposed neural network and the network can be tuned using gradient based methods. Meanwhile, an intelligent method was used to build the architecture of the proposed neural network to achieve better performance. The MNIST dataset which consists of 70,000 handwritten digits was used to train and test the method and 99.18% accuracy was achieved by classifying this dataset.