Design of neural network classifiers for pattern recognition applications

Abstract

Pattern recognition techniques are among the important tools used in the field of machine intelligence. Neural Networks is a computational paradigm inspired by human’s richest sense - vision and perception. Pattern recognition is the principal activity in all real time applications. Decision - making processes of human being are related to the recognition of patterns. In this thesis work, neural network classifiers are designed for pattern recognition applications. The task of recognition of handwritten characters is chosen as an application domain for testing the strengths and weaknesses of the proposed Neural Network Classifiers. The conventional Backpropagation algorithm for training Feedforward Neural Networks is modified with an objective of quick training. Adaptive Backpropagation algorithm, which dynamically adapts the learning constants with the adaptation cycles and the error is proposed. The error is found to decrease faster than in conventional algorithms. The performance of the algorithm is proved with the classification of Iris data set and recognition of handwritten characters. The network size plays a major role in learning and generalization. Adaptive Genetic Algorithms are used to determine the optimum size of the network. The redundant connections that contribute little or nothing are removed based on the sensitivity of the edges to the error. 11 The problem solving capability of a Neural Network can be increased by the combination of multiple networks. The multiple networks are powerful solutions to recognition problems because they make the individual networks to learn independently the input from different point of view. The recognition accuracy of such systems depends not only on the prediction ability of the individual networks but also on the way in which their outputs are combined. A Fuzzy combiner is designed to derive a collective decision from the outputs of the individual networks. This has enhanced the overall recognition rate of the system. A Dynamic self-organizing network for pattern recognition applications is proposed. It dynamically creates neurons in the output layer and builds up the network. Neurons are created whenever a new category of patterns is presented. The winner is determined based on minimum Euclidean distance and its weights are updated accordingly. The neuron that wins for multiple classes are split in such a way that each neuron represents a unique class. This adaptive process is repeated until all patterns are learnt. The performance of Dynamic Self-Organizing Map is demonstrated through recognition of handwritten characters. The time-dependent selective competition is introduced in the Self- Organizing Maps with the help of gated neurons. The gate input of the neuron decides its output. The output is the distance measured when the gate input is non-zero and the output is zero otherwise. A gated neuron classifier is Ill designed using such gated neurons. The performance of the gated neuron classifier is explained with recognition of handwritten characters. The directional features of the input patterns are extracted using Kirsch masks. These features are fed to the input at distinct time intervals. The network records the winning instance of each neuron of various time intervals in a register. Each of these instances is connected to the output class nodes. The strength of these connections are based on the strength of belief / disbelief of the individual network. The label of the class node with maximum output denotes the class of the test sample. The pattern is misclassified / rejected when more than one output neuron produces the same maximum value or when the output of all neurons is zero. A Fuzzy Multilayer Neural Network is proposed along with its supervised learning algorithm. The network is devised with four layers of Fuzzy Neurons. The inputs are first fuzzified. The network remembers the ' minimum and maximum values in all dimensions of a class of inputs as comers of an n - dimensional hyperbox. When additional inputs of same classes are presented the corresponding hyperbox is suitably expanded to include the new patterns also. This process may cause a hyperbox of one class to overlap with that of other classes. In that case they are contracted to minimize the overlap. During testing, the degree to which the pattern is contained inside the box indicates the membership value. The box with maximum membership value labels the pattern. IV Finally, a Genetic Algorithm based Feature extraction scheme is . proposed to extract unique features of the classes and initial population is set with random features. A feasibility check is performed to find whether all these random features are contained within the pattern. The quality of the features is evaluated by its information content. The probability of a pattern belonging to a class and not belonging to other classes is used to measure the information content of the feature. This uses the approximation of Baye’s rule. The pixel sum images of a subset of input patterns are obtained for every class. A feature detector is located in the pattern and compared with the digit image. The genetic operators are applied to the individuals to get better quality features in successive generations. This method of feature extraction provides the advantages of dimensionality reduction. An application package for recognition of handwritten and machine - printed samples using the proposed network models is developed and named as “Rekogonizee”. This is developed using Visual C++ with a user-friendly • menu based interface. It has options for scanning the input patterns, testing and training the network. It has got features to extract the directional features from the input and train the network. Help for various options are provided through HTML files. This package can be used effectively for recognition of handwritten and machine printed numerals and alphabets.