Design of neural network classifiers for pattern recognition applications

Abstract

Patternrecognitiontechniquesareamongtheimportanttoolsusedinthefieldofmachineintelligence.NeuralNetworksisacomputational paradigminspiredbyhuman’srichestsense-visionandperception.Pattern recognitionistheprincipalactivityinallrealtimeapplications.Decision-makingprocessesofhumanbeingarerelatedtotherecognitionofpatterns.Inthisthesiswork,neuralnetworkclassifiersaredesignedforpatternrecognitionapplications.Thetaskofrecognitionofhandwrittencharactersischosenasan applicationdomainfortestingthestrengthsandweaknessesoftheproposedNeuralNetworkClassifiers.TheconventionalBackpropagationalgorithmfortrainingFeedforwardNeuralNetworksismodifiedwithanobjectiveofquicktraining.Adaptive Backpropagationalgorithm,whichdynamicallyadaptsthelearningconstantswiththeadaptationcyclesandtheerrorisproposed.Theerrorisfoundtodecreasefasterthaninconventionalalgorithms.Theperformanceofthe algorithmisprovedwiththeclassificationofIrisdatasetandrecognitionofhandwrittencharacters.Thenetworksizeplaysamajorroleinlearningand generalization.AdaptiveGeneticAlgorithmsareusedtodeterminetheoptimumsizeofthenetwork.Theredundantconnectionsthatcontributelittle ornothingareremovedbasedonthesensitivityoftheedgestotheerror. 11TheproblemsolvingcapabilityofaNeuralNetworkcanbeincreased bythecombinationofmultiplenetworks.Themultiplenetworksarepowerfulsolutionstorecognitionproblemsbecausetheymaketheindividualnetworks tolearnindependentlytheinputfromdifferentpointofview.Therecognition accuracyofsuchsystemsdependsnotonlyonthepredictionabilityofthe individualnetworksbutalsoonthewayinwhichtheiroutputsarecombined.AFuzzycombinerisdesigned toderiveacollectivedecisionfrom theoutputs oftheindividualnetworks.Thishasenhancedtheoverallrecognitionrateofthesystem.ADynamicself-organizingnetworkforpatternrecognition applicationsisproposed.Itdynamicallycreatesneuronsintheoutputlayerandbuildsupthenetwork.Neuronsarecreatedwheneveranewcategoryofpatternsispresented.ThewinnerisdeterminedbasedonminimumEuclidean distanceanditsweightsareupdatedaccordingly.Theneuronthatwinsfor multipleclassesaresplitinsuchawaythateachneuronrepresentsauniqueclass.Thisadaptiveprocessisrepeateduntilallpatternsarelearnt.TheperformanceofDynamicSelf-OrganizingMapisdemonstratedthrough recognitionofhandwrittencharacters.Thetime-dependentselectivecompetitionisintroducedintheSelf-OrganizingMapswiththehelpofgatedneurons.Thegateinputoftheneurondecidesitsoutput.Theoutputisthedistancemeasuredwhenthegateinputisnon-zeroandtheoutputiszerootherwise.Agatedneuronclassifieris Illdesignedusingsuchgatedneurons.Theperformanceofthegatedneuronclassifierisexplainedwithrecognitionofhandwrittencharacters.The directionalfeaturesoftheinputpatternsareextractedusingKirschmasks.Thesefeaturesarefedtotheinputatdistincttimeintervals.Thenetworkrecordsthewinninginstanceofeachneuronofvarioustimeintervalsinaregister.Eachoftheseinstancesisconnectedtotheoutputclassnodes.The strengthoftheseconnectionsarebasedonthestrengthofbelief/disbeliefoftheindividualnetwork.Thelabeloftheclassnodewithmaximumoutput denotestheclassofthetestsample.Thepatternismisclassified/rejectedwhenmorethanoneoutputneuronproducesthesamemaximumvalueorwhentheoutput ofallneuronsiszero.AFuzzyMultilayerNeuralNetworkisproposedalongwithitssupervisedlearningalgorithm.ThenetworkisdevisedwithfourlayersofFuzzyNeurons.Theinputsarefirstfuzzified.Thenetworkremembersthe'minimumandmaximumvaluesinalldimensionsofaclassofinputsascomersofann-dimensionalhyperbox.Whenadditionalinputsofsameclassesarepresentedthecorrespondinghyperboxissuitablyexpandedtoincludethenewpatternsalso.Thisprocessmaycauseahyperboxofoneclass tooverlapwiththatofotherclasses.Inthatcasetheyarecontractedtominimizetheoverlap.Duringtesting,thedegreetowhichthepatterniscontainedinsidetheboxindicatesthemembershipvalue.Theboxwith maximummembershipvaluelabelsthepattern. IVFinally,aGeneticAlgorithmbasedFeatureextractionschemeis.proposedtoextractuniquefeaturesoftheclassesandinitialpopulationissetwithrandomfeatures.Afeasibilitycheckisperformedtofindwhetheralltheserandomfeaturesarecontainedwithinthepattern.Thequalityofthe featuresisevaluatedbyitsinformationcontent.Theprobabilityofapatternbelongingtoaclassandnotbelongingtootherclassesisusedtomeasuretheinformationcontentofthefeature.This usestheapproximation ofBaye’srule.Thepixelsumimagesofasubsetofinputpatternsareobtainedforevery class.Afeaturedetectorislocatedinthepatternandcomparedwiththedigit image.Thegeneticoperatorsareappliedtotheindividualstogetbetterqualityfeaturesinsuccessivegenerations.Thismethodoffeatureextractionprovidestheadvantagesofdimensionalityreduction.Anapplicationpackageforrecognitionofhandwrittenandmachine-printedsamplesusingtheproposednetworkmodelsisdevelopedandnamed as“Rekogonizee”.ThisisdevelopedusingVisualC++withauser-friendly•menubasedinterface.Ithasoptionsforscanningtheinputpatterns,testing andtrainingthenetwork.Ithasgotfeaturestoextractthedirectionalfeatures fromtheinputandtrainthenetwork.HelpforvariousoptionsareprovidedthroughHTMLfiles.Thispackagecanbeusedeffectivelyforrecognitionofhandwrittenandmachineprintednumeralsandalphabets.