The present study shows that prestructuring based on domain knowledge leads to statistically significant generalization-performance improvement in artificial neural networks (NNs) of the multilayer perceptron (MLP) type, specifically in the case of a noisy real-world problem with numerous interacting variables.
The prestructuring of MLPs based on knowledge of the structure of a problem domain has previously been shown to improve generalization performance. However, the problem domains for those demonstrations suffered from significant shortcomings: 1) They were purely logical problems, and 2) they contained small numbers of variables in comparison to most data-mining applications today. Two implications of the former were a) the underlying structure of the problem was completely known to the network designer by virtue of having been conceived for the problem at hand, and b) noise was not a significant concern in contrast with real-world conditions. As for the size of the problem, neither computational resources nor mathematical modeling techniques were advanced enough to handle complex relationships among more than a few variables until recently, so such problems were left out of the mainstream of prestructuring investigations.
In the present work, domain knowledge is built into the solution through Reconstructability Analysis, a form of information-theoretic modeling, which is used to identify mathematical models that can be transformed into a graphic representation of the problem domain's underlying structure. Employing the latter as a pattern allows the researcher to prestructure the MLP, for instance, by disallowing certain connections in the network. Prestructuring reduces the set of all possible maps (SAPM) that are realizable by the NN. The reduced SAPM—according to the Lendaris–Stanley conjecture, conditional probability, and Occam's razor—enables better generalization performance than with a fully connected MLP that has learned the same I/O mapping to the same extent.
In addition to showing statistically significant improvement over the generalization performance of fully connected networks, the prestructured networks in the present study also compared favorably to both the performance of qualified human agents and the generalization rates in classification through Reconstructability Analysis alone, which serves as the alternative algorithm for comparison.
|Advisor:||Lendaris, George G.|
|Commitee:||Hall, Douglas V., Hammerstrom, Dan, Hansen, Brad, Perkowski, Marek|
|School:||Portland State University|
|Department:||Electrical and Computer Engineering|
|School Location:||United States -- Oregon|
|Source:||DAI-B 73/04, Dissertation Abstracts International|
|Subjects:||Music, Statistics, Computer Engineering, Artificial intelligence|
|Keywords:||Clave, Machine learning, Multilayer perceptrons, Neural networks, Prestructuring, Samba|
Copyright in each Dissertation and Thesis is retained by the author. All Rights Reserved
dissertation or thesis. The supplemental files are provided "AS IS" without warranty. ProQuest is not responsible for the
content, format or impact on the supplemental file(s) on our system. in some cases, the file type may be unknown or
may be a .exe file. We recommend caution as you open such files.
supplemental files is subject to the ProQuest Terms and Conditions of use.