Dissertation/Thesis Abstract

A Coherent Classifier/Prediction/Diagnostic Problem Framework and Relevant Summary Statistics
by Eiland, E. Earl, Ph.D., New Mexico Institute of Mining and Technology, 2017, 173; 10617960
Abstract (Summary)

Classification is a ubiquitous decision activity. Regardless of whether it is predicting the future, e.g., a weather forecast, determining an existing state, e.g., a medical diagnosis, or some other activity, classifier outputs drive future actions. Because of their importance, classifier research and development is an active field.

Regardless of whether one is a classifier developer or an end user, evaluating and comparing classifier output quality is important. Intuitively, classifier evaluation may seem simple, however, it is not. There is a plethora of classifier summary statistics and new summary statistics seem to surface regularly. Summary statistic users appear not to be satisfied with the existing summary statistics. For end users, many existing summary statistics do not provide actionable information. This dissertation addresses the end user's quandary.

The work consists of four parts: 1. Considering eight summary statistics with regard to their purpose (what questions do they quantitatively answer) and efficacy (as defined by measurement theory). 2. Characterizing the classification problem from the end user's perspective and identifying four axioms for end user efficacious classifier evaluation summary statistics. 3. Applying the axia and measurement theory to evaluate eight summary statistics and create two compliant (end user efficacious) summary statistics. 4. Using the compliant summary statistics to show the actionable information they generate.

By applying the recommendations in this dissertation, both end users and researchers benefit. Researchers have summary statistic selection and classifier evaluation protocols that generate the most usable information. End users can also generate information that facilitates tool selection and optimal deployment, if classifier test reports provide the necessary information.

Indexing (document details)
Advisor: Liebrock, Lorie M.
Commitee: Evans, Scott, Mazumdar, Subhashish, Shin, Dongwan, Sung, Andrew
School: New Mexico Institute of Mining and Technology
Department: Computer Science and Engineering
School Location: United States -- New Mexico
Source: DAI-B 79/02(E), Dissertation Abstracts International
Source Type: DISSERTATION
Subjects: Applied Mathematics, Operations research, Computer science
Keywords: Classifier evaluation, Efficacious summary statistic axioms, End user efficacy, Summary statistic selection
Publication Number: 10617960
ISBN: 978-0-355-39919-6
Copyright © 2019 ProQuest LLC. All rights reserved. Terms and Conditions Privacy Policy Cookie Policy
ProQuest