Dissertation/Thesis Abstract

Ranked Similarity Search of Scientific Datasets: An Information Retrieval Approach
by Megler, Veronika Margaret, Ph.D., Portland State University, 2014, 305; 3629331
Abstract (Summary)

In the past decade, the amount of scientific data collected and generated by scientists has grown dramatically. This growth has intensified an existing problem: in large archives consisting of datasets stored in many files, formats and locations, how can scientists find data relevant to their research interests? We approach this problem in a new way: by adapting Information Retrieval techniques, developed for searching text documents, into the world of (primarily numeric) scientific data. We propose an approach that uses a blend of automated and curated methods to extract metadata from large repositories of scientific data. We then perform searches over this metadata, returning results ranked by similarity to the search criteria. We present a model of this approach, and describe a specific implementation thereof performed at an ocean-observatory data archive and now running in production. Our prototype implements scanners that extract metadata from datasets that contain different kinds of environmental observations, and a search engine with a candidate similarity measure for comparing a set of search terms to the extracted metadata. We evaluate the utility of the prototype by performing two user studies; these studies show that the approach resonates with users, and that our proposed similarity measure performs well when analyzed using standard Information Retrieval evaluation methods. We performed performance tests to explore how continued archive growth will affect our goal of interactive response, developed and applied techniques that mitigate the effects of that growth, and show that the techniques are effective. Lastly, we describe some of the research needed to extend this initial work into a true “Google for data”.

Indexing (document details)
Advisor: Maier, David
Commitee: Delcambre, Lois M.L., Duh, Jiunn-Der, Halevy, Alon Y.
School: Portland State University
Department: Computer Science
School Location: United States -- Oregon
Source: DAI-B 75/11(E), Dissertation Abstracts International
Source Type: DISSERTATION
Subjects: Information Technology
Keywords: Big data, Information retrieval, Ranked relevance, Scientific databases, Similarity search
Publication Number: 3629331
ISBN: 9781321061307
Copyright © 2018 ProQuest LLC. All rights reserved. Terms and Conditions Privacy Policy Cookie Policy
ProQuest