Dissertation/Thesis Abstract

Integrating Multiple Modalities into Deep Learning Networks
by McNeil, Patrick N., Ph.D., Nova Southeastern University, 2017, 296; 10283417
Abstract (Summary)

Deep learning networks in the literature traditionally only used a single input modality (or data stream). Integrating multiple modalities into deep learning networks with the goal of correlating extracted features was a major issue. Traditional methods involved treating each modality separately and then writing custom code to combine the extracted features.

Current solutions for small numbers of modalities (three or less) showed there are multiple architectures for modality integration. With an increase in the number of modalities, the “curse of dimensionality” affects the performance of the system. The research showed current methods for larger scale integrations required separate, custom created modules with another integration layer outside the deep learning network. These current solutions do not scale well nor provide good generalized performance. This research report studied architectures using multiple modalities and the creation of a scalable and efficient architecture.

Indexing (document details)
Advisor: Cannady, James D.
Commitee: Cerkez, Paul, Mukherjee, Sumitra
School: Nova Southeastern University
Department: Computer Science
School Location: United States -- Florida
Source: DAI-B 78/10(E), Dissertation Abstracts International
Subjects: Artificial intelligence
Keywords: Deep learning, Multiple modality
Publication Number: 10283417
ISBN: 9781369817225
Copyright © 2019 ProQuest LLC. All rights reserved. Terms and Conditions Privacy Policy Cookie Policy