Dissertation/Thesis Abstract

Learning Representations of Text through Language and Discourse Modeling: From Characters to Sentences
by Jernite, Yacine, Ph.D., New York University, 2018, 213; 10680744
Abstract (Summary)

In this thesis, we consider the problem of obtaining a representation of the meaning expressed in a text. How to do so correctly remains a largely open problem, combining a number of inter-related questions (e.g. what is the role of context in interpreting text? how should language understanding models handle compositionality? etc...) In this work, after reflecting on the notion of meaning and describing the most common sequence modeling paradigms in use in recent work, we focus on two of these questions: what level of granularity text should be read at, and what training objectives can lead models to learn useful representations of a text's meaning.

In a first part, we argue for the use of sub-word information for that purpose, and present new neural network architectures which can either process words in a way that takes advantage of morphological information, or do away with word separations altogether while still being able to identify relevant units of meaning.

The second part starts by arguing for the use of language modeling as a learning objective, and provides algorithms which can help with its scalability issues and propose a globally rather than locally normalized probability distribution. It then explores the question of what makes a good language learning objective, and introduces discriminative objectives inspired by the notion of discourse coherence which help learn a representation of the meaning of sentences.

Indexing (document details)
Advisor: Sontag, David
Commitee: Bowman, Samuel R., Cho, Kyunghyun, Petrov, Slav, Rush, Alexander M.
School: New York University
Department: Computer Science
School Location: United States -- New York
Source: DAI-B 79/08(E), Dissertation Abstracts International
Source Type: DISSERTATION
Subjects: Language, Artificial intelligence, Computer science
Keywords: Deep learning, Language understanding, Machine learning, Natural language processing
Publication Number: 10680744
ISBN: 9780355773569
Copyright © 2019 ProQuest LLC. All rights reserved. Terms and Conditions Privacy Policy Cookie Policy
ProQuest