COMING SOON! PQDT Open is getting a new home!

ProQuest Open Access Dissertations & Theses will remain freely available as part of a new and enhanced search experience at

Questions? Please refer to this FAQ.

Dissertation/Thesis Abstract

Learning Representations of Text through Language and Discourse Modeling: From Characters to Sentences
by Jernite, Yacine, Ph.D., New York University, 2018, 213; 10680744
Abstract (Summary)

In this thesis, we consider the problem of obtaining a representation of the meaning expressed in a text. How to do so correctly remains a largely open problem, combining a number of inter-related questions (e.g. what is the role of context in interpreting text? how should language understanding models handle compositionality? etc...) In this work, after reflecting on the notion of meaning and describing the most common sequence modeling paradigms in use in recent work, we focus on two of these questions: what level of granularity text should be read at, and what training objectives can lead models to learn useful representations of a text's meaning.

In a first part, we argue for the use of sub-word information for that purpose, and present new neural network architectures which can either process words in a way that takes advantage of morphological information, or do away with word separations altogether while still being able to identify relevant units of meaning.

The second part starts by arguing for the use of language modeling as a learning objective, and provides algorithms which can help with its scalability issues and propose a globally rather than locally normalized probability distribution. It then explores the question of what makes a good language learning objective, and introduces discriminative objectives inspired by the notion of discourse coherence which help learn a representation of the meaning of sentences.

Indexing (document details)
Advisor: Sontag, David
Commitee: Bowman, Samuel R., Cho, Kyunghyun, Petrov, Slav, Rush, Alexander M.
School: New York University
Department: Computer Science
School Location: United States -- New York
Source: DAI-B 79/08(E), Dissertation Abstracts International
Subjects: Language, Artificial intelligence, Computer science
Keywords: Deep learning, Language understanding, Machine learning, Natural language processing
Publication Number: 10680744
ISBN: 978-0-355-77356-9
Copyright © 2021 ProQuest LLC. All rights reserved. Terms and Conditions Privacy Policy Cookie Policy