Roy Schwartz - CS-Lecture
Thursday, 20.12.2018, 10:30
Despite their superb empirical performance, deep learning models for
natural language processing (NLP) are often considered black boxes, as
relatively little is known as to what accounts for their success. This
lack of understanding turns model development into a slow and expensive
trial-and-error process, which limits many researchers from developing
state-of-the-art models. Customers of deep learning also suffer from
this lack of understanding, because they are using tools that they
cannot interpret. In this talk I will show that many deep learning
models are much more understandable than originally thought.
I will present links between several deep learning models and classical
NLP models: weighted finite-state automata. As the theory behind the
latter is well studied, these findings allow for the development of more
interpretable and better-performing NLP models. As a case study, I will
focus on convolutional neural networks (ConvNets), one of the most
widely used deep models in NLP. I will show that ConvNets are
mathematically equivalent to a simple, linear chain weighted
finite-state automaton. By uncovering this link, I will present an
extension of ConvNets that is both more robust and more interpretable
than the original model. I will then present similar observations
regarding six recently introduced recurrent neural network (RNN) models,
demonstrating the empirical benefits of these findings to the
performance of NLP systems.
This is joint work with Hao Peng, Sam Thomson and Noah A. Smith
Roy Schwartz is a postdoctoral researcher at the University of
Washington and the Allen institute for AI. Roy's research focuses on
improving deep learning models for natural language processing by
gaining mathematical and linguistic understanding of these models.
He received his Ph.D. and M.Sc. in Computer Science and his B.Sc. in
Computer Science and Cognitive Science from the Hebrew University.
Roy has won a best paper award at RepL4NLP 2018, as well as a Hoffman
leadership and responsibility fellowship.