דלג לתוכן (מקש קיצור 's')
Logo of Technion
Logo of CS Department
אירועים

אירועים

On the Generalization of Gaussian Dropout using PAC-Bayesian bounds and Log-Sobolev Inequalities
event speaker icon
יניב נמקובסקי, הרצאה סמינריונית למגיסטר
event date icon
יום שני, 29.7.2019, 10:00
event location icon
טאוב 601
event speaker icon
מנחה:  Dr. Tamir Hazan
The omnipresence of increasingly large deep networks derives primarily from their empirical successes. Indeed, empirical evidence places a strong emphasis on operating at scale, more parameters and layers, in order to help both optimization and generalization. This counter-intuitive trend has eluded theoretical analysis. In this work, we present a PAC-Bayesian generalization bound for the Gaussian dropout that only requires an on-average loss function bound and on-average norm-gradient bounds by relying on log-Sobolev inequalities for Gaussian measures. Our preliminary experimental evaluation shows that our bounds \emph{decrease} when adding more layers or more parameters to the network.
[בחזרה לאינדקס האירועים]