Praneeth Netrapalli, “Pitfalls of Deep Learning”
Praneeth Netrapalli
Research Scientist- Google Research India
Join Zoom Meeting
https://wse.zoom.us/j/99481316345?pwd=UHNWSldld1g1bTc2UnVIbWdGVW8vZz09
Meeting ID: 994 8131 6345
Passcode: Clark
One tap mobile
+13017158592,,99481316345# US (Washington DC) 16465588656,,99481316345#
+US (New York)
Title: Pitfalls of Deep Learning
Abstract: While deep neural networks have achieved large gains in performance on benchmark datasets, their performance often degrades drastically with changes in data distribution encountered during real-world deployment. In this work, through systematic experiments and theoretical analysis, we attempt to understand the key reasons behind such brittleness of neural networks in real-world settings and why fixing these issues is exciting but challenging.
We first hypothesize, and through empirical+theoretical studies demonstrate, that (i) neural network training exhibits “simplicity bias” (SB), where the models learn only the simplest discriminative features and (ii) SB is one of the key reasons behind non-robustness of neural networks. A natural way to fix SB in trained models is by identifying the discriminative features used by the model and learning new features “orthogonal” to the learned feature.
Post-hoc gradient-based attribution methods are regularly used to identify the key discriminative features for a model. But, due to lack of ground truth, a thorough evaluation of even the most basic input gradient attribution method is still missing in literature. Our second contribution is to overcome this challenge through experiments and theory on real and designed datasets. Our results demonstrate that (i) input gradient attribution does NOT highlight correct features on standard models (i.e., trained on original data) but surprisingly, it does highlight correct features on adversarially trained models (i.e., trained using adversarial training) and (ii) “feature leakage”, which refers to the phenomenon wherein, given an instance, its input gradients highlight the location of discriminative features in the given instance as well as in other instances that are present in the dataset, is the reason behind why input gradient attribution fails for standard models.
Our work raises more questions than it answers, so we will end with interesting directions for future work.
Bio: Praneeth Netrapalli is a research scientist at Google Research India, Bengaluru. He is also an adjunct professor at TIFR, Mumbai and a faculty associate of ICTS, Bengaluru. Prior to this, he was a researcher at Microsoft Research. He obtained MS and PhD in ECE from UT Austin, and B-Tech in EE from IIT Bombay. He is a co-recipient of IEEE Signal Processing Society Best Paper Award 2019 and is an associate of Indian Academy of Sciences (IASc). His research interests are broadly in stochastic and nonconvex optimization, minimax/game theoretic optimization and designing reliable and robust machine learning algorithms.