Joshua Vogelstein – “Lifelong Learning”
“Lifelong Learning”
Abstract: To achieve artificial general intelligence (AGI) will require algorithms that can efficiently use data for more than a single task. Specifically, AGI requires algorithms that leverage data to generalize, improving performance on current, previous and future tasks. Naive transfer learning algorithms can improve performance on future tasks, while causing catastrophically forgetting previous tasks. However, avoiding forgetting is not generalizing beyond a particular task. We desire algorithms that, given new data, improves performance on current, past, and future tasks. We propose such an algorithm, called Lifelong Forest (LF), which achieves this goal. Specifically, LF breaks learning into two phases: partitioning and voting. The partitions of feature space serve as a representation that are useful for many related tasks. We propose a formal definition of transfer efficiency, which factorizes into a forward part (characterizing typical transfer learning scenarios) and a reverse part. Using the Split-CIFAR-100 dataset (and several novel variants), we quantify forward and reverse transfer of many state-of-the-art lifelong learning algorithms. LF demonstrates forward transfer with a similar efficiency as existing deep learning approaches. However, only LF demonstrates “reverse transfer”, that is, improving performance on prior tasks given future task data. LF therefore represents a compelling step towards developing AGI systems.
Bio: I received a B.S degree from the Department of Biomedical Engineering (BME) at Washington University in St. Louis, MO in 2002, a M.S. degree from the Department of Applied Mathematics & Statistics (AMS) at Johns Hopkins University (JHU) in Baltimore, MD in 2009, and a Ph.D. degree from the Department of Neuroscience at JHU in 2009. I was a Postdoctoral Fellow in AMS@JHU from 2009 until 2011, at which time I was appointed an Assistant Research Scientist, and became a member of the Institute for Data Intensive Science and Engineering. I spent 2 years at Information Initiative at Duke University, before coming home to my current appointment as Assistant Professor in BME@JHU, and core faculty in both the Institute for Computational Medicine and the Center for Imaging Science, as well as a member of the Kavli Neuroscience Discovery Institute. I married my kindergarten sweetheart in the summer of 2014, and we had our first child in 2017, and a second in 2019.