Jia (Kevin) Liu: “Mitigating Data and System Heterogeneity and Taming Fat-Tailed Noise in Federated Learning”
“Mitigating Data and System Heterogeneity and Taming Fat-Tailed Noise in Federated Learning”
Jia (Kevin) Liu, PhD
Assistant Professor
Ohio State University
Abstract: In this talk, I will present our recent work on mitigating data and system heterogeneity and fat-tailed noise to achieve linear convergence speedup in federated learning. Federated learning (FL) is a distributed machine learning architecture that leverages a large number of workers to jointly learn a model with decentralized data. FL has received increasing attention in recent years thanks to its data privacy protection, communication efficiency and a linear speedup for convergence in training (i.e., convergence performance increases linearly with respect to the number of workers). However, existing studies on linear speedup for convergence are only limited to the assumptions of i.i.d. datasets across workers and/or full worker participation, both of which rarely hold in practice. In the first part of the talk, we first propose a new federated learning paradigm called “anarchic federated learning” (AFL), which features a loose coupling between the server and the workers to enable the workers to participate in the learning anytime in any way they want, thus addressing the system heterogeneity in federated learning while retaining the highly desirable linear convergence speedup. In the second part of this talk, I will introduce a clipping-based method to mitigate the impacts of fat-tailed noise in FL stochastic gradients, which can also be induced from data heterogeneity in FL.
Biography: Jia (Kevin) Liu is an Assistant Professor in the Dept. of Electrical and Computer Engineering at The Ohio State University and an Amazon Visiting Academics (AVA). He received his Ph.D. degree from the Dept. of Electrical and Computer Engineering at Virginia Tech in 2010. From Aug. 2017 to Aug. 2020, he was an Assistant Professor in the Dept. of Computer Science at Iowa State University. His research areas include theoretical machine learning, stochastic network optimization and control, and performance analysis for data analytics infrastructure and cyber-physical systems. Dr. Liu is a senior member of IEEE and a member of ACM. He has received numerous awards at top venues, including IEEE INFOCOM’19 Best Paper Award, IEEE INFOCOM’16 Best Paper Award, IEEE INFOCOM’13 Best Paper Runner-up Award, IEEE INFOCOM’11 Best Paper Runner-up Award, IEEE ICC’08 Best Paper Award, and honors of long/spotlight presentations at ICML, NeurIPS, and ICLR. He is an NSF CAREER Award recipient in 2020 and a winner of the Google Faculty Research Award in 2020. He received the LAS Award for Early Achievement in Research at Iowa State University in 2020, and the Bell Labs President Gold Award. His research is supported by NSF, AFOSR, AFRL, and ONR.
Tuesdays, 12pm-1:15pm
Held virtually in person at Clark 110 & over Zoom
Join Zoom Meeting
https://wse.zoom.us/j/98624413365
Meeting ID: 986 2441 3365
One tap mobile
+13017158592,,98624413365# US (Washington DC)
+16469313860,,98624413365# US
Dial by your location
+1 301 715 8592 US (Washington DC)
+1 646 931 3860 US
+1 309 205 3325 US
+1 312 626 6799 US (Chicago)
+1 646 558 8656 US (New York)
+1 669 900 6833 US (San Jose)
+1 719 359 4580 US
+1 253 215 8782 US (Tacoma)
+1 346 248 7799 US (Houston)
+1 386 347 5053 US
+1 564 217 2000 US
+1 669 444 9171 US
Meeting ID: 986 2441 3365
Find your local number: https://wse.zoom.us/u/asoOElnUp
Join by SIP
Join by H.323
162.255.37.11 (US West)
162.255.36.11 (US East)
115.114.131.7 (India Mumbai)
115.114.115.7 (India Hyderabad)
213.19.144.110 (Amsterdam Netherlands)
213.244.140.110 (Germany)
103.122.166.55 (Australia Sydney)
103.122.167.55 (Australia Melbourne)
149.137.40.110 (Singapore)
64.211.144.160 (Brazil)
149.137.68.253 (Mexico)
69.174.57.160 (Canada Toronto)
65.39.152.160 (Canada Vancouver)
207.226.132.110 (Japan Tokyo)
149.137.24.110 (Japan Osaka)
Meeting ID: 986 2441 3365