Lachlan MacDonald, “Towards a formal theory of deep optimisation”

/ November 21, 2022/

December 13, 2022 @ 12:00 pm – 1:15 pm

“Towards a formal theory of deep optimisation”

Lachlan MacDonald, PhD
Postdoctoral Fellow
Australian Institute for Machine Learning

Abstract: Precise understanding of the training of deep neural networks is largely restricted to architectures such as MLPs and cost functions such as the square cost, which is insufficient to cover many practical settings. In this talk, I will argue for the necessity of a formal theory of deep optimisation. I will describe such a formal framework, introduced recently by myself and collaborators, which elucidates the roles played by skip connections and normalisation layers in global optimisation and facilitates the first proof that a class of deep nets can be trained to global optima with the cross-entropy cost. I will outline how the theory can be applied even to practical architectures such as ResNet in predicting architectural interventions that accelerate training, on practical datasets such as ImageNet. I will conclude with a discussion of intriguing directions for future research stemming from our work.


Biography: Lachlan was awarded his PhD in pure mathematics in December 2019. After 18 months as a postdoc in pure mathematics at the Australian National University and the University of Adelaide, he joined the Australian Institute for Machine Learning in 2021, where he has been researching equivariance, optimisation and generalisation.




Tuesdays, 12pm-1:15pm

Held virtually in person at Clark 110 & over Zoom

Check for event details:



Join Zoom Meeting


Meeting ID: 986 2441 3365

One tap mobile

+13017158592,,98624413365# US (Washington DC)

+16469313860,,98624413365# US


Dial by your location

+1 301 715 8592 US (Washington DC)

+1 646 931 3860 US

+1 309 205 3325 US

+1 312 626 6799 US (Chicago)

+1 646 558 8656 US (New York)

+1 669 900 6833 US (San Jose)

+1 719 359 4580 US

+1 253 215 8782 US (Tacoma)

+1 346 248 7799 US (Houston)

+1 386 347 5053 US

+1 564 217 2000 US

+1 669 444 9171 US

Meeting ID: 986 2441 3365

Find your local number:


Join by SIP


Join by H.323 (US West) (US East) (India Mumbai) (India Hyderabad) (Amsterdam Netherlands) (Germany) (Australia Sydney) (Australia Melbourne) (Singapore) (Brazil) (Mexico) (Canada Toronto) (Canada Vancouver) (Japan Tokyo) (Japan Osaka)

Meeting ID: 986 2441 3365



Share this Post