Deep Learning for AI

 Todays presentation is a lecture by Turing award winner Yoshua Bengio presented at the Heielberg Laureate Forum in 2019.  It provides a higher level overview of the current state of the art and future directions than the kind of info you get in something like yesterdays nuts and bolts full stack lecture.

This lecture will look back at some of the principles behind the recent successes of deep learning as well as acknowledge current limitations, and finally propose research directions to build on top of this progress and towards human-level AI. 

Notions of distributed representations, the curse of dimensionality, and compositionality with neural networks will be discussed, along with the fairly recent advances changing neural networks from pattern recognition devices to systems that can process any data structure thanks to attention mechanisms, and that can imagine novel but plausible configurations of random variables through deep generative networks. 
At the same time, analyzing the mistakes made by these systems suggests that the dream of learning a hierarchy of representations which disentangle the underlying high-level concepts (of the kind we communicate with language) is far from achieved. 
This suggests new research directions for deep learning, in particular from the agent perspective, with grounded language learning, discovering causal variables and causal structure, and the ability to explore in an unsupervised way to understand the world and quickly adapt to changes in it.


1:  The whole discussion on misconceptions associated with local minima in deep learning systems is an interesting one.  

It provides some intuition for the very non-intuitive strategy seen in some architectures to actually increase the dimensionality of the system to improve the ability to get to a solution.


Popular posts from this blog

Simulating the Universe with Machine Learning

CycleGAN: a GAN architecture for learning unpaired image to image transformations

Pix2Pix: a GAN architecture for image to image transformation