HTC Seminar Series #11: Introduction to Deep Reinforcement Learning

Let's take a dive into understanding deep reinforcement learning.  This week's HTC seminar is the first lecture in the 2019 MIT course on deep reinforcement learning course, presented by Lex Fridman. Lex Fridman is an AI researcher at MIT whose research interests include autonomous vehicles, human-robot interaction, and machine learning.

This lecture is a good introduction to what deep reinforcement learning is, and why you might want to try and understand how it works.  You can think of deep reinforcement learning as the shotgun marriage of deep learning and reinforcement learning.

Reinforcement learning has been around in some form since the very beginning of machine learning and artificial intelligence research.  It also has a long history of being used in the video game industry.  Since it's a war horse of old AI approaches that didn't really pan out in the end, why do we care about it in 2020?

Deep learning is the secret sauce that allows reinforcement learning algorithms to jump into the modern AI playpen and duke it out with all of the other algorithms competing in there for the title of 'smartest machine learner'.  Literally battling it out if you are talking about video gameplay algorithms.

Extending reinforcement learning by incorporating deep learning into it allows the algorithm to learn from model data.  So you can use reinforcement learning without requiring an explicitly designed state space.  By design we mean someone hand crafted it.  Getting rid of this 'hand design' part by incorporating deep learning into the algorithm allows reinforcement learning to solve hard problems it could not solve before (sound familiar).

DeepMind has used deep reinforcement learning to achieve human levels of performance in a wide variety of different video games.

If you are dying to learn more about deep reinforcement learning, and how to implement it in TensorFlow, check out this talk from Google I/O 2018.

Lex also has a really interesting series of artificial intelligence podcasts that we might dip into for some future HTC seminars.

Comments

Post a Comment

Popular posts from this blog

CycleGAN: a GAN architecture for learning unpaired image to image transformations

Image
 Like the Doublemint Twins touting the joys of Doublemint gum, 2 GANs are surely better than 1 GAN.  Especially if we package them together inside of one meta GAN module.  And this is exactly what the CycleGAN architecture does. Have you ever harbored dark secrets of turning a horse into a zebra?  The CycleGAN was developed to do just that. Learn how to turn a horse into a zebra.  And more. Now right away you can notice a difference between the image to image transformation GAN architectures we've been discussing over the last few posts.  Those last few posts described systems that learn from a database of matched input-output image pairs. And if your goal is to turn an edge representation into a nicely filled in continuous tone image, it's easy to build your database of matched input-output image pairs that your GAN system can then learn off of.  Take a continuous tone image (which will be the output of the database pair entry), then run it through an edge detector algorithm. 
Read more

Pix2Pix: a GAN architecture for image to image transformation

Image
 I thought following up yesterday's TraVelGAN post with a Pix2Pix GAN post would be useful to compare what is going on in the 2 architectures.  Two different approaches to the same problem. I stole this Pix2Pix Overview slide below from an excellent deeplearning.ai GAN course (note that they borrowed it from the original paper) because it gives you a good feel for what is going on inside of the Pix2Pix architecture.    Note how the Generator part is very much like an auto-encoder architecture, but rebuilt using the U-Net architecture features (based on skip-connections) that fastai has been discussing in their courses for several years before it became more widely known to the deep learning community at large  (and which originally came from an obscure medical image segmentation paper) . So the Generator in this Pix2Pix GAN is really pretty sophisticated, consisting of a whole image to image auto-encoder network with U-Net skip connections to generate better image quality at highe
Read more

Smart Fabrics

Image
"What makes smart fabrics revolutionary is that they have the ability to do many things that traditional fabrics cannot, including communicate, transform, conduct energy and even grow."  This is a quote from Rebeccah Pailes-Friedman, who is a professor at Pratt with a research focus on developing wearable technology and smart textiles. Enhancing textiles can be viewed through different lens.  Enhancements can be aesthetic, or performance boosting, or durability boosting. 
Read more