HTC Education Series: Getting Started with Deep Learning - Lesson 6
This weeks lesson dives back into the nitty gritty details of working with deep learning systems using the fastai api. We finish up Chapter 5 of the course book, look more deeply at Softmax, the specifics of how transfer learning works, and fun things to do with adaptive learning rates to improve system performance.
Then we look at multi-label classification problems (in contrast to the binary classification problems we have examined so far. We show how to work with fastai's DataLoader and DataSet classes in the datablock api to use the datablock api with multi-label classification data. We talk about modifying loss functions to work with multi-label data.
We then move on to take a look at deep learning based collaborative filtering applications (like how one might implement a Netflix recommendation system). Where the deep learning system learns latent factors in the data. The specific example discussed is recommendation systems, but the underlying principals are much more general, and can be applied to many different potential application scenarios.
You can also watch this lecture at the fastai course site here. The advantage of this is that you can look at course notes, a Questionnaire, a transcript of the lecture is available to train your next generative RNN system, etc.
What is covered in this lecture
Choosing a correct learning rate is important (want to train as fast as possible without introducing error by overtraining)
Half Precision floating point calculations
Multi-label Classification - image can have more than one label associated with it
Pandas (Python library to deal with standard data formats)
Sigmoid - nonlinear function to map a number to be between -1 and 1
Additional HTC Course Material
1: Last week Xander got us pumped up about an exciting new neural net architecture called a GAN (Generative Adversarial Network). Specifically he detailed how the StyleGAN works. GANs are a hot research topic and 3 new ones have probably been invented in the time it took me to type this sentence.
1: Classification vs Regression labeling. The 'Regression' terminology i think is confusing for beginners.
Don't forget to read the course book
Finished chapter 5.
Need to review something from the previous lessons in the course.
You can access Lesson 1 here.
You can access Lesson 2 here.
You can access Lesson 3 here.
You can access Lesson 4 here.
You can access Lesson 5 here.
You can move on to the next Lesson 7 in the course (when it posts on 11/09/20).