HTC Education Series: Getting Started with Deep Learning - Lesson 8
This lesson starts off with a great lecture by Jeremy of fastai on Natural Language Processing (NLP). Using deep learning nets and the fastai api.
So we'll be covering tokenization of text data. We'll be doing a deep dive into how to code recurrent neural nets (RNN) using the fastai api. We'll cover methods like LSTM which were developed to prevent exploding gradients in RNNs.
You can also watch this first fastai video lecture on the fastai course site here. The advantage of that is that you can access on that site a searchable transcript, interactive notebooks, setup guides, questionnaires, etc.What is covered in this lecture?
Additional HTC Course Material
The fastai lecture above in HTC Lesson 8 is specifically focused on looking at Recurrent Neural Networks RNN architectures.
2: We do have some additional HTC posts tagged with recurrent neural net. including a lecture and some great blog posts by Andrej Karpathy.
We also have numerous GPT-2 and GPT-3 tagged articles. And need to get a good overview post up of the Transformer architecture used to such prominence in the GPT posts.
Note that fastai takes a very different viewpoint towards tokenization as compared to other systems like OpenAI's GPT Transformer based NLP architectures. What is the difference in how they approach it?
Don't forget to read the course book
Need to review something from the previous lessons in the course.
You can access Lesson 1 here.
You can access Lesson 2 here.
You can access Lesson 3 here.
You can access Lesson 4 here.
You can access Lesson 5 here.
You can access Lesson 6 here.
You can access Lesson 7 here.
You can move on to the next Lesson 9 in the course (when it posts on 11/23/20).