HTC Education Series: Getting Started with Deep Learning - Lesson 1

Part of the goal here at HTC is to put together educational learning resources for Maui residents (or anyone else anywhere on our small planet who wants to follow along on the HTC blog). 

To help people get up to speed on deep learning based AI systems, we want to put together educational resources for anyone to learn about Deep Learning Neural Networks.  The goal is to do this so that anyone can dive in, follow the course, and come out of it with enough knowledge to actually do something useful with deep learning.  

We want to make the course accessible to everyone, so don't worry if you don't have a college degree,  or some kind of intensive math or engineering or computer science background, because you really don't need any of that to take this course.  

Note that this is very different from the conventional university model, where you would be required to take years of advanced math in a highly technical curriculum before they got around to actually teaching you about these topics, and even then would bury you in advanced math symbology when they did.

If you have been reading the HTC blog, you know that we cover all kinds of different aspects of deep learning, and how it fits into the overall revolution going on right now with Artificial Intelligence and deep learning.  Some of the topics we cover on the blog are pretty advanced, others are very much in the getting started category.

We cover advanced topics because we want people to get excited about the possibilities of working with this stuff.  And to educate them about what is possible, or what soon will be possible.

We're also very focused on how artists can take advantage of and utilize advanced technologies like deep learning.  We hope to have courses specifically focused on artists and their needs in the future.

We're going to be presenting a video lesson here once a week along with some additional supplemental information as we work through the course material.  The actual course itself was developed by fast.ai.

Why did HTC choose fastai for this course? Because we feel by far that it is the best one currently available for people getting started.  That includes total beginners, as well as more seasoned professionals. 

Fast.ai is a non profit organization in San Francisco devoted to both educating people about AI, developing tools to make it easy to do this stuff, and to help people understand ethical issues associated with it. So the course talks about the ethics of AI systems as well as how to build them

I have a graduate level engineering background, did neural net research in the early 90s, including apparently publishing the first paper on using convolutional neural networks for image style transfer, over 40 years of programming experience, and i am still finding the course incredibly useful and interesting.

This points out something very important, which is that life long learning is very important. The world of technology, and AI in particular, is advancing at an incredible rate, exploding really at this point, and you are always going to have to learn new things, or new ways of doing things.

You don't have to pay anything to watch the course videos. No one is going to send you endless spam ads asking you to buy more stuff if you take it.

I have personally been exposed to many different kinds of deep learning and AI courses, including ones taught in university settings, as well as multiple other 'pay for it' courses offered on the internet.  Fast.ai is the best by far.  For all of the different reasons we have been discussing.

Let's get started. Lesson 1 covers the course philosophy, early history of neural nets, explains some terminology, gives an intro to Jupyter Notebooks, and then shows how you can use the fastai library in a Jupyter Notebook to very easily generate a working deep learning neural net that can distinguish between images of cats and dogs.

Check out the fastai Lesson 1 video associated with this HTC course.

You can also watch the video on the fastai course site here.  The advantage of that is that you can access on that site a searchable transcript, interactive notebooks, setup guides, questionnaires, etc.

Each week, a new course lesson will be published here every monday, until we conclude the course.  We would encourage you to watch the video, read the appropriate chapters in the book associated with the video, and then spend the rest of your week until the next lesson playing around with what has been discussed.  All you will need to play around with the course material is a web browser.

You are of course free to access any of the video lessons in the course at any time here. So if your style of personal learning is a little bit different then what i just mention, by all means follow your heart.  I do think it's important to spend time learning how to work with the various code examples if you are going to get the most out of the course. But even if you just watch the video lessons, you are going to come away from this course with a much better understanding of what deep learning is all about, ethical considerations associated with it, and how it can be used.

HTC will be providing additional 'extra' lesson material within our Getting Started with Deep Learning track.  These will appear as separate posts within the specific HTC Education Series track for 'Getting Started with Deep Learning' here on the HTC site.  

So if you were wondering why we didn't just send you on to the fastai site, end of story, that's the reason.  One way to think of it is through the lens of object oriented inheritance and open source software.  Rather then doing something from scratch that already has an excellent implementation, we are building on top of that existing excellent implementation.

There is a book associated with the course that we would encourage you to purchase called 'Deep Learning for Coders with fastai and PyTorch: AI Applications Without a PhD'.  You can purchase the ebook on amazon here ($34.99 when this was written).

All of the course material in the book is also available as Jupyter Notebooks. So if you are too dirt poor to buy the book, you can still get access to all of the course material.  We covered Jupyter Notebooks in a previous post.  Fun fact, the actual book was auto-generated from Jupyter Notebooks.

We have been covering Colab notebooks a lot recently on the blog.  Google Colaboratory is great, but does not support the custom widgets used in some of the fastai Jupyter Notebooks. So we are recommending that you use Gradient instead to run the course Jupyter  notebooks on.

You can find instructions on how to get started with Gradient for the course here.

There is a whole 'getting started section' at the main fastai course site that explains this and more, so check it out.

You will be working with Python in the course.  We covered getting started with Python in a previous post.  If you have some kind of programming experience in another language, you should be able to get comfortable with Python pretty quickly. 

If you have never learned anything about computer programming before, you will have a little bit of a hump to get over. But by working with the code examples in the course, you can start to get up to speed.  You really don't need to have done any programming before to run the course example code in a Jupyter Notebook inside of a web browser to create working state of the art deep learning neural networks that can solve all kinds of different problems. And by looking at the code, you can start to understand how programming actually works.  Fun fact, most professional programmers actually learned how to code by looking at existing code and then copying what they saw there in other programs.  

Remember, the whole point of deep learning neural networks is to not manually program a coded solution to a particular problem you want to solve. You present data to a model that learns itself from the data presented to it. 

This is a very different paradigm then conventional programming, where a programmers codes everything to solve a particular problem.  There is some manual coding to setup the neural net you want to work with, but all of the really hard work is done by the neural net itself as it learns from the data you are presenting it.

The course is designed so that you can do everything inside of a browser. So you don't need a fancy computer with expensive gpu cards in it to do any of the work associated with the code.  You don't need complex and overwhelming IDE environments to work with the code.  If you can use a web browser,  and have an inquisitive mind, you are ready to get started.

Additional HTC Course Material

Remember, the HTC deep learning course is built on top of the fastai deep learning course like you might write a code library using object inheritance to build on top of an existing open source code library. So we're going to be presenting additional course material to you in our HTC course track.

Personally, i find being exposed to multiple slightly different presentations about a particular topic a great way to better understand that topic.  I think you will find that to be true as well.

1: With above in mind, we now point you at an alternative 'Intro to Deep Learning' lecture.  This particular lecture is the first introductory lecture in a week long intensive deep learning bootcamp that was taught at MIT in January 2020.  It's got a great 'wizzy' demo at the very beginning to get you super jazzed about what you could do with deep learning.  

I'm including the 2nd HTC lecture videos directly in the HTC lessons like above now rather then just pointing to a link like i originally did in the original text above the inserted video.  This change came after some feedback from some earlier beta students for the HTC course when it became apparent that most people were totally missing the fact that each HTC course lesson had multiple lectures in it.  

The fastai lecture is only one of the lectures in a HTC course lesson, which sits on top of fastai and has additional material in it we think is important for you to be exposed to.  We think fastai is great, which is why we're building on top of it.  But HTC has a somewhat different viewpoint of the deep learning material, so we're going to come at it from multiple directions.  By being exposed to all of these different directions, we feel you will be better educated (better then if you just took the fastai course by itself). 

The 'wizzy demo' in this second klecture is basically Barack Obama introducing the deep learning course, or does he? So, not only is the 'wizzy' demo kind of amazing if you think about it, it also brings up all kinds of challenging ethical considerations for you to think about that are associated with deep learning systems.

This MIT lecture covers some material that will be covered in later fastai lectures.  Like the whole section on how deep learning systems learn via back-propagation and stochastic gradient descent.  They may flash a few scary math equations on the screen from time to time.  

Don't worry if you don't get it all the first time through.  Think of this alternative lecture as a way to help solidify your thinking a little it about how deep learning systems work.  The material will be covered again in more detail from a different perspective in the fastai lectures. 

So we will encourage you to come back to it after later fastai lectures and take a look at it again.  I think you will be surprised by how much more of the material you grok as the course progresses.  By being exposed to it in different ways.

I view this kind of lecture as a way to help wrap my head around different important concepts i want to ultimately get a good intuitive feel for.  Jeremy will then break the topics apart in his later fastai lectures, and show you how they actually work in code.  And how the 'math' behind them is actually pretty simple to understand when you see how it's actually implemented in code.

Two different viewpoints of the world, both important to get the best understanding of the underlying material.

2:  The fastai course book always has a set of questions at the end of each chapter that you should try to answer.  Same for the lectures.  This is a great way to help see if you understood everything presented, or if you need to go back and reread or re-listen to some particular topic to really get it.

We would also encourage you to do something else kind of like this, but a little bit different.  After you watch a given lecture, take a few minutes and open up a text editor and jot down a few notes about the important topics you learned.  Can you succinctly describe what the lecture covered as a series of topic bullet points.  And can you write a simple short explanation of what each of these bullet point topics really means.

If you can do this, then great.  It will help organize your thinking, and solidify the memories associated with the material you were exposed to.  If you can't do it yet for a particular lecture or book chapter, then go back through the material again, with the goal of paying attention to it in a way that let's you build your bullet list of topics covered and short descriptions of what those topics mean.

Don't initially use the fastai question list to build your 'topics covered' notes off of.  Try to do it just from rethinking what was covered in the lecture or chapter first.  Then review the material to fill in gaps.  You can then look at the fastai summary questions, and try to answer them.

Fun Stuff related to the first lecture

If you want to move on to Lesson 2 in the Getting Started with Deep Learning course, head on over to the lesson 2 post.