Deep Dive into Deep Learning

MIT's "Introduction" to Deep Learning course is a comprehensive and intensive program that offers an overview of Machine Learning. It includes a historical context, some common Machine Learning techniques, a deep dive into Neural Networks with hands-on Google Colab labs using TensorFlow, and an exploration of Generative AI, all delivered at a brisk pace! 😬

Once again MIT release excellent content into the public domain.

1. Introduction to Deep Learning

Starts off fairly "simple" with the mighty Perception, weights, activation functions, layers, loss functions and back propagation. The first video gives you a nice overview of Neural Networks straight off the bat.

2. Recurrent Neural Networks, Transformers, and Attention

Pay attention to this one! Attention is what has given LLMs their recent boost.

3. Convolutional Neural Networks

Convolutional Neural Networks (CNNs) are often used for visual classification. i.e. feed in photo of a dog and hopefully it'll label it as a dog.

4. Deep Generative Modeling

An intro in generative modeling which is all about creating new data (e.g. images). Among other things it looks at GANs and VAEs.

NOTE: Later on in the course there's a lecture on Diffusion Models which are used by todays popular generative systems, e.g. DALL-E. It's worth first trying to wrap your head around GANs (I need to go back and look at this more and then revisit Diffusion Models!).

5. Robust and Trustworthy Deep Learning

Looks at ways to detect and improve certainty.

6. Reinforcement Learning

This is how we can build models to beat world champion GO and Chess players. Also, how we teach cars not to drive off the road.

7. New Frontiers (inc Diffusion Models)

Focusing on limitations of AI but it also covers mind blowing Diffusion Models towards the end of the video. Basically starting from random noise and iteratively "removing" the noise until you get something that looks real. 😲

8. Text-to-Image (TTI) Generation

Showcases Google's highly efficient MUSE which is the next step on from Diffusion Models.

My plan is to rewatch the videos that caught my attention (GANs, Attention, Diffusion Models etc) and then explore these areas in more depth whilst playing around with TensorFlow. I've found ChatGPT to be useful helping to clear up any confusion as it's quite intimidating if you're not coming from a maths background.

Enjoy!