In this week, you'll learn the practical aspects of how to make your neural network work well. Ranging from things like hyperparameter tuning to how to set up your data, to how to make sure your optimization algorithm runs quickly so that you get your learning algorithm to learn in a reasonable amount of time.
⇒ Making good choices in how you set up your training, development, and test sets can make a huge difference in helping you quickly find a good high-performance neural network.
⇒ When training a neural network, you have to make a lot of decisions, such as
how many layers will your neural network have?
how many hidden units do you want each layer to have?
what's the learning rate?
what are the activation functions you want to use for the different layers?
When you're starting on a new application, it's almost impossible to correctly guess the right values for all of these, and for other hyperparameter choices, on your first attempt.
So, in practice, applied machine learning is a highly iterative process, in which you often start with an idea, such as you want to build a neural network of a certain number of layers, a certain number of hidden units, maybe on certain data sets, and so on. And then you just have to code it up and try it, by running your code.
You run an experiment and you get back a result that tells you how well this particular network, or this particular configuration works. And based on the outcome, you might then refine your ideas and change your choices and maybe keep iterating, in order to try to find a better and a better, neural network.
so today, applied deep learning is a very iterative process where you just have to go around this cycle many times to hopefully find a good choice of network for your application. So one of the things that determine how quickly you can make progress is how efficiently you can go around this cycle.
<aside> 💡 maybe a researcher with a lot of experience in speech recognition might, you know, jump in and try to do something on advertising.
→ intuitions from one domain or from one application area often do not transfer to other application areas.
And the best choices may depend on the amount of data you have, the number of input features you have through your computer configuration and whether you're training on GPUs or CPUs.
</aside>