Lesson 2: Introduction to Neural Networks_I

Udacity: Intro to Deep Learning with PyTorch:

Lesson 2: Introduction to Neural Networks

– Why “Neural Networks”?

Screen Shot 2019-07-20 at 10.51.08 PM

– Perceptron Algorithm

  Screen Shot 2019-07-20 at 10.53.36 PMScreen Shot 2019-07-20 at 10.53.51 PM

   — Coding the Perceptron Algorithm

Recall that the perceptron step works as follows. For a point with coordinates , label , and prediction given by the equation :
  • If the point is correctly classified, do nothing.
  • If the point is classified positive, but it has a negative label, subtract and  from  and b respectively.
  • If the point is classified negative, but it has a positive label, add  and  to and  respectively.

 

– Error Function

An error function is simply something that tells us how far we are from the solution

 

– Log-loss Error Function

Screen Shot 2019-07-20 at 11.11.44 PM

 

– Discrete vs Continuous PredictionsScreen Shot 2019-07-21 at 11.24.42 PM

Screen Shot 2019-07-21 at 11.22.39 PM
Screen Shot 2019-07-21 at 11.24.56 PM

 

 

Day 22 of #60daysofudacity: 

– The Softmax Function

softmax function, which is the equivalent of the sigmoid activation function, but when the problem has 3 or more classes.

Screen Shot 2019-07-21 at 11.28.57 PMScreen Shot 2019-07-21 at 11.29.51 PMScreen Shot 2019-07-21 at 11.32.42 PM

 

– One-Hot Encoding

Screen Shot 2019-07-23 at 10.12.29 PM

 

Day 23 of #60daysofudacity: 

– Maximum Likelihood

 

Cross-Entropy

There’s definitely a connection between probabilities and error functions, and it’s called Cross-Entropy. This concept is tremendously popular in many fields, including Machine Learning. Screen Shot 2019-07-23 at 11.01.59 PMScreen Shot 2019-07-23 at 11.05.47 PMScreen Shot 2019-07-23 at 11.08.50 PMScreen Shot 2019-07-23 at 11.08.56 PM

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s