Lesson 2: Introduction to Neural Networks_I

Udacity: Intro to Deep Learning with PyTorch:

Lesson 2: Introduction to Neural Networks

– Why “Neural Networks”?

Screen Shot 2019-07-20 at 10.51.08 PM

– Perceptron Algorithm

  Screen Shot 2019-07-20 at 10.53.36 PMScreen Shot 2019-07-20 at 10.53.51 PM

   — Coding the Perceptron Algorithm

Recall that the perceptron step works as follows. For a point with coordinates , label , and prediction given by the equation :
  • If the point is correctly classified, do nothing.
  • If the point is classified positive, but it has a negative label, subtract and  from  and b respectively.
  • If the point is classified negative, but it has a positive label, add  and  to and  respectively.


– Error Function

An error function is simply something that tells us how far we are from the solution


– Log-loss Error Function

Screen Shot 2019-07-20 at 11.11.44 PM


– Discrete vs Continuous PredictionsScreen Shot 2019-07-21 at 11.24.42 PM

Screen Shot 2019-07-21 at 11.22.39 PM
Screen Shot 2019-07-21 at 11.24.56 PM



Day 22 of #60daysofudacity: 

– The Softmax Function

softmax function, which is the equivalent of the sigmoid activation function, but when the problem has 3 or more classes.

Screen Shot 2019-07-21 at 11.28.57 PMScreen Shot 2019-07-21 at 11.29.51 PMScreen Shot 2019-07-21 at 11.32.42 PM


– One-Hot Encoding

Screen Shot 2019-07-23 at 10.12.29 PM


Day 23 of #60daysofudacity: 

– Maximum Likelihood



There’s definitely a connection between probabilities and error functions, and it’s called Cross-Entropy. This concept is tremendously popular in many fields, including Machine Learning. Screen Shot 2019-07-23 at 11.01.59 PMScreen Shot 2019-07-23 at 11.05.47 PMScreen Shot 2019-07-23 at 11.08.50 PMScreen Shot 2019-07-23 at 11.08.56 PM

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s