Das Video kommt von YouTube: erst beim Abspielen verbindet sich die Seite mit YouTube (Google).
Forward Propagation of Neural Networks | El Mahdi El Mhamdi
Das Wichtigste aus dem Video
Tipp auf eine Zeit – das Video springt genau dorthin.
Transkriptautomatisch erstellt · 92 Zeilen
- in this lesson we will see how neural networks actually compute so in technical terms we will describe the forward pass in a neural network let's
- start by reminding the basic structure of a neural network actually there is a video on that but we can start start by a short reminder so let's start with a
- very simplified view you have an image where there is a man and a bike from the neural network point of view this image is simply a grid of course this is not
- the the real view the grid will be uh much more granular uh and you would have as much squares as much um points here as there are pixels in your
- camera One megapixels it means that you would have one million points in your grid and so in this grid each point has a value which is the color of this pixel
- so so a pixel will look like this so this will be a blue pixel another one would look like this which would be a green a red pixel a pixel can be green
- and of course a pixel can be any other color which can be a combination of the three previous colors but to simplify and go in a higher level of obstruction
- we can just say that a pixel has a value a number can be in like for example a number in the gray scale or the coordinates of the pixel in
- red blue and green uh uh color systems but you let's say that a pixel is each square of this uh each square of this grid is is a value is a number once we
- understand this abstraction we can simplify our scheme and say that this image actually a list of values this would be our input we can say that this
- is the first value second third Etc and then you start over here so you are listing all the grids in this fashion what we want in the output is a set of
- answers which could also be a list of values say that this one tells you how likely a bike is present in the in the image so this this would tell you so for
- example if this if this output is 0.9 it means that it's very likely it's 90 90% chances that there is a bike in the in the picture and then so this is of
- course this is oversimplified so you can have for example a bike uh women men Etc and if you go back to the example of Facebook tagging system this
- can be for example the lists of Facebook users so for example the first node would tell you Morgan Freeman the second node you tell you uh I don't know Arnold
- schwarzeneger and and then You' have 2.2 billion possible answers and we want to map those inputs to some useful outputs telling us what is there in an image and
- we we'll do it in a hierarchical form so we'll have a set of nodes called neurons organized in layers to which those inputs will be broadcast those neurons
- are or organized in layers that can be as what we call Feit forward in in the sense that the signal only goes this way there's no
- Loop between a neuron on the output side and the neuron in the input side but of course we can have other topology so here we'll focus on the Feit forward
- structure so neurons would process the values they receive and then broadcast a value to the neurons on their outut side of course this is a very simplified view
- of the of the way the the optical Cor works because in this case neurons don't have values they don't send values they either Spike or not and the useful from
- the information theoretic point of view the useful value of a neuron is the rate with which it is spiking so let's let's stick to the artificial neurons point of
- view which is that neurons have values so they they receive something they produce a value after after that they have processed what they received and
- then they broadcast this value to to the next layer so so we have a a series of broadcasts until we reach the output where uh final layer will say oh here I
- see 90% chance that the photo contains Bruce and then this neuron says that I see 5% chance that this photo
- contains Alice Etc so this is the schematic view now the exact computation is as follows to just to provide simple formulas to
- start with I will start with so I will describe a very simple neural network where you have a set of inputs X1 X2 X3 then you would have a hidden layer
- containing five neurons y1 Y2 Y3 until y5 and at the end we have two output neurons o1 and O2 so each value sent by an input
- so for example X1 will send its value so the the value of the pixel corresponding to X1 and then y1 the neuron one will give a weight weight one one means this
- is how much of importance this neuron gives to this input and then this would be the importance neuron 2 gives to pixel one the importance neuron Tre
- gives to pixel one Etc and then w 5 1 the same would happen to pixel 2 and to pixel 3 and then the output noes would also weight those neurons so in a very
- simplified view so this would be the weight from neuron to to Output one and just to avoid confusion we would put an index one here say that this is the ways
- go into the first layer and then an index two here go saying that we are talking about the second layer so W12
- w11 superscript 2 Etc W13 superscript 2 so in a very simplified view let's say that W2 is the skirt and that output one tells us whether the photo is a photo of
- Arnold schwarzeneger while neuron two while output two gives us the likelihood that the photo is a photo of Maru so this means that W12 superscript 2 is
- very likely way lower than w22 because the maruy neuron would likely say that this photo is of maruy because if there is a skirt because a
- skirt means a lady and maruy is a lady arer is a guy it's very it's very unlikely that Arnold fer wears the skirt so this was about waiting so the
- importance of waiting is to say that from a point of view of a neuron not all the neurons on my left have equal in importance some neurons would be
- important in my decision to say that this is a man or a woman some neurons I'll just ignore and I will give them a very low weight now what the what do
- neurons do with with with the with the values they receives they receive and the weights they they put on those values is that so each neuron w i among
- those five here is actually applying an activation function to the weighted sum it is receiving from the inputs so if this is Neuron I all the synapses soal
- synapses go into neuron I we start with I so this convention in this convention I means the post synaps neuron the neuron that comes after the
- synapse so this is the post synaptic neuron while J is the pre synaptic the neuron and so the pre synaptic neuron is sending this value XJ
- the post synaptic neuron is waiting it with W J puts it inside a weighted Sun running through all the inputs so from one to three here and then apply in an
- activation function what is the activation function so remember this simplified view that neuron Y2 is the skirt neuron let's say
- that Y2 when being excited by the presynaptic neuron so when receiving values from its left side is not reacting until it reaches a threshold
- after which you say oh there is a skirt and here there is no skirt this is a simple yet very useful abstraction and we can do it either in a discontinuous
- way like this but would prefer and we see this later differentiable modeling so for example we can say that neuron Y2 has a has an S curve that's modeled this
- yes and no behaviors below this value it's very likely that the neuron will say no skirt and above it it starts to reach high probability so it's it starts
- to to say that there is actually a skirts in what I receive there are V there are there are lots of functions that model this Behavior one of them is
- the sigmoid function which is five of xal 1/ 1 + exponential minus X you can have also hyperbolic tangent that goes from minus1 to 1 and here you would have
- so hyperbolic tangent of X and of course you can come up with other activation functions a popular choice is the so-called Ru function which is a
- rectified linear function below below threshold you say zero and above you just say Alpha of X so Alpha can be for example four a linear function of X you
- have also so activation function like this where you have a first slope enter your threshold and then you increase the slope or you decrease the
- slope um you can you can also use quadratic or exponential whatever so you can have also yeah the rectified exponential where you have you say zero
- until until a value where you say exponential this might be important in practice but to understand the concept you can just say that the neuron has an
- activation function that models some threshold Behavior so with this in mind what actually our what what what our simple and small neural
- network does is actually producing two outputs 01 and O2 yeah the output neurons can apply an activation function or cannot so let's say that the act the
- output neurons are linear they don't they don't do this sigmoid or this nonlinear activation function so they just re reive a weighted sum so they
- receive weights so one would receiv weight would receive weights with superscript two because it's the second layer for the synapsis so it's just
- receiving a weighted sum of values produced by neurons J's J J going from one to five O2 will receive a weighted sum of the same kind and then each YJ is
- itself the activation function applied to the weighted sum received by the neuron from all the inputs so if we want to detail o1 o1 would be a weighted sum
- weighted by the synapsis to the output layer of activation functions applied to weighted sums coming from the coming
- from the input and this abstraction can be recursively called to model much deeper networks so for example if you have n
- neurons so in the general case if you have n neurons in the input then so sub superscript one means first layer you would have y on
- y1 until y N1 N1 means the number of neurons of layer one so this would be the width of layer one then you would have neurons of Layer Two where the
- width it goes until neuron N2 and then neurons of layer small L with wids n l those would be called in Hidden layers so those are actually the
- layers doing the work doing the job of the neural network right before the output layer so at the end You' have superscript big L which
- means the Big L is the total number of this is the depths of the neural network we have W1 superscript Big L capital L until
- w n capital L subscripts capital L and at the end you would have the output which of course can be a set of values 01 until o something so if you
- have a neuron so neuron J and neuron I here and maybe a pixel x k here so input XK is linked to neuron J with weight J K
- superscript one neuron I is linked to neuron J with weight wi J superscript to something going to layer l would be superscript small
- L and then at the end superscripts Capital l+ One go into the output so for example here if it's
- linking the neuron number one and the output number two it would be 2 one so yeah at the end the output would be a weighted sum superscripts capital L +
- one so if this is output let's say I yeah if the output if the output is itself a set of values so 01 O2 Etc and so if if we are talking about
- the output index I so this output for example it will be receiving a weighted sum superscripted by the depth plus one indexed by the index of this output and
- the index of the incoming neurons so those will be the neurons indexed by J from the last layer capital L which themselves is the activation function
- applied to a weighted sum so those weighted sums are going to the neuron index J it would be run through indexes K and K would be the neuron of layer l
- minus one which in turn is the activation function of some K Prime w k k Prime neurons K Prime of layer l minus 2 Etc recursively
- until we reach the weighted sum on the input which looks like the simple single layer single handle layer Network we we presented before and of course the
- output node can be just a linear weighted sum it can be also a weighted s to which we can apply an activation function so this is this is the way we
- compute the forward pass and of course the interesting the interesting the interesting thing in the neural network so what what makes it
- work is the good set of values wi so so a good set of synoptic value so learning is actually about setting the weights or trying to find
- the best set of weights and since machine learning is about avoiding to program so a avoiding as much as possible manual programming will not
- have a set of experts in image recognition telling us yeah that yeah the neuron that recognizes a skirt should be weighted with point5 or 3.2 to
- correspond to an image of that and then the neuron of the Hat should be given a weight of 5.7 to model whatever personality that wears hats we can't we
- can't do that manually so learning is about finding an automated way to set those synaptic weights so to set those coefficients that neurons give to each
- other so each neuron gives an importance to another another another neuron and we want to do this in a in an automated fashion given
- examples so what we will see in another video is how so this is so this is the forward pass so this is how a neuron Network processes the input through a
- series of weighted sums and activation functions applied to those weighted fun sums until reaching the output what we would see in another video is how to
- learn that how to set an algorithm that will automatically adjust the values of the weights to find the best prediction we want at the output
- level