iam hopeing you can help igg inc aka texas holdem poker deluex is useing bots to cheat its players i have tryed many times to get igg to remove them and nothing happens i spoke out on there fourm and two days later my acct was locked igg excuse i transfered chips that is a lie seems igg programers made one big mistake on there bot program if you go on poker deluex 3d you can see the bots plain as day there blue with f markes on them when i brought this to igg great service deparment they turned into bunch clams wont answer my emails i have screen shots at four differnt tables showing them
I seriously can’t thank you enough for the step by step backprop explanation! Easy to follow and you didn’t skip steps. This has helped so much with my AI class. Thank you
hi, Sir I have read your article and I got an understanding on artificial neural network and it is really valuable article but what I have a question is how we can train if the input is not in binary or if it is an ASCII character?
Thanks! This explains it very well! A couple things that would be neat to see as improvements would be to alter your java program you use to show the neural net in action and have it be a bit more interactive. By this I mean be able to step through say the first handful of calculations or just be able to stop it at will and then have it display the equations with real numbers as you click or mouse over the different parts of the model.
Thanks Mazur, “A Step by Step Backpropagation Example” is very well written compared to many other similar ones found in the net. I have a question, lets say the target output values are ’12’ and ’78’. Should I normalized these values to [0 ,1] ? If not, how do I calculate the output error where the training result is in range [0,1] ?
It would be even better if you could talk about the iterations
I mean you just took 1 training example
Please explain how to iterate all examples in dataset and epochs.
I don’t know how will I thanks to you for this excellent step by step explanation for the Backpropagation algorithm . Its a excellent work .and helpful for future work.
Concerning the backpropagation example, it was great for me to understand it. However, it doesn’t seem to mention the change of the biases in the process. Moreover, the image shows that the same bias is used on all neurons in the same layer which confused me a bit since each neuron usually has its own bias. It would be nice if you manage to modify the article based on these two ideas.
Very nice! I’ve used it to check my own implementation in Max/MSP and Python. However, after the first iteration my MSE equals 0.291027773694 instead of your 0.291027924. If I use the exact weight representations as in your example my MSE becomes 0.291027773717. Can’t exactly figure out what causes the difference(s).
Thanks for great example, but I have a question. When you calculated a partial derivative dEtotal/dOut1 you’ve got in the result a (-1) multiplier. Why?
(x^n)’ = n x^(n-1)
Thank you so much for your great article :). one question left, what will be the backpropagation look like when we do dropout? especially I was reading this paper https://arxiv.org/pdf/1705.02033.pdf which is a very interesting paper, but Im not able to figure out the backpropagation regarding their approach!. it says that for their approach in feedforward step, they will keep the highest neurons and give other neurons zero, but sum up their activation and add up to the highest neuron. it means the activation of the highest neuron will be Z=0.8+alpha* 0.3, which 0.8 is the activation of the highest neuron and 0.3 is the summation of the other neurons. then in backpropagation, this 0.3 will be propagated to the neurons which they made inactive with alpha amplification. how it possible really? how the inactive neuron will participate only with alpha amplification to update the weight. do you mind have a look on the proposed method and share your idea?
Thanks :)
Thanks for the note, it’s very well explained.
-I would suggest you take care the change in the weights for the bias.
-if you use the relu function instead of the sigmoid fuinction, the back prop number of iterations will fall down from 10K to 30.
relu(x)=0 if x<0 else x and
dRelu(x)/dx=0 if x<0 else 1
Hey Matt! One of your preferences for SQL style that I disagree with is keeping operators at the end of a line, rather than on a new line. For many, the first words/commands of a line are the most obvious. For example, in a case statement, it is more readable in the following format:
case
when {condition 1}
then {output 1}
when {condition 2}
…
end as {field name}
Similarly, it is easier to read the following, with “and” on a new line:
select
id,
updated_at_utc
from {table name}
where id is not null
and updated_at_utc is not null
Interesting… I never considered the parallel between having “when” at the start of the line and having “and” at the start of the line. Totally get the argument for having operators first, though I’m probably gonna stick with it at the end – just looks cleaner to me.
Matt Mazur 
iam hopeing you can help igg inc aka texas holdem poker deluex is useing bots to cheat its players i have tryed many times to get igg to remove them and nothing happens i spoke out on there fourm and two days later my acct was locked igg excuse i transfered chips that is a lie seems igg programers made one big mistake on there bot program if you go on poker deluex 3d you can see the bots plain as day there blue with f markes on them when i brought this to igg great service deparment they turned into bunch clams wont answer my emails i have screen shots at four differnt tables showing them
Thank you so much, I can’t understand back propagation algorithm before read your article.
I seriously can’t thank you enough for the step by step backprop explanation! Easy to follow and you didn’t skip steps. This has helped so much with my AI class. Thank you
Thanks ,It was a very clear explanation of back propagation .
Thank you so much for the detailed explanation on back propagation
I spent entire day doing what you did on paper myself. It was an awesome day since you inspired me. Appreciate your brave and kind efforts.
Very good explanation. Thanks.
hi, Sir I have read your article and I got an understanding on artificial neural network and it is really valuable article but what I have a question is how we can train if the input is not in binary or if it is an ASCII character?
Doubt: do we need to update biasing
Thanks! This explains it very well! A couple things that would be neat to see as improvements would be to alter your java program you use to show the neural net in action and have it be a bit more interactive. By this I mean be able to step through say the first handful of calculations or just be able to stop it at will and then have it display the equations with real numbers as you click or mouse over the different parts of the model.
You are a great teacher!
how to determin the initial weight and bais dependend on the actual input and output
Hi – how can I update my comments, I dont see an option.please let me know.
i just have one question about biases.
bias’s weight doesn’t change?
The tutorials I read when writing this didn’t update it so I didn’t here. It is something you can add on though.
Thanks Mazur, “A Step by Step Backpropagation Example” is very well written compared to many other similar ones found in the net. I have a question, lets say the target output values are ’12’ and ’78’. Should I normalized these values to [0 ,1] ? If not, how do I calculate the output error where the training result is in range [0,1] ?
It would be even better if you could talk about the iterations
I mean you just took 1 training example
Please explain how to iterate all examples in dataset and epochs.
sir i need your help in my final year project……….
I am going to join the others on here and say thanks for great explanation of back propagation!
I don’t know how will I thanks to you for this excellent step by step explanation for the Backpropagation algorithm . Its a excellent work .and helpful for future work.
Concerning the backpropagation example, it was great for me to understand it. However, it doesn’t seem to mention the change of the biases in the process. Moreover, the image shows that the same bias is used on all neurons in the same layer which confused me a bit since each neuron usually has its own bias. It would be nice if you manage to modify the article based on these two ideas.
Very nice! I’ve used it to check my own implementation in Max/MSP and Python. However, after the first iteration my MSE equals 0.291027773694 instead of your 0.291027924. If I use the exact weight representations as in your example my MSE becomes 0.291027773717. Can’t exactly figure out what causes the difference(s).
thanks you are great i confused before know is better so please tell me where can i dowload for this exciting example
Example Max/MSP patcher using the included Mac 64-bit externals as attached.
Mac external sources on GitHub: https://github.com/mdehoogh/max_nn.git
Best,
Marc de Hoogh
Delft University of Technology, The Netherlands
Thank you for the explanation of MLP.
This is great :) Thank you!
This is great :) Thank you Matt!
Thank you ! about “step by step backprop explanation”!
Thanks for great example, but I have a question. When you calculated a partial derivative dEtotal/dOut1 you’ve got in the result a (-1) multiplier. Why?
(x^n)’ = n x^(n-1)
Hi, a small typo in net-h1, it should be 0.25*0.1, instead of 0.2 * 0.1
Hai…
I neee boot poker to play in dewapoker. Can you help me sir…
Nice bro…and very helpful
look like you work is copied
Thank you so much for your great article :). one question left, what will be the backpropagation look like when we do dropout? especially I was reading this paper https://arxiv.org/pdf/1705.02033.pdf which is a very interesting paper, but Im not able to figure out the backpropagation regarding their approach!. it says that for their approach in feedforward step, they will keep the highest neurons and give other neurons zero, but sum up their activation and add up to the highest neuron. it means the activation of the highest neuron will be Z=0.8+alpha* 0.3, which 0.8 is the activation of the highest neuron and 0.3 is the summation of the other neurons. then in backpropagation, this 0.3 will be propagated to the neurons which they made inactive with alpha amplification. how it possible really? how the inactive neuron will participate only with alpha amplification to update the weight. do you mind have a look on the proposed method and share your idea?
Thanks :)
Excelente and useful article.
Really take better help… i am having one query: what do we say of error equation? is there any name ? please describe me.
Thanks for the note, it’s very well explained.
-I would suggest you take care the change in the weights for the bias.
-if you use the relu function instead of the sigmoid fuinction, the back prop number of iterations will fall down from 10K to 30.
relu(x)=0 if x<0 else x and
dRelu(x)/dx=0 if x<0 else 1
Hey Matt! One of your preferences for SQL style that I disagree with is keeping operators at the end of a line, rather than on a new line. For many, the first words/commands of a line are the most obvious. For example, in a case statement, it is more readable in the following format:
case
when {condition 1}
then {output 1}
when {condition 2}
…
end as {field name}
Similarly, it is easier to read the following, with “and” on a new line:
select
id,
updated_at_utc
from {table name}
where id is not null
and updated_at_utc is not null
Interesting… I never considered the parallel between having “when” at the start of the line and having “and” at the start of the line. Totally get the argument for having operators first, though I’m probably gonna stick with it at the end – just looks cleaner to me.