Question 2:
Input pattern used: \(\left[0,\ 1\right]\)
Learning Rate, \(\alpha\ =\ 0.25\)
Desired output, \(d\ =\ 1\)
Usinng Binary Sigmoid as activation function:
\(\)\(\phi=\frac{1}{1\ +\ e^{-x}}\)
\(\phi'=\left(\phi\right)\left(1-\phi\right)\)
Forward Pass:
Input layer to middle layer:
\(Z_1in\ =\ \left(0\right)\left(0.6\right)\ +\ \left(1\right)\left(0.1\right)\ +\ \left(1\right)\left(0.3\right)\ =\ 0.4\)
\(Z_2in\ =\ \left(1\right)\left(0.4\right)\ +\ \left(0\right)\left(0.3\right)\ +\ \left(1\right)\left(0.5\right)\ =0.9\)
Hidden Layer:
\(Z_1out=\ \frac{1}{1\ +\ e^{-0.2}}\ =\ 0.5498339973\)
\(Z_2out=\ \frac{1}{1\ +\ e^{-0.9}}\ =\ 0.7109495026\)
Output layer:
\(Yin\ =\ \left(0.5498339973\right)\left(0.4\right)\ +\ \left(0.7109495026\right)\left(0.1\right)\ =\ 0.09102854918\)
\(Yout\ =\ \frac{1}{1\ +\ e^{-0.09102854918}}\ =\ 0.5227414361\)
Backward Pass:
Output Layer:
\(e\ =\ d\ -\ y\ =\ 1\ -\ 0.5227414361\ =\ 0.4772585639\)
\(\delta\ =\ 0.5227414361\left(1-0.5227414361\right)\left(0.4772585639\right)\ =\ 0.1190678158\)
Hidden Layer:
\(e_{11}\ =\ \left(0.4\right)\left(0.1190678158\right)\ =\ 0.04762712632\)
\(\delta_{11}\ =\ \left(0.5498339973\right)\left(1-0.5498339973\right)\left(0.04762712632\right)=\ 0.01178850307\)
\(e_{12}\ =\ \left(0.1\right)\left(0.1190678158\right)\ =\ 0.01190678158\)
\(\delta_{12}\ =\ \left(0.7109495026\right)\left(1-0.7109495026\right)\left(0.01190678158\right)=\ 2.446885169\ x\ 10^{-3}\)
Weight Updates:
\(Weights\ to\ Y:\)
\(w_{01}\ =\ w_{01}+\alpha\delta x_{w_{01}}=0.4\ +\ \left(0.25\right)\left(0.1190678158\right)\left(0.5498339973\right)=\ 0.4163668833\)
\(w_{02}=\ 0.1+\ \left(0.25\right)\left(0.1190678158\right)\left(0.7109495026\right)\ =\ 0.1211628011\)
\(w_{03}=\ -0.2\ +\ \left(0.25\right)\left(0.1190678158\right)\left(1\right)\ =\ -0.1702330461\)
\(Weights\ to\ Z_1:\)
\(w_{11}=\ 0.6\ +\ \left(0.25\right)\left(0.01178850307\right)\left(0\right)\ =\ 0.6\)
\(w_{12}=\ -0.1\ +\ \left(0.25\right)\left(0.01178850307\right)\left(1\right)\ =\ -0.099705287423\)
\(w_{13}=\ 0.3\ +\ \left(0.25\right)\left(0.01178850307\right)\left(1\right)\ =\ 0.3029471258\)
\(Weights\ to\ Z_2:\)
\(w_{21}=\ 0.4\ +\ \left(0.25\right)\left(2.446885169\ x\ 10^{-3}\right)\left(1\right)\ =\ 0.4006117213\)
\(w_{22}=\ -0.3\ +\ \left(0.25\right)\left(2.446885169\ x\ 10^{-3}\right)\left(0\right)\ =\ -0.3\)
\(w_{23}=\ 0.5\ +\ \left(0.25\right)\left(2.446885169\ x\ 10^{-3}\right)\left(1\right)\ =0.5006117213\)