A simple attempt to implement Back Propagation Algorithm in XI (Part -2)
The part 2 of the blog “XI: The future of neural networks”.
Before even going into the blog, please go through the following blogs and come back.
XI: The future of neural networks (Part - 1) and A critical analysis and not criticism
I would like to thank Sudhir Porumamilla for giving his excellent analysis on my first part of this blog series.
On the first part of this blog series, more about BPN and planning to implement it on XI with a few designs are explained. In this blog, I took a simple design to implement XOR gate learning using XI. Since I had already explained much about BPN and it’s implementation designs, here I am going directly into the technical implementation of XOR gate learning in XI.
Data types Used:
Input and output data type structure for training:
Input and output data type for production:
I had included the hidden and output layer weights as an input so that it makes easy for debugging and control the data flow inside BPM. Both input and output data types are same for both training as well as for production.
The BPM design
The above BPM is a very simple one. It’s actually two different flows integrated into one BPM. The flow is forked and spitted into two, one for training and the other for production. The training trains the network and updates the weights which is again written as a file (same name) and again fetched by the training receiver in BPM. The production flow is very simple in fact the weights are manually sent into the BPM as XML tags.
Weights can be kept in memory with the help of container variables and fetched again with training loop as infinite and production forked inside. But to avoid any complexity, I haven’t used any such variables and all data goes in and out as XML files. I would like to mention a wonderful blog by Michal Krawczyk, regarding The specified item was not found. which can be used to store and retain variables temporarily, which gives more information for anyone wants to implement this using container variables and infinite loop. Please remember to kill the BPM process using transaction SWWL if you want to stop an infinite loop in BPM.
Mappings Used
Common for both training and production
Input to Hidden
How pattern is switched for every training pattern tag from ‘n’ occurrences to individual tags
The given snapshot is for input2 in training pattern. This is same for input1 as well as actual_output. There is no difference at all.
Pattern selector
Pattern Switcher
Calculation of Hidden1 value
This is same for Hidden2 also.
The above snapshot is the implementation of the following formulas
and
Sigmoid Function
The above screen shot implements the following formula:
The above set of snaps is for Input to Hidden layer. Similar implementations are required for Hidden to Output.
Again for training we need Output to Hidden and Hidden to Input mappings to implement backpropagation.
Some snaps showing important implementation steps.
Calculation of output layer weight (just a part of the complete is visible)
Calculation of hidden layer weight (just a part of the complete is visible)
The above both are the implementation of the following formula
All the above mentioned have more weightage on XI rather than Java in it. Better designs with java modules are most welcomed since performance can improve exponentially.
This is just a simple design proving the ability of SAP-XI to implement neural networks taking backpropagation net as an example. Regarding performance issues and other designs were discussed in the above mentioned blog. I will never say that this is the best design. This is just to prove the ability of SAP - XI in the field of neural networks. It may be in version 3.0 a bit slower, but I am sure in next version and if not yet it the next coming versions, SAP XI will fine tune itself to prove much more than what I have blogged today. It is just the beginning and I will surely see how XI is going to transform the world in the field of neural networks in years to come. 😉
Other neural networks, analysis and ANN applications in SAP-XI (Part - 3) |
1 Comment
