spikeprop for multiple spikes
For my Master of Science degree I did research on Spiking (or Pulsed) Neural Networks (SNNs) under supervision of Hieu tat Nguyen. Like other artificial neural network paradigms, SNNs model a group of connected biological neurons, as found in neural tissue (see this site on the simulation of SNNs or Wikipedia for an explanation of this model).
As for conventional neural networks a technique is needed to learn the free parameters of the model. A common approach is to use an unsupervised technique, for example a Hebbian learning rule. The reason for using an unsupervised rule, is that it is believed that biological neural networks use such a method, but on the other hand it is simply because SNNs lack a good supervised method of learning. The most practical supervised rules was the SpikeProp method, which works like the error-backpropagation rule for conventional neural networks. The only problem with this method was that it could only be applied on a restricted SNN-model, namely the neurons could spike only once.
In my thesis "Temporal Pattern Classification using Spiking Neural Networks" I describe how to extend the SpikeProp learning rule so it does not restrict the neurons to fire only once. The neurons that represent the input of the network can thus fire an unlimited number of spikes. Because of this a network can be fed with time-varying data such as time series or sound samples. As an example I apply an SNN which uses the new learning rule on small pieces of video of a talking mouth. The algorithm can learn the network to lipread, classifying the videos on the basis of what word is pronounced.
In addition I show that an SNN with only one layer can represent the exclusive-or (XOR) function. This is a well-known function in the field of artificial neural networks, because it is a simple function that can not be represented by a one-layered conventional neural network.
Journal Paper
Thanks to Sander Bohte I had the opportunity to compress my thesis into a journal article. My work appeared in a special issue on spiking neural networks in Information Processing Letters.
Software
I implemented the spikeprop algorithm and the tests in C++. The code clearly shows that it was one of my first C++ experiences. For those who dare to look, the code I used for my thesis can be found in this zipped tar (20 kb) and a slightly modified version, used for the article can be found in this one (71 kb). This code is actually owned by the University of Amsterdam and (thus...) not open source.
Award!
I got awarded the KION-price for my thesis for best Dutch AI-thesis for academic year 2004-2005 (Joepie!). Credit goes to Leo Dorst for submitting my thesis.
Copy-pasted
Samanwoy Ghosh-Dastidar and Hojjat Adel reproduced my results in "A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection", and gave the name "MultiSpikeProp" to it. A very good name indeed (it was also suggested to me by a reviewer of my 2005-paper, but I thought the contribution to the original SpikeProp was too little to give it its own name). If I ever re-implement my SpikeProp algorithm I will probably focus on the fact that it can be used on recursive networks, and will rename it to ReSpikeProp (TODO).
