Information Theory provides a way to understand neural activity patterns (spike trains). Most information theoretic analyses hinge on calculating the Shannon entropy of a sequence of action potentials. (Note, I will follow the convention of calling Shannon entropy just entropy and using the word entropy interchangeably with information.) The catch is one can't, in general, calculate entropy for spike trains. Noise confounds the calculation. You can average out the noise by repeating the experiment if you know how to reptitively stimulate that neuron. We can do this this for only a few cases like the early visual system, inner ear, and touch receptors. All those nerves have something in common- they respond to specific things. This narrow focus makes it easy to recreate the stimuli to which those neurons respond.
Most of the brain, unlike those early sensory neurons, receives thousands of poorly defined inputs. And we have no idea how to realistically stimulate them.
Enter the Boltzmann Machine. To calculate a firing pattern's entropy all you need to know is how likely that pattern is to occur. Put simply, if we can calculate the chance that some activity pattern occurs, we can then calculate entropy.
Luckily, the Boltzmann machine has a readily calculable probability distribution. So you can calculate the entropy of a pattern of firing in the system's Boltzmann machine representation.There's a leap of faith here. You can't observe all the activity patterns. So you won't know whether the Boltzmann machine representation of your network accurately represents the chance of some pattern occuring that you've never seen before. I'm working right now to see how outlandish that assumption is.
No comments:
Post a Comment