Artificial Neurons, A Hardware Problem First

Categories Uncategorized0 Comments

Always have been fascinated by our brain’s plasticity; neurons are constantly rebuilding and creating new connections to reflect a person’s current activities and needs!

The Neuron Machine

It is very hard for a binary rigid based computer as most of these connections are soldered or wired. Little to no advances have been made on a system that can “restructure” itself as our brains do (there are respectful efforts made, but on a bigger and limited scale).

Although it is possible to mimic the “physical” process by using switches in order to create new connections, it would still be hard to approach the number of connections a neuron can have. The average number of synaptic connections a neuron has in the human brain is 7000! To emulate that on current computers, we’d need a machine just to manage the connections for the secondary “neuron” machine ( I’m not even considering bottlenecks on the bus lines!).

So how can we improve or “evolve” the machine to resemble the neuron? The focus needs to go deeper than software. Onto the land of Hardware!

Let’s look at a neuron:

The main three parts of the neuron that are of particular interest are the Dendrites, Soma and Axon. Transposing them to the computer/machine world and we have Input, Processor and Output respectively. Now let’s look at a commercially available small motherboard 

Hopeless! We have to rethink the hardware before tackling the higher levels!

Restructuring the Hardware

Starting at the hardware makes perfect sense, but once we consider this, new problems arise. How will this impact our current computing paradigms? Will we still have a Von-Neumann Architecture or will we switch to something radically different? As seen above the transposition between the neuron and a computer is pretty straightforward but as much as a machine has inputs, they can only be so many and far behind the 7000 count of an average neuron! Similarly the same limitations exist for outputs.

Let us say we figured out a solution for the latter, we still need a bus line to handle the high amounts of gigabytes of throughput coming from these ports into the processor. Last I checked, processors average seven registers….!

Drafting a quick mathematical model, comparing the processor vs neuron computational power:

Processor-A has 3.0 GHz.(and let’s ignore pipe-lining which divides that number by roughly seven, as typical in pipelines that are partitioned by that amount; including branching). Resulting in roughly 428,571,428 Ops/Second with pipe-lining or 3×10^9 Ops/Second without. Now that is a respectable number, but if we look at a neuron and it’s 7000 connections, and dividing that in inputs and outputs, 3500 each, and further constrain one operation per connection across inputs and outputs which leaves us with: 3500 * 3500 = 12,250,000 between any two neurons. Adding 6998 to the picture and the number becomes literally un-typable (but if you insist, type this in any machine/calculator 12,250,000 ^ 7000… warned you!)

These speculations are made to further the known realization that we have to radically rethink the hardware to even begin to think about approaching the impressive computational power of neurons and don’t even get me started on the brain!