Difference between parallel processing done by human brain and by computers

Question

I am asking a question regarding parallel processing as done by billions of Neurons inside our brain and parallel processing done by our computers in a cluster for example or even on a Graphics processor having 500 processing cores.

Recently we have seen great success in applying computers to solve complex problems applying parallel processing, such as in the field of scientific simulations, modeling, video/image processing etc. Also due to limitations on semiconductor devices, chip makers are now forced to develop multi-core CPUs. This encourages and motivates software developers to apply parallel programming to gain the required performance, exploiting all the processing cores inside the CPU.

While people are encouraging to solve problems using parallel programming on the computers, at the same time generally discouraging to do parallel processing inside the brain. Expert say that we must concentrate in one dimension to get best results. Why can't human brains be used to do massive parallel processing in the same way computers are doing today? The brain is having billions of computational units called Neurons, and thus it provides an opportunity to do processing at both coarser level and at finer level. Why can't we use our brain to do certain jobs in parallel in the same way as computers, for example, while I am writing this question using my right hand, why cant I solve a problem in geometry at the same time, using my left hand? Using much less powerful computers having just two processing cores (vs billions in brain) I can do these two jobs of writing and solving geometry problem, in parallel by launching two separate threads. Why it is so?

Answer 1

There is a passage in On intelligence about the differences between parallel processing in human versus computers :

From the dawn of the industrial revolution, people have viewed the brain as some sort of machine. They knew there weren't gears and cogs in the head, but it was the best metaphor they had. Somehow information entered the brain and the brain-machine determined how the body should react. During the computer age, the brain has been viewed as a particular type of machine, the programmable computer. And as we saw in chapter 1, AI researchers have stuck with this view, arguing that their lack of progress is only due to how small and slow computers remain compared to the human brain. Today's computers may be equivalent only to a cockroach brain, they say, but when we make bigger and faster computers they will be as intelligent as humans.

There is a largely ignored problem with this brain-as-computer analogy. Neurons are quite slow compared to the transistors in a computer. A neuron collects inputs from its synapses, and combines these inputs together to decide when to output a spike to other neurons. A typical neuron can do this and reset itself in about five milliseconds (5 ms), or around two hundred times per second. This may seem fast, but a modern silicon-based computer can do one billion operations in a second. This means a basic computer operation is five million times faster than the basic operation in your brain! That is a very, very big difference. So how is it possible that a brain could be faster and more powerful than our fastest digital computers? "No problem," say the brain-as-computer people. "The brain is a parallel computer. It has billions of cells all computing at the same time. This parallelism vastly multiplies the processing power of the biological brain."

I always felt this argument was a fallacy, and a simple thought experiment shows why. It is called the "one hundred–step rule." A human can perform significant tasks in much less time than a second. For example, I could show you a photograph and ask you to determine if there is cat in the image. Your job would be to push a button if there is a cat, but not if you see a bear or a warthog or a turnip. This task is difficult or impossible for a computer to perform today, yet a human can do it reliably in half a second or less. But neurons are slow, so in that half a second, the information entering your brain can only traverse a chain one hundred neurons long. That is, the brain "computes" solutions to problems like this in one hundred steps or fewer, regardless of how many total neurons might be involved. From the time light enters your eye to the time you press the button, a chain no longer than one hundred neurons could be involved. A digital computer attempting to solve the same problem would take billions of steps. One hundred computer instructions are barely enough to move a single character on the computer's display, let alone do something interesting.

But if I have many millions of neurons working together, isn't that like a parallel computer? Not really. Brains operate in parallel and parallel computers operate in parallel, but that's the only thing they have in common. Parallel computers combine many fast computers to work on large problems such as computing tomorrow's weather. To predict the weather you have to compute the physical conditions at many points on the planet. Each computer can work on a different location at the same time. But even though there may be hundreds or even thousands of computers working in parallel, the individual computers still need to perform billions or trillions of steps to accomplish their task. The largest conceivable parallel computer can't do anything useful in one hundred steps, no matter how large or how fast.

Here is an analogy. Suppose I ask you to carry one hundred stone blocks across a desert. You can carry one stone at a time and it takes a million steps to cross the desert. You figure this will take a long time to complete by yourself, so you recruit a hundred workers to do it in parallel. The task now goes a hundred times faster, but it still requires a minimum of a million steps to cross the desert. Hiring more workers— even a thousand workers— wouldn't provide any additional gain. No matter how many workers you hire, the problem cannot be solved in less time than it takes to walk a million steps. The same is true for parallel computers. After a point, adding more processors doesn't make a difference. A computer, no matter how many processors it might have and no matter how fast it runs, cannot "compute" the answer to difficult problems in one hundred steps.

So how can a brain perform difficult tasks in one hundred steps that the largest parallel computer imaginable can't solve in a million or a billion steps? The answer is the brain doesn't "compute" the answers to problems; it retrieves the answers from memory. In essence, the answers were stored in memory a long time ago. It only takes a few steps to retrieve something from memory. Slow neurons are not only fast enough to do this, but they constitute the memory themselves. The entire cortex is a memory system. It isn't a computer at all.

The point made here is that the computing paradigm (that is, the way the whole thing works) of the brain and the computer are completely different. The computer is a Turing machine, and the brain is something else, possibly a memory system if you think that Jeff Hawking is right. Whatever it is, the brain is not a Turing machine.

To go back to your question:

Why can't human brains be used to do massive parallel processing in the same way computers are doing today?

It has to do with the way the human brain works. If you assume that the brain will do any task in a parallel fashion, and the more neurons involved, the better the performance; then in order to maximize your performance you should use your whole brain. 1 task: 100% performance, 2 tasks: 50% performance, 3 tasks: 33% performance, and so on.

But if you add an "attention switching cost" to go from one task to another, then you are better off just focusing on one task where the switching cost is zero.

So you can multitask, but it won't be efficient.

Answer 2

Maybe I just don't get it, but I see your question as confusing because:

1/ Your brain is capable of running multiple parallel processes. Actualy each one of tasks you've mentioned consists of number of processes that are done at the same time. Lots of your neurons and neuronal networks are being used at the very same moment.

2/ If you can do something automatically, without thinking about it, but not without using our brain to guide the action (there are very few things you can do without actually using your brain and its neurons), you will find that you actually are able to do even more complex tasks in parallel. Even the two you've mentioned.

3/ So the problem of you not being able to do some tasks together is a problem of you not being able to concentrate at both of them as good, as you do with one task. Your question asks for the difference using a metaphor and comparison of the 'hardware' of your brain, but the problem you are asking about goes on higher cognition. You not being able to effectively 'double' your attention, you having limited working memory capacity.

Answer 3

I think you are beginning to push at the limits of human brain versus computer metaphor.(mildly related link here) I'll list the objections as:

1. While neurons firing can be translated/compared to 0 or 1 states, am not convinced it is a valid equivalence. (i.e: simulating 100 billion neurons on a computer would still not be able to match a human brain). My suspicions are while we have gone through a very accelerated evolution in building and controlling the performance of transistors, evolution hasn't progressed to that stage in terms of brain development.

2. Our neuronal cells do exhibit variations(range of electrical activity, not just spikes and flats) and in some sense you can call procedural learnt tasks to be parallel processing. i.e: you can speak and drive to some extent etc..Not to mention the Autonomic nervous system and it's functionalities like breathing, heart beat etc..Here's the paper i was referring to. My point being this can be considered as parallel processing.

3. Another ignored factor is the resolution at which we can control/program processors to compute and co-ordinate with each other on a single problem. Parallel processing is hardly a solved problem in the programming world.

4. Yet another factor that is ignored is that if you do zoom down on the computers to the level of capacitors, they are not all binary and have some analog activities too.This article mentions it, though it goes on to make a different point.

5. Another relevant point is that there are algorithms that cannot be sped up by using parallel computing. For ex: iterative algorithms like Newton's method, in these cases the next computation depends on the immediately preceding computation and so has to wait till the previous computation is done.

So to summarize, our brains do some forms of parallel processing already.It may just be that the jobs of writing and solving geometry problem have a set of processes too much in common that they cannot be done parallel.

Answer 4

This probably won't be a very good answer, but I studied this a bit in a Human Factors class I took this semester. The metaphor that the professor used is a bucket of mental resources. We have several buckets for certain things. When we are using resources from the bucket the bucket has less resources for other things.

We can walk and talk at the same time because they are using different buckets( motor-physical and language processing ). However we are very bad at reading and talking to someone at the same time because both task are taking up mental resources from the same bucket( language processing ).

So, yes we are able to do various things well that aren't requiring resources from the same bucket, but these buckets are only so deep.

Maybe that makes since. Oh yeah there's a more technical term, maybe Multi-Channel Human Information Processing. I know if you researched Human Information Processing( HIP ) you would stumble upon what you may be looking for.

Answer 5

First of all I should say that I'm not a neuroscientist so take my answer with a grain of salt. Your first question was "Why can't human brains be used to do massive parallel processing in the same way computers are doing today?"

I think the answer to this question has to do with the difference between the multi-core processors computers have and the nuro networks that exist in the human brain. Most processors on a regular computer, with the exception of the GPU, are general purpose processors so that any program or algorithm can be run on any core of the computer.

The human brain on the other hand has specific areas of the brain specializing in processing different kinds of input.

In the human brain the part of the human brain which controls higher cognitive function is called the prefrontal cortex and one of the jobs of the prefrontal cortex is to determine what from other parts of the brain to listen to and what parts of the brain should be ignored.

Also I question your assertion that the human brain does not in fact already engage in parallel processing. Anybody able to hold a conversation while walking or driving is engaging in parallel processing. This is possible because the areas of the brain engaged in language processing are different than the areas associated with image processing and movement.

Answer 6

As computers have a different architecture, you are all just comparing an Apple with a Pie, or a Melon with a Pineapple. How can such a comparison ever be correct? I have never seen any fruit tasting the same, and you all think the human brain(whereof we believe is conscious) is comparable with a computer whereof we don't even know if it is conscious after all. I am not a computer, so how will I ever know for sure a computer is conscious. How do I even know the people around me are conscious? I try to say, that you cannot only rely on a observation that's just a assumption of reality! That's why the holy bible said: Don't judge! And just to clarify, I do not deny any religion, neither do I try to prove it's legitimacy. Grow up please, and leave unanswerable questions out of the question. The get back to what we do know.

Humans can give input to a computer as we assume. Computers can't give input to humans, as far as we assume. Humans can decide as far as we assume, computers can't as far as we assume.

What's the specific problem in this case?

If I do just walk through a "Don't walk sign", I might get troubles! If a computer walks through a "Don't walk sign", it asks us what to do. Which decision is faster?

When calculating what to do next, a "Don't walk sign" involves abstract thinking, which is almost impossible to computers. Computers can do commands very easily, humans simply can't follow up commands easily. What do you think that happens if humans have a faster brain, but simply review their decision many times before making a decision, talking about emotions and fears and other stuff. We have to keep track of much more parameters, of which computers cannot even be aware of. Have you ever seen a small kid of about five years old, keeping track of all the parameters adults have to be aware of? Guess why adults learn slowly! Our brain isn't just placing ones and zeros on a harddrive! We are a living organism, and we cannot just place information somewhere. Our brain is continuously fragmenting, just like computers, but humans are defragmenting all of the time, where computers start do have the ability to perform such a task simultaneously to it's calculations. But humans are basically performing a few hundreds, maybe even a few thousands of brain tasks before we even do something. Have you ever seen computers keeping track of these parameters? Till then, it's just comparing a cockroach to a human. Does a cockroach know?

Answer 7

Others have answered to explain that the brain can indeed do parallel processing, or how the structure of the brain and a computer differ. I'm gonna answer your "Why?":

Because your brain was "built" to do specific tasks, and not others.

Your question is like asking why a car cannot fly, if planes can fly. The answer is simple: Because cars were optimized to drive on a road.

Your brain evolved to be perfect at helping you hunt antelopes, interact with other humans in a complex society, and survive virus infections.

---

That was the answer to the "Why?" of your question. What follows here are some thoughts on the extent and the form of the differences between a computer and the brain.

Your brain – which, as others have pointed out, actually does massive parallel processing – did not evolve to do "scientific simulations, modeling, video/image processing etc.".

Oh, wait, doesn't the brain process images? Sure, it processes the input from the eyes, and the constant, video-like flow of visual information your eyes send to the brain is certainly larger than any digital image or video file (which, by the way, a computer may be able to perform interesting looking video effects on, but quite simply cannot even begin to understand). Oh, and doesn't the brain also do scientific simulations and modeling? Sure, or how do you think we have an idea of the world and solve problems? When I sit down and wonder "what if", that isn't any different from a "scientific simulation".

So, it seems to me that the brain does everything that you say it doesn't. It even scans a huge database (i.e. your whole memory) in milliseconds.

But let's get back to the part where I said that the brain even fights a viral infection of the body. This seems exceptionally important to me:

The brain is connected and regulates the whole of the body that it resides in.

A computer processor does not even note if a key on the keyboard no longer works, much less repair itself. In a computer, the processor is separate and barely connected to most of the other parts that make up the whole of the computer. Your brain is not a distinct part of your body. It is so intimately interwoven with it that it is difficult to tell where it begins and ends. From everywhere in your body nerve fibres lead to your brain, and your brain is in fact the collection of the endpoints of these nerve fibres. You may have heard of the second nerve center, the so-called "second brain" in your guts. You have more neurons in your intestines than in your spinal cord. In a computer, there are no other parts where there are processors or their axons, and the connections of the processor to the interfaces, the memory and a handful of other parts, are few.

And then neurons and their electricity are only one channel of information flow in the brain/body-whole. Information in the human body is stored and processed in the form of chemicals also. And in the form of growth and decay of cells. The form and shape of your body is in fact also memory, and this information is available to your brain.

In short, the brain and the body are an inseparable unit. As research into emotion shows,

"processing" of information does not take place only in the brain, but in the whole of the body,

all the time, and the amount of information that the sum of these – parallel! – processes make up is as yet unfathomed.

A computer is a badly designed box in which a processor is screwed that does mathematical calculations. A human being is an organism that survives and procreates. How anyone could think of comparing the two is a mystery to me.

Yes, the human brain cannot count from 1 to one billion as quickly as a computer can, but that seems like an irrelevant exercise to me anyway. If you think of other complex calculations like the weather, a human who is given the current weather data can predict the weather almost as accurately as any computer can. What does that tell us about the impressive calculation power of computers?

What? Google's computers can scan billions of web pages and create translations that are almost correct? Interesting. I can come up with translations that are correct. Well, sure, someday Google may have perfected that algorithm to no longer make mistakes, but that's a lot of effort, people, time, and money, put into doing something that half of humanity can already do, and the other half could learn if they could go to school.

What? Computer software can render a photorealistic landscape? Looks good. With all the time the modelling, texturing and light set up took, a good human painter can do the same in acrylic or oils. And the painting even looks less sterile and more alive :-)