Crunching Numbers

I listened to the 2/11/11 Science Podcast about a month ago where they interviewed the author of a paper about the World's technological capacity to handle information.

One of the co-authors of the paper, Martin Hilbert discussed how they were trying to determine how the world has changed in handling information, which he defines as storing, communicating, or computing the information. Communication of data is either one way, broadcasting, or two way, telecommunication. Computation is compiling information by computers or little controllers that are in everything.

Nature is awash in information, but we don't notice or capture most of it. Much of the information that we do capture has a tendency to be lost and little of the rest of it is actually compiled into a useful form.

My thought when I hear this was that the power of information goes up exponentially as it is stored, communicated, and then compiled. In fact, the biggest limitation in using information in the past was computational power. This is why individuals like Newton, who developed Calculus and found a way to reduce information into its simplified essence were responsible for great leaps in thinking.

Genius, or even a well trained mind, is the human equivalent of an efficient or effective compiler of information. Someone that has absorbed a lot of information and has a brain that is good at sorting it out and compacting it into core kernels, essential facts, and basic truths. Sometimes this happens with some insight because a person is in the zone or attuned to the phenomenon they are observing. Sometimes it happens because the person’s mind is a combination of a steel trap, but also a fine filter of the information, or a good discriminator of good information and bad information. This is where written history multiplies human capacity. The deeper the knowledge bed for a fertile mind to till, the more productive the output can be. When linked with the rapid access of information, even people with only average compilation skills can put together powerful constructs and conclusions.

This power of computation, of compiling information to its essence, is an idea whose time has come. Recently, a computer named Watson was put on the game show Jeopardy to compete against humans. He readily beat the humans, but the designers of Watson were frank about the machine’s limitations. It can compile huge amounts of data quickly, but it doesn't know what the information means, or which computations are more valuable than others. They gave an example of one possible application for Watson by saying that a computer with his capabilities could be designed and tasked to read all the medical journals available from this point back and then continuing into the future after it was put into service as new information is learned or collected. If provided with a medical case, the information and symptoms of a particular patient, the computer could spit out all cases that correlated with the data. Some of it would be gibberish, coincidentally related data that is not actually pertinent to the particular case. The power of the computer could be exploited by mating it with a skilled medical person, who could look at the possibilities that the computer suggested and determine which is most likely. The computer crunches vast amounts of information, but the person interprets what that information means.

I've often thought about this in my work, how the information, if compiled properly, could work to shortcut the amount of time spent fumbling around trying to figure out things. Can you imagine if your workload could be culled down to a small amount of more certain actions? The amount of time you could save just nailing down sure things would be tremendous. I’ve stopped listening to news fluff programs and have been getting a lot of my information from podcasts that I’ve vetted and culled until I have a small core that is a source of facts that I trust. I find it easier to compile the truth of the situation when you listen to just the pure sources that dig deep into core truths and try not to waste time shouting obscenities people as they try to walk by and mind their own business.

So the fascinating interview about the interesting paper left me with a lot of hope about the future, but I had to spoil it by thinking about how this truly translated into current reality. People now are awash in broadcast information. Much of it is advertising, a case of questionable information at best, as it is designed to try to persuade you to purchase something, whether you need it or not. Then there is political information, which is very much like advertising. It is designed to try to get you to buy a political ideology and to try to persuade others to join this ideology, and to contest those that do not believe your positions. This poisoning of information has corrupted news sources, as people with pre-conceived ideas are choosing their news sources based on whether these sources are putting out information that they already agree with, and ignoring any data source that conflicts with their dearly held beliefs. The information is “compiled” in a deliberately deceptive manner. This is fed to the public, complete with instructions to disregard or react with hostility to anyone that questions or contradicts the core ideology.

This is bad enough, but even more problematic is the dumbing down of the data. Compilations that amount to “bad man hurts animals” or “tragic tale is villain’s fault” are often the main thrust of news programs. It’s as if people attention spans are too short to absorb the full truth and nuances of actual events. They would rather be fed a pre-digested take on events, even if it lacks depth of understanding or even basic veracity. People sometimes turn away from the complicated dialog to fixate on the simple and easy to comprehend story about the iconic events that often dominate news cycles, yet really amount to nothing important. It’s noise. Just like when a signal is corrupted by static, our information stream sometimes gets clogged by rubbish. This makes it hard to hear the real truth when it is being drowned out by random loud annoying noise. Sometimes noise such as this can generate heat, as if the masses have listened to the lunatic scratching his nails on the chalkboard and screaming about the end of the world. You listen to such cacophony at your own risk.

I believe that what happened in Egypt and is trying to happen in the Middle East and places like China is that the raw compilation power of the general public is having the wool stripped off of their eyes for the first time. When the raw truths emerge, populations turn to their governments and leaders that have been contradicting those truths and become enraged and intolerant toward them. Propaganda and misdirection break down when we are awash in information. This can be truly enlightening and empowering for those under the thumb of a repressive regime, but you can imagine that they can be terrifying for those powerful leaders who built their massive house of cards on a foundation of lies and misdirection. This is not only true for governments, but for personalities and pundits and for large corporations. The key here is that pure unadulterated information is available to the masses so that they can decide what is right, what is the compiled essence of that information.

This information revolution is on an exponential track. That is what is really scary about it. What is going to happen to us when our ability to understand starts to approach our desire to focus our attention to a subject. Verner Vinge is the author of a book called Beyond Realtime, where he speculated that people’s understanding will accelerate so rapidly that we will simply evolve to the next realm. In his fictional account, most people simply disappeared. They had been linked mind to mind through a vast internet of humans that could communicate directly with the network and each other by their thoughts. Eventually, this lead to a hive mind kind of mentality and speculation that humans graduated to beings that no longer required physical bodies. Ray Kurzweil is a futurist and author who has written a book called The Singularity is Near: When Humans Transcend Biology. His work contains much the same conclusion. You can feel this acceleration of information and looming growth of computation embedded in society. The only question is whether it’s more like the coming of great things or doom.