It from Bit

The universe is composed of energy, matter, and information. However, it is not energy nor matter that makes the Earth unique, but information. No known place in the known universe holds more information than Earth.  It is said, our planet is to information what a black hole is to matter and what a star is to energy. The Earth is where information lives and grows in an otherwise information-less universe. For billions of years, information has continued to grow in our planet from chemistry to simple unicellular life forms and then to multicellular organisms. “What lies at the heart of every living this is not a fire, warm breath, not a spark of life. It is information, words, instructions….a billion discrete, digital characters carved in tablets of crystal.” The measures of information have climbed up an exponential scale as information has grown exponentially. By 2010, the Library of Congress had collected 160 terabytes (10¹²) of information. Every day, we create 2.5 quintillion bytes of data — so much that 90% of the data in the history of the world has been created in the last two years alone. We live in a world about which we have more and more information and we slave away to serve the growth of that information. “Man the food-gatherer [has reappeared] incongruously as the information – gatherer.”  However, what is information? What are its components? Also (and maybe more importantly), what is it not?  

Information theory began as a bridge from mathematics to electrical engineering and from there to computing. Like force, mass, motion, and energy before, information had to be mathematically defined. In 1948, Claude Shannon published “A Mathematical Theory of Communication” in which he defined information as the minimum volume of communication required to uniquely specify a message. He coined the term bit as the unit for measuring information. Information is the number of bits needed to communicate an arrangement or the state of a system. The more ambiguity or uncertainty or entropy in the message or a system, the higher the number of bits required to communicate that message or system. However, information is more than just the number of bits. In the steady state of out of equilibrium physical systems, information (or negative entropy) is the opposite of entropy. It involves highly correlated configurations that are difficult to arrive at spontaneously. Although the scale at which it is embodied is not important, information itself is physical and is what gives shapes and structure to things. Structured, information rich macroscopic states such as the Sumerian script transcribed on clay tablets microscopic double helix strands of DNA are both embodiments of information. Furthermore, information exists independent from our ability to decipher it. It has been there and exists independent of meaning. It is hard for us to separate information from meaning because our brains innately seek out patterns but the meaning of the message is a function of the agent processing that information. We infuse messages with meaning automatically, fooling ourselves to believe that the meaning of a message is carried in the message. Meaning is derived from context and prior knowledge.  

Information theory created a formal, mathematical definition of information and has served as the bedrock of the explosion in information transmission, processing, and collection. It has found application in a wide field of areas ranging from statistical inference, natural language processing, and cryptography to linguistics, psychology, and anthropology. Shannon himself foresaw this development as he said, “this present century in a sense will see a great upsurge and development of this whole information business; the business of collecting information and the business of transmitting it from one point to another, and perhaps most important of all, the business of processing it.” The physicist, John Archibald Wheeler, offered It from Bit because “every particle, every field of force, even the space-time continuum itself – derives its function, its meaning, its very existence….from bits.” In the next few blogs, I will attempt to develop this idea of information as it relates to topics of personal interest – biology, health, and physician decision making.