Structured Communication and Collective Cohesion Measured by Entropy

by Arlin Cooper and Rush Robinett III
Albuquerque, New Mexico
 


Interpersonal communication is essential to the success of large organizations and as a way to motivate personnel toward continuously improving performance. The implications are direct for company productivity, safety, security and quality. This article addresses strategic methods that can be used to improve communication, and metrics that can measure success or failure.

The following scenario will help to lay a foundation. Consider two people who contemplate going into business together, for example to purchase and operate an almond orchard. In order to succeed in the face of uncertain potential threats and risks, the two people should be well matched in their business interests, business philosophy, business skills, management philosophy, and amount of passion and drive. They should have similar levels of risk aversion and respond similarly to business-related stress. They should be mutually supportive, and without the tendency to deceive or let each other down. Both should have similar short-term and long-term expectations. They should respond similarly to external effects (e.g., damaging weather, water shortages, crop diseases or market downturns). Finally, they should be able to build trust in each other, and should have similar expectations of business exit strategies, should that become necessary.

Such matches are not likely to occur by accident, but they can be made more likely by a strategic, directed effort over a period of time. Planting early seeds that can result in later productive results is a manifestation of latent effects. An information-gathering effort over time can help build a common basis, or reveal that this is not possible.

As a common context for communication is developed, essential information can be exchanged. Each person needs to derive supporting information about the other. It is important for each to understand the other’s basic views and approaches, to know that there will not be devastating surprises brought to light, and to know how the other responds to stress. Obtaining these sorts of data is not often as direct as asking a question and listening to the answer, although that can be a contributor. Some information can come by strategic application of various forms of discussion, some from other people, and some from observation of responses to planned or unplanned situations. These forms of information may appear unpredictable, but they can be part of an overarching strategy. As the information aggregates, each piece is treated as new and useful (memory of previous data is not used to suppress importance). This form of information sending and receiving is termed unique signals.

As the latent effects and unique signal information exchanges take place, interpersonal communication effectiveness generally improves. When crucial business decisions must be made, often quickly and under stress, there is more likely to be effective communication between the partners with minimal chance of obfuscation or misunderstanding. This can be termed a low entropy communication, which opens the door to quantifiable metrics.

The concept of entropy serves as a basis for measuring interpersonal communication problems. Entropy is familiar to most technical personnel who have studied thermodynamic and/or electronic communication processes. Thermodynamic entropy generally increases as the available energy and organization in the known universe dissipate. This form of entropy becomes a measure of dispersion, or disorder. The Second Law of Thermodynamics states that the entropy of the universe will continue to increase. However, the presence of life, with its attendant cellular organization, information accumulation and skills development, is in temporary opposition to the overall order decrease. For each individual, beginning with apparent disorganization of basic cells and DNA, entropy decreases to a virtual minimum during the prime of life, until aging begins to relentlessly increase entropy, finally resulting in death. A general plot of the development and degradation of a living organism’s cellular (and, in some ways, intellectual) “order” is shown in Figure 1. Individuals, collective teams and civilizations can counter entropy by producing exergy [Ref. 1] (a measure of the ability of an entity to do work), information, learning, skills and strategies. All of these are ways in which the tendency of entropy to increase can be at least temporarily reversed.

Figure 1 — A Depiction of Human Cellular Order.

“Communication entropy” was developed by Claude Shannon [Ref. 2] to relate the probabilities of communication entities sent over a communication channel to “uncertainty,” with higher entropy measuring higher uncertainty about what entities might appear on the channel. Weaver [Ref. 3] equates high entropy with high “information.” This viewpoint has caused some confusion in the literature [e.g., Ref. 4], because it appears that information requires “organization,” which should relate to low entropy. We claim to have unraveled this confusion by noting that Weaver’s assertion was based on the transmitter’s (originator’s) viewpoint, which is different from the receiver’s. The transmitter has a high measure of information available if it has complete freedom of choice (no restrictions) among information entities (high entropy). The receiver has a high measure of entropy if any received information entity is equally likely (i.e., there is nothing noteworthy about the reception). This means that high entropy is an attribute for a transmitter, and low entropy is an attribute for a receiver (an apparent “zero-sum” situation).

"The concept of entropy serves as a basis for measuring interpersonal communication problems."


Because thermodynamic entropy and communication entropy both measure a quantity associated with disorganization, it is apparent that the two can be related. Reif [Ref. 5] and Frieden [Ref. 6] are authors who have demonstrated the basic relationship between thermodynamic entropy and communication entropy mathematically. The basis for this development was the exponentially distributed probabilistic nature of available thermodynamic states available for a system containing particular energy levels. The result shows that:

(1)

where is the number of system states containing particular levels of energy, k is a constant, and Pr is the probability of states at the rth energy level. The r probabilities (which can analogously represent communication entities) must sum to one. The natural logarithm results from the exponential distribution. A logarithmic relationship is essential to all expressions of entropy.

Entropy changes have a reversible and an irreversible component, as shown in Equation 2.

(2)

In a system such as that shown in Figure 1, there can be energy derived from matter, such that the entropy change represented by the first term is negative, and there can be non-equilibrium over a period of time. However, the change represented by the irreversible entropy term is positive, so that entropy will eventually increase. It is the time dynamics that allow human ordering to be achieved. The implications of the above associations will be addressed subsequently.

Another nuance of communication involves social interaction as a potential basis for later communication. Consider two1 people who participate in various activities together, such as playing sports, attending school or church, dining, discussing weather, health, family, etc. The communication efficiency in such situations does not appear high, because considerable time is expended and a relatively small amount of focus is generally placed on information exchange. The amount of immediately meaningful information is low, so this would be measured as relatively high communication entropy. However, these types of interactions tend to build a common context for future communication and understanding. There is also enhancement of the synchronization between the transmitters and the receivers. For these reasons, the long-term impact of the exchanges can be significant. From this overarching viewpoint, the communication entropy can be low.

From these foundations, entropy concepts can be developed as a communication analysis approach that can assist in measuring the details of, and the overall effects of, interpersonal communication. The following sections present the general concepts of entropy, and then address specific applications of various forms of entropy to interpersonal communication. Lastly, this article ties the concepts together in an analysis approach that can be used to measure the success of interpersonal communication strategies.

Thermodynamic Entropy
An important basis for the properties of entropy that apply to the measurement of interpersonal communication is thermodynamic entropy. Thermodynamic entropy is associated with energy, heat and temperature [Ref. 7], and it is limited by definition to values no lower than zero at absolute zero degrees Kelvin. Its maximum value is not specifically limited. The basic relation can be expressed by the “Gibbs” equation [Ref. 4] as:

(3)

where is the increase in entropy as ability to do useful work is lost, is absolute temperature, is change in free energy, and is the change in enthalpy (heat content).

The Boltzman/Schröedinger equation [Ref. 4] relates entropy to “atomistic disorder,” introducing a symbol D (disorder, or ratio of final microstates to initial microstates). This form of entropy can take on values ranging from zero (unity order) to infinity.

(4)

In Equation 4, k is the Boltzmann constant, 1.38 x 10-23J/K. The Second Law of Thermodynamics assures that “order” in the universe will ultimately be lost. However, the existence of entities such as life and the generation of information show that order within subsystems can temporarily increase. In fact, the existence of human life, coupled with learning and information, is arguably the most important example of entropy decrease and exergy increase. This means that utilizing the theory of entropy can contribute to measured increases in the exergy-based quality of life (e.g., energy infrastructures) in the relative short term of human existence.

Communication Entropy
A similar concept is communication entropy. Communication entropy, developed from earlier concepts by Claude Shannon [Ref. 2], is a metric that relates communication channel efficiency to the likelihood of communication entities. It can be normalized between 0 and 1, with highest entropy for optimally efficient systems. Shannon called this form of entropy a measure of uncertainty. For ordinary block coding (e.g., ASCII), high-entropy systems are the most uncertain, most efficient, and least associated with meaningful information transfer. This is because maximum-entropy systems place minimal constraints on the transmitter; i.e., “random” information is available. Non-random information is an important potential means of decreasing entropy. The mathematical measure of Shannon communication entropy is:

(5)

where there are j potential “messages” (units of information), the ith having a probability pi of occurring. Note that a natural logarithm is not used, nor is it necessary to be compatible with Equation 1. A common approach in the literature is to use a base-two logarithm and measure multi-symbol entropy in terms of the number of equivalent “bits.” For our development, the form of the logarithm in Equation 5 (more general than base two) is chosen so that the entropy can be normalized between zero and one.

From a transmitter’s viewpoint, highest-entropy systems are advantageous because they are most efficient, and the transmitter can more freely choose information symbols. For example, the maximum solution to Equation 5 occurs for equally likely (random) inputs. From a receiver’s viewpoint, highest-entropy randomness is associated with less noteworthiness, which reduces the most useful amount of received information (equal likelihood occurrences minimize noteworthy information). This is what is known in game theory [Ref. 8] as a zero-sum game (benefiting one entity is detrimental to another). However, we will show how to use particular strategies and the important lever of time to create mutual advantage and a beneficial non-zero-sum game.

As an example application of entropy metrics, unique signals [Ref. 9] are used in the U.S. weapons program for signifying an unambiguous intent to use a weapon. Unique signals are carefully engineered to have high entropy for all subset bit lengths so that an unintended reception has minimal useful information. Conversely, transmission of intended unique signals conveys unambiguous low-entropy information. This property provides important insight into the enhancement of long-term interpersonal communication and into the different bases associated with transmission and reception.

____________________
1
This could apply to any number of communicators, but two simplifies the description.