The Thermodynamics of Free Will

The scientific discoveries of the last centuries have disenchanted human existence in many ways: once living in a world full of spirits and gods and primeval forces beyond understanding, our place in the universe was moved from its center right to the fringes of one among unimaginably many galaxies in an unbelievably large, mostly empty space.

We were equally thrown from the throne of creation: Darwin showed that we are just animals with a particularily large brain which is, while having the potential to write fugues and build cars in its free time, still mostly concerned with thinking about the four F’s: fighting, fleeing, feeding and reproduction.

In the genes, we discovered the blueprint for all living structures.

The magical life force, the Pneuma of the Ancient Greeks, the Prana of India, the spiritus of the Latin world, the élan vital of Bergson, was found, after slight adjustments to the theory, to be better explained by oxidation driving the ADP/ATP cycle within our cells.

Life was reduced to its biochemical mechanisms, and thus Life can now be seen as to emerge from lifeless matter without any magic left.

And then Cartesian and Leibnizian dualism with their assumption that the mind is something wholly different from the world of matter and the rationalist creed that our reason was quasi-divine and able to understand the true nature of the world through insights (in the Platonist sense, as “methexis”of the realm of Ideas: in the allegory of the cave, this means man leaving the cave and seeing the sun), have been on the decline for a long time, and most working scientists today are functional reductionists.

Concept after concept is getting disenchanted, and slowly but gradually our mind stops being something manifestly otherwordly: it is becoming an established paradigm of neuroscience that all our mental processes can be closely linked to physical processes in the brain (I won’t go into the ongoing discussion about questions like Chalmer’s hard problem of consciousness), and that every phenomenal property can, at least in principle, be mapped onto a functional equivalent in the brain (how that mapping works is a different question, but that it exists seems pretty clear).
As an example, seeing a face can always be traced to appropriate activity in the visual cortex and fusiform cortex that can be reliably replicated by providing the same stimuli again and again, the hearing of a note relates to activity in the auditory cortex etc. And vice versa, phenomenal properties can be created by stimulating the brain externally, e.g. by Transcranial Magnetic Stimulation of the relevant brain regions.

And then there is free will, once thought to be endowed to us by our creator in order to allow us choose between good and evil. And as it is the place on which many of our moral intuitions reside, still a bit magical.
Discussions about it are raging on, and tend to be fraught with confusions and frustrations, as parties involved in the dispute usually use widely differing conceptions of what free will is supposed to be and talk past each other. I have had many such conversations, and have probably talked past a couple of people myself.

I want to look at free will from the limited perspective of Thermodynamics, which can hopefully provide a new angle on how to think about it and maybe give a perspective that is both compatible with determinism, but also shows how something like will can be a useful concept worth preserving.

First, I need to clarify what I mean by Thermodynamics.

Generally speaking, Thermodynamics deals with the behaviour of hot bodies (in the rather boring sense of that expression) and cold bodies, with entropy and radiation and with building machines that can transform work into heat and vice versa.

On a different level, Thermodynamics is the study of how macroscopic (large scale properties of our everyday world) properties arise out of the interplay of very large quantities of microscopic properties. Boltzmann brilliantly connected this through one of the great laws of physics (that is written on his tombstone depicted below), which states that the entropy of a system is given by the natural logarithm of the number of possible microstates of the ensemble (in other words, the disorder of the system) times a constant.

Boltzmanns tombstone with his famous equation.

As an example, think of how to model the behaviour of a single atom. It’s complicated stuff, but physicists have a pretty good grasp about what’s going on with it. Take two atoms interacting, it gets much more complicated.
Take one hundred, and it‘s impossible even for the largest supercomputer to model precisely where each one of the atoms will be after five seconds of putting them together.

But take 10²³ atoms (written out, that’s 1 000 000 000 000 000 000 000 000 atoms), and suddenly, you can describe pretty well what they are going to do by leaving out much unwanted detail and focusing on what’s important.
That’s essentially what you do when you define a temperature. Your body temperature is a statistical property of the 7*10²⁷ atoms that make up your body, giving you their average kinetic energy (a measure of how much they move and wiggle around). You obviously have no idea how any one of these atoms behaves individually, and the computing power necessary to do so would be any Google employees wet dream, but it’s still rather easy to find out your average body temperature: ten seconds and a thermometer will quickly determine whether you have a fever or not.

The second law of thermodynamics can tell you even more about how these large ensembles of particles will behave: entropy has to increase, so heat flows from warm to cold bodies, work will turn into heat (for those fans of existential anxiety: thermodynamics tells us that all the work you will ever do will just bring the universe closer to its heat death) and so on and so on.

Describing an unimaginably large ensemble of particles by “emergent” properties can be conceptually advantageous, and these properties are not only conceptually advantageous, but inten a sense they can also be seen as well-defined, real properties of the system (leaving aside philosophical debates about the ontology of emergent properties).

There are many other large ensembles of small things whose microscopic dynamics are incredibly hard to model. One of pressing concern is you. Or rather, your brain. The human brain is an ensemble of roughly 80 billion neurons with 10¹⁵ synapses connecting these neurons. It’s incredibly complex, with building blocks that are already individually pretty hard to model (and only slowly are people realizing how much computation might already take place within the substructure of the neuron itself), and with large scale dynamics that are equally impossible to calculate for all intents and purposes as they are for large numbers of atoms.

But nevertheless the brain behaves in predictable ways. None of these neurons knows about you, but you probably woke up this morning and knew your own name, even had a sense of yourself. And none of these neurons knows how to read, but you manage to read these sentences with relative ease. So through the interaction of a large number of neurons, we can get predictable, planned behaviour. But how does a brain manage to plan, and, amidst all the chaos, actually decide on what to do? In other words: Where does something like will come in?

A possible definition of will is as the ability to predict multiple possible paths into the future and then, based on our internal motivational framework, pick out one path by performing appropriate actions. These willed actions can then be thought of as events caused by an explicit goal representation in the conscious mind of a rational agent, as Thomas Metzinger puts it.

Imagine a time hundreds of millions of years back when there were only fish that lived in murky waters and trees that stood around on the surface of the earth. As it is very hard to see underwater, fish can only see their proximal environment. They mainly react, as everything that’s going to happen to them which they need to worry about happens 3 seconds after presenting itself to them for the first time. A tree, on the other hand, is out in the open, but is fixed to the ground and can’t really do much even if he senses there is a woodcutter coming up to him.

But once you get animals on the land that can move around, things change. Because once you are able to see predators out in the distance, you have to think about what to do next, and what the predator will do next.

And thus, very broadly speaking, the future is invented, and slowly but gradually the brain regions that can do the appropriate processing about how to act in that future develop. There is emerging evidence that this is precisely what happened a long long time ago in our evolutionary history (Malcolm McIver at Northwestern Engineering does some interesting research on this).
And once you get a future in which to act, you have to make some decisions about how to act in the future.

The neuroscientist Karl Friston at UCL developed a theory called the Free Energy Principle. According to him, all biological, living systems have three characteristic features:

  1. An internal model of the world
  2. External sense data about the world
  3. The ability to perform actions in the world

The living system is separated from the world by something called a Markov blanket, sitting in between its internal model of the world and the external world. These internal models and actions can take on many forms, and there are great subtleties to the theory, but taking our brain as an example of a living system, these three properties are easy to understand: we have a model of what’s going to happen in the world around us, we constantly perceive the world through our sense data, and we can do things in this world based on actions we plan.

The central idea of the theory is that every living system will look like it’s trying to minimize the difference between its internal model and the external sense data called Bayesian model evidence, which can be associated with a free energy. We know this Bayesian model evidence from everyday experience: it allows us to judge how good our hypothesis is after new evidence came in. A couple of outliers can usually still be accomodated with in a given hypothesis, but once evidence becomes overwhelming, we adjust our model. The speed of with which new evidence accounted for depends on how confident we are in our hypothesis. If we are ultra-confident, it takes a long time. A great example of when this new evidence is discarded is for followers of conspiracy theories. Their internal model of the world is so strong that all evidence is discarded and the model is not updated.

As Friston states, this minimisation could take place over evolutionary time (during natural selection) or milliseconds (during perceptual synthesis). I will focus on the short time scales associated with cognition and action.

All living systems are countering the dissipative forces in their environment, and to counter that they are maximizing the Bayesian model evidence. Living systems are precisely that: they are non-dissipative because they can counter dissipation through their self-organizing principle, by having a model of what things should look like that is conserved over time.

This is part of the many-faceted principle of homeostasis, that keeps up the chemical and physical balance in living systems while they live in a ever-moving world.

The flow of information in all living systems. The Markov blanket is in between the environment and the agent, separating internal from external states. Through adjusting internal states and choosing the right actions, the free energy is minimized.

This free energy difference that is to be minimized can be called “surprise”, as we can intuitively understand that if there is a big difference between what you expect to happen and what is happening, you are indeed surprised. This idea can be formalised, and the “surprise” within the theory is linked to the thermodynamic free energy. Watch out that this surprise does not fully equate to being surprised in the sense. Exploratory behaviour can also be rewarded and justified, as it can potentially minimize surprise farther down the future: when Columbus discovered America, everyone was pretty surprised, but now we have a much better model of the world that avoids us wondering daily where the hell McDonalds came from.

Surprise also relates to entropy: the time-average over the surprise gives us a measure of the entropy. This is in turn can be understood by an information theoretical interpretation of entropy, which states that the entropy of a signal is largest if it is maximally uncertain, e.g. if we cannot make good predictions about its state/outcome. The entropy of a signal is therefore largest when every outcome has equal probability, e.g. when no outcome is more likely than any other outcome.

This draws further analogy to thermodynamics, where maximal entropy is reached if the the state of the system has reached equilibrium, and thus every microstate of the ensemble is equally likely.

Don’t worry if this is a bit much to take in. The bottom line is this: Shannon, the father of information theory, managed to equate the information theoretic quantity of uncertainty to the thermodynamic quantity of entropy, and based on this insight, we can apply thermodynamic models to processes that involve information processing and vice versa.

The time average over the free action relates to the time average over the surprise, which is equivalent to the entropy of the system.

The biological system can make two different adjustments after the inflow of information through sense data:

  1. Through adjusting the internal model of the world
  2. By performing actions

Through point 2, we get something like (motivated) actions into the mix.

Once the brain finds itself in a situation with possible future worlds and possible actions to perform, it necessarily has to decide on which actions to take and what to do. We can perceive this competition between competing paths and actions taking place in our head. The free energy principle states that our brain is evolved to minimize the implicit surprise function by comparing our model of the world with our sensory input, adjusting that model and by finding the right strategies for action that minimize surprise/maximize Bayesian model evidence. People are trying to find out how precisely the brain processes these calculations. It should be noted that there are many layers at which these actions can take place in the brain: releasing a hormone or a neurotransmitter is similarily an action in the world that changes the behaviour of the system.

I defined will as the ability to predict futures and possible actions and then, based on our internal motivational framework, pick out one path by performing appropriate actions. The point is that through this self-organizational principle of living things, by a wider perspective on homeostatis, determinism takes on a different guise. Out of the thermodynamics of non-dissipative, “living” systems emerges directed, motivated action following an optimization principle from the “pure deterministic matter” that makes up our brain.

Earlier I stated that thermodynamic properties can also be seen as well-defined, real properties of the system. In that sense, through the free energy principle, goal-directed action acquire a conceptual usefulness and physicality that tends to be denied by pure reductionist “everything is just atoms” approaches. That does perhaps not mean it is real in the same way that an atom “is real”. But as Anderson famously said: more is different. And what many things do is not something that can not always be causally understood by looking at every thing individually.

The internal states and actions can take on different levels of complexity. Human beings are, as in many cases, systems with their own special properties. Accordingly, people have criticized this approach as being overly reductionist. And of course there are many other layers to discussing free will that I can’t and won’t address here, namely the usefulness and sensibility of the concept in courts of justice, in moral judgements etc.

And I have to admit that there is a distinction to be made between agency and will. I believe the most conceptually useful way of thinking about free will is, in the language of Thomas Metzinger, as a property of our internal self model with respect to our agency. This model has its own evolutionary advantages. You can easily have agency without a self model, as when trees or fungi “act” to increase their chances of survival. But in us humans, within our advanced self model, the agency we already had, so to speak, is represented to us and appears as a will.

If you’re still not convinced that thermodynamics has anything to with your free will, I hope you can nevertheless agree that it’s fascinating to see how science can adress questions about how and why we act on such a deep, physical level, and in which way our directed, motivated actions connect to very general properties of self-organizing systems that slowly emerged evolutionarily and whose traces we can find by looking at life and our brain on many different scales (like evolution, genetics, cognition etc.) and from many different perspectives.

I love new ideas. My science podcast: Connect via LinkedIn:

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store