Markov is the eponym of a concept called a Markov blanket, which in machine learning is essentially a shield that separates one set of variables from others in a layered, hierarchical system. The psychologist Christopher Frith—who has an h-index on par with Friston’s—once described a Markov blanket as “a cognitive version of a cell membrane, shielding states inside the blanket from states outside.”
According to the most popular modern Bayesian account, the brain is an “inference engine” that seeks to minimize “prediction error.”
Free energy is the difference between the states you expect to be in and the states your sensors tell you that you are in. Or, to put it another way, when you are minimizing free energy, you are minimizing surprise.
Friston’s free energy principle says that all life, at every scale of organization—from single cells to the human brain, with its billions of neurons—is driven by the same universal imperative, which can be reduced to a mathematical function. To be alive, he says, is to act in ways that reduce the gulf between your expectations and your sensory inputs. Or, in Fristonian terms, it is to minimize free energy.
And indeed, Friston regards the Bayesian model as a foundation of the free energy principle (“free energy” is even a rough synonym for “prediction error”). But the limitation of the Bayesian model, for Friston, is that it only accounts for the interaction between beliefs and perceptions; it has nothing to say about the body or action. It can’t get you out of your chair.
Reinforcement learning doesn’t require humans to label lots of training data; it just requires telling a neural network to seek a certain reward, often victory in a game. The neural network learns by playing the game over and over, optimizing for whatever moves might get it to the final screen, the way a dog might learn to perform certain tasks for a treat.
But reinforcement learning, too, has pretty major limitations. In the real world, most situations are not organized around a single, narrowly defined goal.
Julie Pitt, head of machine-learning infrastructure at Netflix, discovered Friston and the free energy principle in 2014, and it transformed her thinking. (Pitt’s Twitter bio reads, “I infer my own actions by way of Active Inference.”)
But a free energy agent always generates its own intrinsic reward: the minimization of surprise. And that reward, Pitt says, includes an imperative to go out and explore.
The Fristonian agent started slowly, actively exploring options—epistemically foraging, Friston would say—before quickly attaining humanlike performance.
But for all the people who are exasperated by Friston’s impenetrability, there are nearly as many who feel he has unlocked something huge, an idea every bit as expansive as Darwin’s theory of natural selection. When the Canadian philosopher Maxwell Ramstead first read Friston’s work in 2014, he had already been trying to find ways to connect complex living systems that exist at different scales—from cells to brains to individuals to cultures. In 2016 he met Friston, who told him that the same math that applies to cellular differentiation—the process by which generic cells become more specialized—can also be applied to cultural dynamics. “This was a life-changing conversation for me,” Ramstead says. “I almost had a nosebleed.”
In 2017, Ramstead and Friston coauthored a paper, with Paul Badcock of the University of Melbourne, in which they described all life in terms of Markov blankets. Just as a cell is a Markov-blanketed system that minimizes free energy in order to exist, so are tribes and religions and species.
After the publication of Ramstead’s paper, Micah Allen, a cognitive neuroscientist then at the FIL, wrote that the free energy principle had evolved into a real-life version of Isaac Asimov’s psychohistory,11 a fictional system that reduced all of psychology, history, and physics down to a statistical science.
Full article from Wired