4 ms·
tl;dr: This is a (cute!) technical detail of an approximation technique used in studying the thermodynamics of computation. It has nothing to do with fundamenta
by madhadron 8y ago
tl;dr: This is a (cute!) technical detail of an approximation technique used in studying the thermodynamics of computation. It has nothing to do with fundamental physics.
Imagine you have a system with N states, represented as a vector of length N. You have a linear operator that transforms that vector into the state in the next step of time. That's basic linear algebra. Now make the linear operator random, so you have stochastic transitions. That makes it into something called a Markov chain.
Now, physicists love differential equations. They've got lot of tools for working with them, and to bring those tool to bear, some folks a while back took a Markov chain and wrote down a set of linear differential differential equations for how the probability distribution over states evolves in time. That's called a "master equation."
Now say that you have an arbitrary function from initial state to final state. Can you describe it as a dynamics governed by a master equation? Not in general. That's not a huge surprise. We know there are lots of things you can't describe with linear differential equations.
If you're working on thermodynamics of computation, though, it would be nice to salvage the master equation framework because its relationship with thermodynamics is well understood, and you don't have to rebuild that. And getting to thermodynamics from straight up stochastic processes is beyond the mathematical abilities of most physicists. That's not a slight on physicists. The mathematical path from a stochastic process to a thermodynamics is an area of deep, difficult specialization in mathematical physics and if you go down that rabbit hole, it will likely be your career (see, for example, Elliott Lieb).
There are two ways to extend this linear world to get better approximations of things:
1. Imagine you have two points on a curve. A linear approximation is drawing a straight line between the two points. To get a better approximation, you take some more points between the two on the curve and draw a sequence of linear going through those points. The equations of each of those lines are going to be different (they have different slopes and intercepts or however you want to parameterize them). Or, in the context of master equations, you insert some additional, "hidden" steps in between your primary time steps with different stochastic matrices.
2. Imagine you have a dynamical system that moves in time steps, and depends on the last two steps for its current move. You can make it into a system that depends only on the last step by expanding the state to include the previous time step as well (that is, instead of the dynamics of x(t), I track the dynamics of (x(t), y(t)) where y(t) = x(t-1)). These are called "hidden variables." It's the same idea as hidden variables in quantum mechanics. Or even in classical mechanics, I can't write down a first order equation for the position of a classical particle...but I can write down a pair of first order equations for the position and momentum.
Lots of folks working on thermodynamics of computation have done both these things to patch their tool.
What this paper does is try to calculate how many hidden steps and how many hidden variables you need to patch the tool for a given arbitrary function that you're trying to approximate, and shows that if you use more hidden variables you need fewer hidden steps and vice versa.
For thermodynamics of computation, they point out that if you are engineering a system with master equation dynamics, you have to pay for the extra hidden states/steps that you need, so the simple statement that invertible functions are thermodynamically free and noninvertible ones require work isn't the only cost accounting to do.
Aside: I have complained before about only whiz bang articles rising on HN. Here we have a straightforward, technical calculation on the front page. Progress!