Entropy and Emergent Structures - juhameskanen/abstract GitHub Wiki

Emergent Time in Static Timeless Universe

Time is the ordinal index of random walk through the most compressible configurations. The minimal and hence most compressible path implies the minimal change between consecutive configurations, which can be described as fair random bit-flip mutation of system with $n$ bits.

Ehrenfest process

The process that describes this most compressible path is the Ehrenfest process.

Setup

At each discrete tick, pick one of $n$ bits uniformly and flip it. Let $m$ = number of 1-bits, $p=m/n$. In one tick, $E[\Delta m] = \frac{n-m}{n} - \frac{m}{n} = 1-2p$ (probability of hitting a 0-bit minus probability of hitting a 1-bit). In the continuum limit this gives:

$$ \frac{dm}{dt} = 1 - \frac{2m}{n} \quad\Longrightarrow\quad \frac{dp}{dt} = \frac{1-2p}{n} $$

where $t$ here is tick count ("bit-flip time"). Solving with $p(0)=0$:

$$ p(t) = \frac{1}{2}\left(1-e^{-2t/n}\right) $$

This is where the $n$-dependence lives. In flip-count time $t$, the relaxation timescale is $O(n)$ ticks — system relaxes after about $n/2$ flips.

Time (bit-flip space) can be rescaled by $n$ to get scale independent universal curve shape that does not depend $n$.

$$ p(\tau) = \frac{1}{2}\left(1-e^{-2\tau}\right), \qquad \tau \in [0,\infty) $$

This has no $n$ in it, meaning the curve is universal. And the entropy (Shannon, per-bit fraction of max):

$$ \frac{S(\tau)}{n} = H_2\big(p(\tau)\big), \qquad H_2(p) = -p\log_2 p - (1-p)\log_2(1-p) $$

So the closed form:

$$ p(\tau) = \frac{1}{2}\left(1-e^{-2\tau}\right), \qquad \tau = \frac{\text{flip count}}{n} $$

Turning this into a properly bounded $[0,1]$ time variable allows us to map it to any time scale. $\tau$ itself is unbounded ($\tau\in[0,\infty)$, asymptotic approach to equilibrium, never exactly reaching it at finite $\tau$). But there's a natural change of variable that is exactly bounded on $[0,1)$ without needing an arbitrary cutoff:

$$ x(\tau) = 1-e^{-2\tau} \in [0,1), \qquad \tau = -\tfrac{1}{2}\ln(1-x) $$

Since $p=x/2$, the entropy becomes a clean function of $x$ directly:

$$ \boxed{\frac{S(x)}{n} = H_2!\left(\frac{x}{2}\right)}, \qquad H_2(q) = -q\log_2 q-(1-q)\log_2(1-q) $$

That's the universal curve, and it's what we plug physical time into once we know how $t_{phys}$ relates to $x$ (or to $\tau$).

Emergence of patterns

Setup

For an $n$-bit string relaxing under the symmetric single-bit Ehrenfest process (start all-zero, flip a uniformly random bit each tick), each bit is independently $1$ with probability:

$$ p(\tau) = \frac{1}{2} \left(1 - e^{-2\tau}\right), \quad \tau = \frac{\mathrm{flip_count}}{n} $$

For a fixed bit-pattern of length $L$ with $a$ ones and $b$ zeros ($a + b = L$), the probability of that exact pattern occurring at a random position is, in the independent-bit continuum limit:

$$ P(\text{pattern} \mid \tau) = p(\tau)^a (1 - p(\tau))^b $$

Result 1 — Zero/one-start complementation symmetry (exact)

Starting from all-ones gives $q(\tau) = 1 - p(\tau)$. Because the product formula is symmetric under $p \to 1-p$ combined with bit-complementing the pattern:

$$ P_{\text{zero-start}}(X \mid \tau) = P_{\text{one-start}}(\text{complement}(X) \mid \tau) \qquad \text{exactly, all } \tau $$

e.g. "001" from an all-zero start is identical to "110" from an all-ones start.

Result 2 — Arrangement doesn't matter, only composition does

Since the formula depends only on counts ($a$, $b$), not order, every permutation of a given composition produces an identical curve at every $\tau$, not just asymptotically. E.g. "010", "001", "100" ($a=1, b=2$) are one curve; "110", "101", "011" ($a=2, b=1$) are a different curve — but all six converge to the same equilibrium value $(0.5)^L$, since composition-dependence cancels exactly at $p=0.5$.

Result 3 — There is no single universal shape; there are exactly three

On the reachable domain $p \in [0, 0.5]$, classified purely by the sign of $a - b$:

Composition	Shape	Closed form behavior
$a > b$	monotonic rise, no peak	peak would sit at $p^* = \frac{a}{a+b} > 0.5$, unreachable
$a = b$	monotonic rise, flattens at boundary	peak exactly at $p^* = 0.5$, the edge
$a < b$	rises, peaks, then declines to plateau	genuine interior hump ("lognormal-like")

Only the $a < b$ case produces the hump shape. This corrects an earlier working assumption that the hump was universal — it was only ever tested on filters that happened to have more required-zero positions than required-one positions.

Result 4 — Peak location, amplitude, and decay rate (for the $a < b$ case)

$$ p^* = \frac{a}{a+b} $$

$$ \tau^* = -\frac{1}{2} \ln(1 - 2p^*) $$

$$ \text{peak/plateau ratio} = 2^{L (1 - H_2(p^*))} \qquad [\text{grows exponentially with } L] $$

where $H_2(q) = -q\log_2(q) - (1-q)\log_2(1-q)$.

Tail decay rate (large $\tau$): pattern deviation from its plateau decays as $e^{-2\tau}$ for asymmetric patterns ($a \neq b$) — half the exponent of entropy's own approach to its max ($e^{-4\tau}$), because entropy sits at a flat maximum at $p=1/2$ (quadratic sensitivity) while a generic pattern's count function does not (linear sensitivity). Symmetric patterns ($a=b$) decay at the same rate as entropy, $e^{-4\tau}$.

Conclusion

The model does not produce an arbitrary or unpredictable range of curves. It produces exactly one of three classes, fully determined in advance by a filter's own bit-composition ($a$ vs $b$) — no simulation required to know which shape a given filter will produce. Any apparent "universal lognormal" result should be checked against this classification before being treated as a general property of the model.

Verification

See ehrenfest_entropy.py — pattern_probability() implements the closed form above; simulate_ehrenfest() confirms it against direct stochastic simulation.

Result 5 — The hump class dominates almost the entire relaxation

Result 3 classifies a single composition's shape by the sign of $a - b$. A separate question is which class is typical — i.e. if you pick a random window at time $\tau$, which class does its composition actually tend to land in? Answering this requires weighting each composition by how many arrangements share it ($\binom{L}{a}$, per Result 2), not just looking at one representative pattern.

$$ M_{\text{hump}}(\tau) = \sum_{a < b} \binom{L}{a} p(\tau)^a (1-p(\tau))^b $$

$$ M_{\text{nonhump}}(\tau) = \sum_{a \ge b} \binom{L}{a} p(\tau)^a (1-p(\tau))^b $$

Weighed globally: the window's 1-count $a$ is distributed as $\text{Binomial}(L, p(\tau))$, and since $p(\tau) < 0.5$ for every finite $\tau$, that distribution's mean/mode sits strictly below $L/2$ — i.e. inside the hump region ($a < b$) — for the entire relaxation, only approaching the boundary as $\tau \to \infty$.

Numerically, $M_{\text{hump}}$ holds effectively all the mass from $\tau$ just above $0$, and only cedes majority status to $M_{\text{nonhump}}$ at a crossover $\tau_c$ that increases with $L$ and whose corresponding $p(\tau_c)$ converges to $0.5$ as $L$ grows:

$L$	$\tau_c$ (crossover)	$p(\tau_c)$
12	1.257	0.4595
40	1.849	0.4876
200	2.650	0.4975
1000	3.454	0.4995

So the hump class isn't merely one of three possible shapes — for any window length $L$, it is the typical outcome of the relaxation process, dominant across almost the full range of $\tau$, with the non-hump classes only catching up once the system has nearly reached equilibrium ($p \to 0.5$). The crossover point recedes toward $\tau = \infty$ as $L$ grows, so for large $L$ the hump class can be considered dominant for all practically observable $\tau$.

Verification

Confirmed by direct summation of the binomial-weighted composition masses ($\sum_a \binom{L}{a} p^a (1-p)^{L-a}$, split by $a \gtrless L/2$) across $L \in {12, 20, 40, 100, 200, 400, 1000}$; crossover $\tau$ found via root-finding on $M_{\text{hump}}(\tau) - M_{\text{nonhump}}(\tau)$.

Conclusion

Bump-shaped curve is the most typical.

↑ Up | ← Previous | Next →