Discrete Information Geometry of Computational Universes:\\ Relative Entropy, Fisher Structure, and Task-Aware Distances

Discrete Information Geometry of Computational Universes: Relative Entropy, Fisher Structure, and Task-Aware Distances Haobo Ma1Wenlin Zhang2 1Independent Researcher 2National University of Singapore November 24, 2025 Abstract Within the axiomatic framework of “computational universe” Ucomp = (X, T,C,I), complexity geometry characterizes “how much time/cost is needed to reach a configuration.” However, complexity geometry alone is insufficient to describe “what quality of information is gained for these costs.” To address this, we construct a “discrete information geometry” theory compatible with computational universes within a fully discrete setting. We first introduce observation operator families O={Oj}j∈J, where each Oj maps configuration x∈Xto a probability distribution p(j) xover some finite outcome set. Under fixed tasks or observation schemes, these distributions provide “visible information states” for each configuration x. We define task-aware relative entropy structures DQ(x∥y) and derive information distances such as Jensen–Shannon distance dJS,Q(x, y). These distances locally induce discrete Fisher structures: near a reference configuration x0, the Hessian of second-order relative entropy DQ(x∥x0) yields a discrete information metric tensor around x0. We prove that under natural regularity assumptions, discrete information structures converge in appropriate limits to a Riemannian information manifold (SQ, gQ) with Fisher-type metric gQ. Correspondingly, “information geometry on configuration space” is realized through mapping ΦQ:X→ SQsending each configuration xto its visible information state. We further discuss volume growth of information balls Binfo R(x0) and “information dimension,” providing general inequalities between information dimension and complexity dimension, characterizing “limits of information resolution achievable under given complexity budgets.” Finally, we construct a task-aware information–complexity joint action AQwhose local Euler–Lagrange equations provide local descriptions of optimal computational trajectories “maximizing information quality” under finite time budgets, establishing discrete information geometry foundations for subsequent complete “time–information–complexity variational principles.” Keywords: Computational Universe, Information Geometry, Relative Entropy, Fisher Information, Task-Aware Distance, Information Dimension MSC 2020: 94A17, 62B10, 68Q15, 53B12 1 1 Introduction In the axiomatic system of computational universes, the universe is abstracted as discrete configuration space X, one-step update relation T, single-step cost C, and information quality function I, whereby any actual computational process corresponds to a finite path on the configuration graph, with complexity distance d(x, y) characterizing minimal cost required from xto y. Previous work has constructed “discrete complexity geometry” based on this foundation, describing problem difficulty and complexity horizons through complexity ball volumes and discrete Ricci curvature. However, complexity geometry concerns “how far traveled” rather than “what was observed.” Understanding geometric structure of “information quality” in computational universes requires introducing another dimension: observation and tasks. Specifically, “useful information” of the same configuration xdepends not only on xitself but also on how we read it out and what tasks we care about. Different tasks correspond to different “information geometries,” and computational process trajectories on these information geometries truly reflect “how much information we extracted in given time.” This paper’s goal is to establish task-related “discrete information geometry” for computational universes in completely discrete settings:  At discrete level, assign each configuration xa probability state pxdetermined by observation scheme, constructing information distances using relative entropy, Jensen–Shannon distance, etc.;  Locally, obtain Fisher-type metrics through second-order expansion of relative entropy, establishing discrete information manifold structures;  Globally, characterize “complexity of distinguishable states in the universe under certain tasks” through information ball volumes and information dimension. More importantly, information geometry must coordinate with complexity geometry: complexity geometry tells us which configurations we can move between under resource constraints; information geometry tells us how much information gain these movements bring in “task-relevant state spaces.” Coupling of the two ultimately leads to unified “time–information–complexity action.” Main structure: Section 2 introduces observation operators and task-aware discrete relative entropy structures. Section 3 constructs discrete information distances and information balls, defining information dimension. Section 4 discusses local Fisher structure and information manifold limits. Section 5 provides information–complexity inequalities and a task-aware joint action prototype. Appendices provide detailed proofs of main propositions and theorems. 2 Observation Operators and Task-Aware Relative Entropy This section introduces observation operators and task-aware probability structures at configuration level of computational universes. 2 2.1 Observation Operator Families and Visible States In computational universe Ucomp = (X, T,C,I), configuration x∈Xis the universe’s internal state. Observers can only access it through finite experimental or readout processes within time windows. We introduce observation operator families to characterize this. Definition 2.1 (Observation Operator Family).Let (Yj)j∈Jbe a family of finite outcome sets. An observation operator family is a set of mappings O={Oj:X→∆(Yj)}j∈J, where ∆(Yj) is the probability simplex on Yj, and for each x∈X,j∈J,Oj(x) = p(j) xis the outcome distribution on result set Yjfrom one experiment. Intuitively, Ojdescribes an observation process implementable on configuration x, with output distribution p(j) xbeing statistical information the observer “sees” on that configuration. To avoid redundancy, we often denote tasks or observation schemes as finite subsets Q⊂J, defining “joint visible states” under that task. Definition 2.2 (Joint Visible State Under Task Q).For given finite task set Q⊂J, define visible outcome set YQ=Y j∈Q Yj, and define configuration x’s joint visible state as a joint distribution p(Q) xon YQ. The simplest construction assumes independent observations: p(Q) x(y) = Y j∈Q p(j) x(yj), y = (yj)j∈Q∈YQ. More generally, known coupling structures between different observations can be allowed, where p(Q) xis given by a task-specific observation model. This paper mainly considers independent cases. 2.2 Task-Aware Relative Entropy After fixing task Q, each configuration xis mapped to probability distribution p(Q) x∈ ∆(YQ). This allows us to introduce relative entropy for task Q. Definition 2.3 (Relative Entropy Under Task Q).For configurations x, y ∈X, if for all z∈YQ,p(Q) y(z)>0 implies p(Q) x(z)>0, define DQ(x∥y) = X z∈YQ p(Q) x(z) log p(Q) x(z) p(Q) y(z), otherwise define DQ(x∥y) = +∞. DQ(x∥y) is the “distinguishability degree” of configurations xand yunder task Q: larger values mean xand yare more “informationally distant” under that task. Clearly, DQ(x∥y)≥0, and DQ(x∥y) = 0 if and only if p(Q) x=p(Q) y. Note DQis generally not symmetric and does not satisfy triangle inequality, thus is not a metric. To obtain information distance, we use symmetrized forms derived from DQ. 3 3 Discrete Information Distance, Information Balls, and Information Dimension This section defines task-aware information distances and information balls based on DQ, introducing the concept of information dimension. 3.1 Jensen–Shannon Information Distance For probability structures on finite sets, Jensen–Shannon divergence provides natural symmetrization, with its square root being a metric. We adopt similar constructions in task information contexts. Definition 3.1 (Jensen–Shannon Divergence and Information Distance Under Task Q). For x, y ∈X, define mixture distribution m(Q) x,y =1 2p(Q) x+p(Q) y, Jensen–Shannon divergence JSQ(x, y) = 1 2Dp(Q) x∥m(Q) x,y +1 2Dp(Q) y∥m(Q) x,y , and information distance dJS,Q(x, y) = q2 JSQ(x, y). where D(·∥·) is standard Kullback–Leibler relative entropy. By known results, dJS,Q is a metric on Xrelative to task Q: satisfies non-negativity, symmetry, triangle inequality, and dJS,Q(x, y) = 0 if and only if p(Q) x=p(Q) y. We call dJS,Q the task-aware information distance. 3.2 Information Balls and Information Volume Definition 3.2 (Information Ball and Information Volume).For reference configuration x0∈X, task Q, and radius R > 0, define information ball Binfo,Q R(x0) = {x∈X:dJS,Q(x, x0)≤R}, information volume Vinfo,Q x0(R) = Binfo,Q R(x0). This volume characterizes the number of configurations “information distance at most Rfrom x0” from information geometry perspective of task Q. 3.3 Information Dimension Similar to complexity dimension, we define information dimension using growth rate of information ball volumes. 4 Definition 3.3 (Information Dimension).For given task Qand reference x0, define upper information dimension diminfo,Q(x0) = lim sup R→∞ log Vinfo,Q x0(R) log R, lower information dimension diminfo,Q(x0) = lim inf R→∞ log Vinfo,Q x0(R) log R. If the two are equal, their common value is called information dimension, denoted diminfo,Q(x0). Intuitively, diminfo,Q(x0) describes growth order of distinguishable configuration numbers within information distance radius Runder task Q. If information dimension is finite, task Qactually only involves some low-dimensional information structure; if infinite, the task has high complexity and high distinguishability at information level. 3.4 Preliminary Relationship Between Information and Complexity Dimensions Let complexity distance be dcomp(x, y), complexity ball volume Vcomp x0(T), complexity dimension dimcomp(x0). Generally, information dimension and complexity dimension have no simple equality, but we can provide rough inequality characterizing “upper bound of information distinction ability under complexity constraints.” Proposition 3.4 (Information Volume Constrained by Complexity Volume).Assume there exists constant LQ>0such that for all adjacent configurations x, y (i.e., (x, y)∈T), dJS,Q(x, y)≤LQC(x, y), then there exists constant C > 0such that for all R > 0, Vinfo,Q x0(R)≤Vcomp x0R C. Thus diminfo,Q(x0)≤dimcomp(x0). Proof in Appendix A.1. This inequality shows that under local Lipschitz conditions, information geometry “dimension” does not exceed complexity geometry “dimension,” conforming to intuition: computable distinguishing ability is limited by achievable complexity resources. 4 Local Fisher Structure and Information Manifold Limits This section introduces local Fisher structure under task Q’s information distance, constructing Riemannian information metrics near reference configurations through secondorder expansion of relative entropy. We then discuss how information geometry converges to Fisher manifolds in limits when continuous parameterizations exist between configurations. 5 4.1 Second-Order Expansion of Relative Entropy and Discrete Fisher Matrix Let x0∈Xbe reference configuration with visible state p0=p(Q) x0under task Q. Consider several adjacent configurations x(1), . . . , x(k)with visible states pi=p(Q) x(i). Assume there exists local parameterization θ∈Θ⊂Rk7−→ p(θ)∈∆(YQ), such that p(0) = p0, and each pican be written as p(εei), where eiis standard basis and ε > 0 is small parameter. We can use θas “information coordinates” near x0. Definition 4.1 (Local Task Fisher Matrix).Under above setting, define local Fisher information matrix for task Qas g(Q) ij (0) = X z∈YQ p0(z)∂θilog p(θ)(z)θ=0 ∂θjlog p(θ)(z)θ=0. This is the Fisher information matrix at θ= 0, completely determined by local variation of p(θ). Theorem 4.2 (Fisher Form of Relative Entropy Second-Order Expansion).Under above setting and standard regularity conditions, for sufficiently small θ∈Θ, DQθ∥0=Dp(θ)∥p(0)=1 2X i,j g(Q) ij (0) θiθj+o(|θ|2). Proof in Appendix B.1. This theorem shows task-aware relative entropy locally has standard Fisher second-order structure: its Hessian yields a local Riemannian information metric. 4.2 Information Manifolds and Configuration-to-Information Mapping The above discussion is based only on finitely many adjacent configurations and local parameterization. However, in many cases, the set of visible states {p(Q) x:x∈X}of configuration space Xunder task Qcan be approximated by some continuous parameter manifold SQ. Assumption 4.3 (Manifold Structure of Task Visible States).There exists finite-dimensional manifold SQwith embedding map ΨQ:SQ,→∆(YQ), and mapping ΦQ:X→ SQ, such that: 1. For each x∈X,p(Q) xapproximates ΨQ(ΦQ(x)); 2. Standard Fisher information metric on SQvia ΨQis consistent with second derivative of relative entropy. 6 Under this assumption, we can view SQas “information manifold of task Q,” while ΦQprovides mapping from configuration space Xto information manifold. Definition 4.4 (Task Information Manifold and Information Metric).Under Assumption 4.3, the information manifold of task Qis (SQ, gQ), where gQis Fisher information metric. For configuration x∈X, its information geometry position is ΦQ(x)∈ SQ. 4.3 Consistency of Information Distance and Fisher Distance Under suitable regularity conditions, Jensen–Shannon information distance dJS,Q near x0 is consistent with Fisher distance. Theorem 4.5 (Local Information Distance Consistency).Let x, x0∈Xsuch that ΦQ(x0) = θ0,ΦQ(x) = θ, with θclose to θ0. Then dJS,Q(x, x0) = q(θ−θ0)⊤gQ(θ0)(θ−θ0) + o(|θ−θ0|). Proof in Appendix B.2. This theorem shows that in local coordinates, Jensen–Shannon information distance is first-order equivalent to geodesic distance induced by Fisher metric, thus Riemannian information geometry of SQis locally compatible with discrete information geometry on X. 5 Information–Complexity Inequality and Task-Aware Action This section provides information–complexity inequalities at the intersection of discrete information geometry and complexity geometry, constructing a task-aware joint action prototype. 5.1 Information–Complexity Inequality Proposition 3.4 already provided general constraint between information ball volume and complexity ball volume. We can strengthen this to a local “information gradient–complexity gradient” relation in the information manifold framework. Let γ= (x0, x1, . . . , xn) be a complexity shortest path with complexity length C(γ) = n−1 X k=0 C(xk, xk+1), with corresponding information path ΦQ(γ) = (θ0, θ1, . . . , θn) and information distance LQ(γ) = n−1 X k=0 dSQθk, θk+1, where dSQis geodesic distance induced by Fisher metric. Under local Lipschitz assumptions, we have the following proposition. 7 Proposition 5.1 (Local Information–Complexity Lipschitz Inequality).If there exists constant Lloc Q>0such that for all adjacent configurations x, y (i.e., (x, y)∈Twith x, y in some local region), dSQΦQ(x),ΦQ(y)≤Lloc QC(x, y), then for any local path γ, LQ(γ)≤Lloc QC(γ). In particular, minimal information distance and minimal complexity distance satisfy dSQΦQ(x0),ΦQ(x)≤Lloc Qdcomp(x0, x). Proof in Appendix C.1. This inequality shows that in local regions, “information displacement” is controlled by “complexity displacement,” with complexity providing resource upper bound for information geometry. 5.2 Task-Aware Joint Action To unify complexity geometry and information geometry, we construct a task-aware discrete action for evaluating “cost-effectiveness” of computational paths under given task Q. Definition 5.2 (Joint Action Prototype for Task Q).Let γ= (x0, x1, . . . , xn) be a path with complexity length C(γ) and terminal information quality IQ(xn) (quality function defined by task). Define joint action for task Q: AQ(γ) = αC(γ)−βIQ(xn), where α, β > 0 balance complexity and information. In continuous limits, introducing time parameter ton information manifold (SQ, gQ) and complexity manifold (M, G), letting configuration path x(t) and information path θ(t) = ΦQ(x(t)), with complexity velocity qGab(θ)˙ θa˙ θband information quality IQ(θ(T)), the continuous form of joint action is AQ[θ(·)] = ZT 0 αqGab(θ(t)) ˙ θa(t)˙ θb(t) dt−β IQ(θ(T)). This action balances “complexity length of path” and “information gain at terminal,” with minimizing trajectories corresponding to optimal information acquisition strategies under resource constraints. Specific forms of Euler–Lagrange equations for discrete and continuous versions are left for future work; this paper only provides structural prototype. 6 Conclusion Under discrete axiomatic framework of computational universes, this paper introduces observation operator families and task-aware relative entropy structures, constructs discrete information distances, information balls, and information dimension, and locally obtains Fisher information matrices through second-order expansion of relative entropy, establishing the concept of task information manifold (SQ, gQ). Through mapping ΦQ:X→ SQ, 8 configuration space of computational universe is embedded under task Qinto a finitedimensional information manifold, with information distance locally compatible with discrete Jensen–Shannon distance, and information dimension satisfying natural inequality with complexity dimension. Based on these structures, we propose information–complexity Lipschitz inequality and prototype task-aware joint action, providing discrete information geometry foundations for constructing complete “time–information–complexity variational principles” under unified time scales. Next steps will combine this paper’s information geometry with previous complexity geometry, systematically constructing computational worldlines on joint manifold (M, G;SQ, gQ) and interfacing with boundary time geometry and unified scattering time scales of physical universes. A Proof of Information–Complexity Dimension Inequality A.1 Proof of Proposition 3.4 Proposition Restatement Assume there exists constant LQ>0 such that for all adjacent configurations x, y, dJS,Q(x, y)≤LQC(x, y). Then there exists constant C > 0 such that for all R > 0, Vinfo,Q x0(R)≤Vcomp x0R C, thus diminfo,Q(x0)≤dimcomp(x0). Proof For any x∈Binfo,Q R(x0), by definition dJS,Q(x, x0)≤R. Take any complexity shortest path γ= (x0, x1, . . . , xn) from x0to xwith cost C(γ) = dcomp(x0, x). By triangle inequality and local Lipschitz condition, dJS,Q(x, x0)≤ n−1 X k=0 dJS,Q(xk, xk+1)≤LQ n−1 X k=0 C(xk, xk+1) = LQC(γ). If dJS,Q(x, x0)≤R, then dcomp(x0, x)≤R LQ. Thus Binfo,Q R(x0)⊆Bcomp R/LQ(x0). Taking C=LQgives required inclusion, thus Vinfo,Q x0(R)≤Vcomp x0R C. Taking upper limit as R→ ∞ yields dimension inequality. 9