Discrete Information Geometry\\ of Computational Universe:\\ Relative Entropy, Fisher Structure\\ and Task-Sensitive Distance

Discrete Information Geometry of Computational Universe: Relative Entropy, Fisher Structure and Task-Sensitive Distance Haobo Ma1Wenlin Zhang2 1Independent Researcher 2National University of Singapore November 24, 2025 Abstract Under axiomatized framework of “computational universe” Ucomp = (X, T,C,I), complexity geometry characterizes “how much time/cost needed to reach certain configuration”. However, complexity geometry alone insufficient to describe “how high quality information these costs exchange for”. For this, this paper constructs set of “discrete information geometry” theory matching computational universe in completely discrete setting. We first introduce observation operator family O={Oj}j∈J, where each Oj maps configuration x∈Xto probability distribution p(j) xon some finite outcome set. Under fixed task or observation scheme, these distributions provide “observable information state” for each configuration x. Based on this, we define task-sensitive relative entropy structure DQ(x∥y), from which derive family of information distances, e.g., Jensen–Shannon type distance dJS,Q(x, y). These distances locally induce discrete Fisher structure, i.e., near some reference configuration x0, Hessian of second-order relative entropy DQ(x∥x0) gives discrete information metric tensor around x0. This paper proves, under natural regularity assumptions, discrete information structure can converge in appropriate limit to Riemannian-type information manifold (SQ, gQ), where gQFisher-type metric; correspondingly, “information geometry on configuration space” can be realized through map ΦQ:X→ SQ, projecting each configuration xto its observable information state. We further discuss volume growth of information balls Binfo R(x0) and “information dimension”, give general inequality between information dimension and complexity dimension, characterizing “information resolution limit achievable under given complexity budget”. Finally, we construct task-sensitive information–complexity joint action functional AQ, whose local Euler–Lagrange equation gives local description of optimal computation trajectory “maximizing information quality” under finite time budget, providing discrete information geometry foundation for subsequent complete “time–information–complexity variational principle”. Keywords: Discrete information geometry; Relative entropy; Fisher information metric; 1 Jensen–Shannon distance; Task-sensitive distance; Information dimension; Complexityinformation inequality 1 Introduction In computational universe axiomatic system, universe abstracted as discrete configuration space X, one-step update relation T, single-step cost Cand information quality function I, such that any actual computation process corresponds to finite path on configuration graph, complexity distance d(x, y) characterizes minimum cost needed to go from xto y. Previous work already constructed “discrete complexity geometry” based on this, describing problem difficulty and complexity horizon through complexity ball volume and discrete Ricci curvature. However, complexity geometry concerns “how far walked”, not “what seen”. To understand geometric structure of “information quality” in computational universe, need to introduce another dimension: observation and task. Specifically, “useful information” of same configuration xdepends not only on xitself, but also on how we read it out, what kind of task we care about. Different tasks correspond to different “information geometries”, and computation process trajectories on these information geometries are true objects reflecting “how much information we extracted within given time”. Goal of this paper is to establish set of task-related “discrete information geometry” for computational universe in completely discrete background:  At discrete level, assign each configuration xprobability state pxdetermined by observation scheme, construct information distances using relative entropy, Jensen– Shannon distance, etc.;  Locally, through second-order expansion of relative entropy obtain Fisher-type metric, establish discrete information manifold structure;  Globally, through information ball volume and information dimension characterize “under certain task, complexity of distinguishable states in universe”. More importantly, information geometry and complexity geometry must match: complexity geometry tells us which configurations allowed to move between under resource constraints, information geometry tells us how much information gain these movements bring in “task-relevant state space”. Coupling of both will ultimately lead to unified “time–information–complexity action functional”. Main thread structure of this paper as follows. Section 2 introduces observation operators and task-sensitive discrete relative entropy structure. Section 3 constructs discrete information distances and information balls, defines information dimension. Section 4 discusses local Fisher structure and information manifold limit. Section 5 gives information–complexity inequality and task-sensitive joint action functional prototype. Appendix provides detailed proofs of main propositions and theorems. 2 Observation Operators and Task-Sensitive Relative Entropy This section introduces observation operators and task-sensitive probability structure at configuration layer of computational universe. 2 2.1 Observation Operator Family and Observable States In computational universe Ucomp = (X, T,C,I), configuration x∈Xis internal state of entire universe. Observer within certain time window can only access it through finite experiments or readout processes. To characterize this point, introduce observation operator family. Definition 2.1 (Observation Operator Family).Let (Yj)j∈Jbe family of finite outcome sets. An observation operator family is map collection O={Oj:X→∆(Yj)}j∈J, where ∆(Yj) probability simplex on Yj, and for each x∈X,j∈J,Oj(x) = p(j) xis outcome distribution on result set Yjfrom one experiment. Intuitively, Ojdescribes observational process executable on configuration x, whose output distribution p(j) xis statistical information observer can “see” on this configuration. To avoid redundancy, we often denote task or observation scheme as finite subset Q⊂J, define “joint observable state” under this task. Definition 2.2 (Joint Observable State under Task Q).For given finite task set Q⊂J, define observable outcome set YQ=Y j∈Q Yj, define configuration x’s joint observable state as joint distribution p(Q) xon YQ. Simplest construction assumes observations independent, in which case p(Q) x(y) = Y j∈Q p(j) x(yj), y = (yj)j∈Q∈YQ. More generally, can allow known coupling structure between different observations, then p(Q) xgiven by task-specific observation model. This paper mainly considers independent case. 2.2 Task-Sensitive Relative Entropy After fixing task Q, each configuration xmapped to probability distribution p(Q) x∈∆(YQ). This allows us to introduce relative entropy for task Q. Definition 2.3 (Relative Entropy under Task Q).For configurations x, y ∈X, if for all y∈YQhave p(Q) y(y)>0 implies p(Q) x(y)>0, define DQ(x∥y) = X z∈YQ p(Q) x(z) log p(Q) x(z) p(Q) y(z), otherwise define DQ(x∥y) = +∞. DQ(x∥y) is “distinguishability degree” of configurations xand yunder task Q: larger means more “information distant” between xand yunder this task. Clearly, DQ(x∥y)≥0, and DQ(x∥y) = 0 if and only if p(Q) x=p(Q) y. Note DQgenerally not symmetric and doesn’t satisfy triangle inequality, thus not metric. To obtain information distance, we will use symmetrized form derived from DQ. 3 3 Discrete Information Distances and Information Balls This section defines family of information distances from task-sensitive relative entropy, constructs information ball structure and information dimension. 3.1 Jensen–Shannon Distance Most natural symmetrized form is Jensen–Shannon divergence. Definition 3.1 (Jensen–Shannon Distance under Task Q).Define JS divergence JSQ(x, y) = 1 2DQ(x∥mxy) + 1 2DQ(y∥mxy), where mxy =1 2(p(Q) x+p(Q) y) midpoint distribution. Then dJS,Q(x, y) = qJSQ(x, y) defines metric on configuration space (up to equivalence relation p(Q) x=p(Q) y). Proposition 3.2 (Metric Properties of JS Distance).dJS,Q satisfies: 1. Symmetry: dJS,Q(x, y) = dJS,Q(y, x); 2. Triangle inequality: dJS,Q(x, z)≤dJS,Q(x, y) + dJS,Q(y, z); 3. Positivity: dJS,Q(x, y)≥0, equals zero iff p(Q) x=p(Q) y. Proof. Symmetry obvious from definition. Triangle inequality follows from Endres-Schindelin (2003) proof. See Appendix A. 3.2 Information Balls and Information Volume Given information distance, can define information balls. Definition 3.3 (Information Ball).For configuration x0∈Xand radius R > 0, Binfo R(x0) = {x∈X:dJS,Q(x, x0)≤R} is information ball of radius Rcentered at x0under task Q. Information ball characterizes set of configurations “informationally close” to x0under task Q. Its cardinality |Binfo R(x0)|measures “how many distinguishable states exist within information distance Rfrom x0”. 4 3.3 Information Dimension Definition 3.4 (Information Dimension).If limit diminfo(x0) = lim R→0 log |Binfo R(x0)| log(1/R) exists, call it information dimension at x0under task Q. Information dimension measures “how densely information states pack” near x0. High dimension means many distinguishable states nearby, low dimension means sparse information structure. 4 Local Fisher Structure and Information Manifold Limit This section constructs local Fisher information metric from second-order expansion of relative entropy, discusses continuous limit of discrete information geometry. 4.1 Discrete Fisher Information Matrix Consider small perturbations near reference configuration x0. Assume configuration space has local parameter representation: near x0exist parameters θ= (θ1, . . . , θn) such that configurations uniquely correspond to θvalues. Definition 4.1 (Discrete Fisher Information Matrix).For parameterized configuration family x(θ) near x0=x(θ0), define discrete Fisher information matrix at θ0as g(Fisher) ab (θ0) = ∂2 ∂θa∂θbDQx(θ)∥x(θ0)θ=θ0 when this quantity well-defined. 4.2 Second-Order Expansion Proposition 4.2 (Quadratic Approximation of Relative Entropy).Under smoothness assumptions on p(Q) x(θ)in θ, DQx(θ)∥x(θ0)=1 2X a,b g(Fisher) ab (θ0)δθaδθb+O(|δθ|3) where δθ =θ−θ0. Proof. Taylor expansion to second order. First-order term vanishes by definition. See Appendix B. This shows Fisher matrix defines local Riemannian metric on parameter space, measuring information distance for small perturbations. 5 4.3 Continuous Limit and Information Manifold When configuration space has continuous limit (e.g., discretized field configurations converging to continuous fields), discrete information geometry converges to continuous information manifold. Theorem 4.3 (Convergence to Information Manifold).Under appropriate regularity conditions on observation operators and refinement sequence of discrete configurations, discrete Fisher metrics converge to continuous Fisher information metric gQon continuous configuration manifold SQ, forming Riemannian manifold (SQ, gQ). Proof. Uses standard techniques from information geometry. See Amari-Nagaoka (2000) and Appendix C. 5 Information–Complexity Inequality and Joint Action Functional This section establishes quantitative relationship between information dimension and complexity dimension, constructs joint action functional coupling information and complexity. 5.1 Information–Complexity Trade-off Theorem 5.1 (Information–Complexity Inequality).For any computation path γ: [0, T]→ Xof complexity cost C(γ), information gain along path bounded by ∆IQ(γ)≤fC(γ),dimcomp,diminfo where ffunction of complexity cost, complexity dimension dimcomp and information dimension diminfo. Proof. Combines complexity ball volume bounds from discrete complexity geometry with information ball bounds. See Appendix D. This theorem characterizes fundamental limit: given finite complexity budget, maximum achievable information resolution bounded by interplay of complexity and information dimensions. 5.2 Task-Sensitive Joint Action Functional To unify complexity cost and information gain, define joint action. Definition 5.2 (Information–Complexity Joint Action).For computation path γin time interval [0, T], define AQ[γ] = ZT 0αC(˙γ(t)) −βIQ(γ(t))dt where C(˙γ) instantaneous complexity cost rate, IQ(γ) instantaneous information quality under task Q,α, β > 0 trade-off weights. Minimizing AQyields trajectories balancing complexity cost and information gain. 6 5.3 Euler–Lagrange Equation Proposition 5.3 (Optimal Trajectory Condition).Critical points of AQsatisfy α∇˙γC(˙γ) = β∇IQ(γ) where ∇appropriate derivatives on configuration space. This gives local characterization of optimal computation trajectories maximizing information quality under complexity constraints. 6 Discussion and Outlook This paper constructed discrete information geometry framework for computational universe, complementing complexity geometry from previous work. Key achievements: 1. Defined task-sensitive relative entropy and information distances; 2. Established discrete Fisher structure and information manifold limit; 3. Introduced information dimension and proved information–complexity inequalities; 4. Constructed joint action functional coupling information and complexity. Future directions:  Extend to quantum information geometry for quantum computational universe;  Develop numerical methods for computing information metrics;  Apply to concrete problems in machine learning and optimization;  Complete unified time–information–complexity variational principle. This framework provides foundation for understanding not just “how computation happens” but “what information computation extracts”. A Proof of Triangle Inequality for JS Distance This appendix proves Proposition ??. A.1 Endres-Schindelin Proof The key result (Endres-Schindelin, 2003): √JS satisfies triangle inequality. For three distributions p, q, r, define midpoints mpq,mqr,mpr. Through careful convexity arguments and data processing inequality, show pJS(p, r)≤pJS(p, q) + pJS(q, r) Applied to our setting with p=p(Q) x, etc., gives triangle inequality. 7 B Second-Order Expansion of Relative Entropy This appendix proves Proposition ??. B.1 Taylor Expansion Write DQ(x(θ)∥x(θ0)) = X y p(Q) x(θ)(y) log p(Q) x(θ)(y) p(Q) x(θ0)(y) Expand both numerator and denominator to second order in δθ: p(Q) x(θ)(y) = p0(y) + X a ∂ap0(y)δθa+1 2X ab ∂a∂bp0(y)δθaδθb+O(|δθ|3) where p0=p(Q) x(θ0). After substitution and simplification using normalization conditions, first-order terms cancel, second-order terms give Fisher matrix. C Convergence to Continuous Information Manifold This appendix sketches proof of Theorem ??. C.1 Refinement Sequence Consider sequence of discrete configuration spaces Xnwith lattice spacing an→0. Assume observation operators Onconverge appropriately to continuous observation functionals. Discrete Fisher matrices g(n) ab form approximations to continuous Fisher metric gab. Under regularity (Sobolev estimates on probability densities), g(n) ab →gab in suitable topology. Details involve careful measure-theoretic arguments, see Amari-Nagaoka (2000) for standard proofs in classical information geometry setting. D Proof of Information–Complexity Inequality This appendix proves Theorem ??. D.1 Volume Comparison Key idea: complexity ball of radius Rcomp contains at most certain number of informationdistinguishable states, bounded by ratio of volumes in complexity vs. information geometries. Specifically, if complexity ball Bcomp Rcomp (x0) has volume Vcomp ∼Rdimcomp comp , and typical information separation scale is ϵinfo, then number of distinguishable states 8 Ndist ≲Vcomp ϵdiminfo info Information gain along path of complexity cost Cbounded by log Ndist with appropriate Rcomp ∼C. Detailed calculation shows inequality of Theorem ??. 9