Relating Almost Perfect Condensation Conditions and the Conditioned Score

This work was done at the Iliad fellowship under the mentorship of @Daniel C. Thanks to Daniel C and Sam Eisenstat for comments on the draft of this post.

In Eisenstat (2025) and in Demski (2025), condensation is expressed in terms of some functions, the simple score and the conditioned score, which relate it to compression. Almost perfect condensation and related ideas are helpful for understanding why the objectivity theorem gives good bounds in certain situations, but the hypotheses of almost perfect condensation don’t have a clear relation to compression. It would be interesting if these two perspectives could be unified.

In Eisenstat (2025) and in Demski (2025), condensation is expressed in terms of some functions, the simple score and the conditioned score, which relate it to compression. Almost perfect condensation and related ideas are helpful for understanding why the objectivity theorem gives good bounds in certain situations, but the hypotheses of almost perfect condensation don’t have a clear relation to compression. It would be interesting if these two perspectives could be unified.

I have attempted to unify the two perspectives here.

A low conditioned score is stronger than the almost perfect condensation conditions.

A low conditioned score gap gives bounds for the reconstruction and Markov conditions.

A small Markov condition bound together with a strengthened reconstruction condition gives a low conditioned score gap.

Let be finite-alphabet random variables and let be a condensation, where indexes the latent variables. All slots outside are taken to be constant.

I will often refer to as the “active latents”.

Its lower bound is , so I will call the conditioned-score gap.

Call a family upward closed in if , , imply . For any write .

the maximum running over all pairs that are each upward closed in .

An -almost perfect condensation has (size condition) ; (reconstruction condition) for all with ; and (Markov condition) .[2]

2. Breaking Up the Conditioned Score

The conditioned-score gap can be decomposed into two nonnegative information-theoretic charges. This comes from the exact characterization of perfect condensation in Sam's Theorem 5.10.[3]

Let be the joint distribution of the active latents . Define a reference distribution that retains only the dependence of each on the latents indexed by strict supersets .

By Gibbs' inequality this is nonnegative. It measures how much dependence is not explained by the reverse-inclusion hierarchy.

The remaining difference from is the reconstruction defect, which measures the randomness remaining in the active latents once the observables in are known.

By our assumption that the latents reconstruct the observables (), the active latents completely determine . Therefore, , which gives:

These defects are related to but distinct from the corresponding conditions of almost perfect condensation.

3. Low scores imply the reconstruction and Markov conditions

Proof.[6] If , every slot containing meets . So is a subtuple of , and the score gap anatomy gives

Because and are upward closed, the block order internally ordered by reverse inclusion, is a valid reverse-inclusion order. Conditioning each score term on the preceding blocks and applying the chain rule:

Therefore the Markov defect is at most , which is at most by the score gap anatomy.

If the relevant conditioned score gaps are at most , then the reconstruction condition holds with error wherever , and the Markov condition holds with error . Therefore low conditioned-score gaps imply the almost-perfect condensation conditions (other than the size condition).

4. The reconstruction and Markov conditions imply low conditioned scores

Thus is the set of active indices that are minimal under inclusion. Every contains some . Therefore, every active latent occurs in at least one of the joint variables with .

Proof.[7] Let , and choose an ordering of such that strict supersets occur before their subsets: if , then .

is its reverse-inclusion factorization. Expanding the KL divergence gives

The summand for is zero, as has no strict superset in , since any such superset would have to occur before . Hence

Thus the KL divergence is a sum of potentially nonzero terms.

The families and are both upward closed by definition.

Therefore, and .

The second equality holds because the variables in other than are being conditioned on.

It remains to bound the other term in the conditional score gap decomposition. Recall: every contains some , and therefore is a component of for at least one . The joint variables consequently contain all the active latents, and subadditivity gives

This proves the stated bound.

Suppose a condensation is -almost perfect. The Markov condition gives

for every satisfying . Therefore, ifthen

The value of determines which sets receive this bound. When , it always holds because every intersects . When , the bound applies to those that intersect every minimal active index in more than elements.

We can modify the score to make it more equivalent to the almost perfect condensation conditions. I will provide only a sketch here; you can follow the exact arguments above to see that it works. Recall the score gap anatomy

then we can decompose the reconstruction defect into parts.

So if we want a score which is bounded by the almost perfect condensations without the added assumption that for every , we can simply subtract away that second piece of the reconstruction defect.

^As defined in Definition 4.3 of Sam Eisenstat, Condensation: a theory of concepts. Proposition 5.2 proves .

As defined in Definition 4.3 of Sam Eisenstat, Condensation: a theory of concepts. Proposition 5.2 proves .

^Defined in 1.1 of Sam Eisenstat, “Research directions in condensation: varieties of objectivity”.

Defined in 1.1 of Sam Eisenstat, “Research directions in condensation: varieties of objectivity”.

^Cf. Theorem 5.10 of Eisenstat, which characterizes the perfect condensation case by exact reconstruction from each contributing observable together with the ordered Markov condition.

Cf. Theorem 5.10 of Eisenstat, which characterizes the perfect condensation case by exact reconstruction from each contributing observable together with the ordered Markov condition.

^In this line is overloaded; it means first the mutual information and then the index set.

In this line is overloaded; it means first the mutual information and then the index set.

^For the reconstruction direction, cf. the chain-rule argument in footnote 7 of Gillen and Daniel C; we're using the same argument here. For the Markov direction, cf. Proposition 5.9 and the proof of Theorem 5.10, , in Eisenstat.

For the reconstruction direction, cf. the chain-rule argument in footnote 7 of Gillen and Daniel C; we're using the same argument here. For the Markov direction, cf. Proposition 5.9 and the proof of Theorem 5.10, , in Eisenstat.

^Cf. the proof of Theorem 5.10, , in Eisenstat.

Cf. the proof of Theorem 5.10, , in Eisenstat.