Hyperinvariant Subspace Problem: The Backward Intertwiner Behind OpenAI’s Proof

Apply OpenAI’s operator to a vector supported at coordinates 0, 1, 2, and onward, and its support stays in that same tail. Now apply a second operator that commutes with the first. Even a vector concentrated at coordinate 0 can land at coordinate −1, in a different fiber. That backward move is the mechanism behind a negative answer to the hyperinvariant subspace problem. OpenAI constructs a nonzero quasinilpotent operator on a separable complex Hilbert space that has many ordinary invariant subspaces but no nonzero proper closed subspace invariant under its entire commutant.

1. What OpenAI’s Proof Establishes About the Hyperinvariant Subspace Problem

Theorem 1.1 of Backward Intertwiners and a Transitive Commutant builds a bounded operator SS out of a measurable family of bilateral weighted shifts. Each fiber has exactly the coordinate tails as closed invariant subspaces. A contraction couples successive fibers and sends each tail with a finite cutoff densely into the preceding tail. The shift, the intertwiner, and the scalar multipliers generate a transitive algebra. An algebra is transitive when its only closed common invariant subspaces are zero and the whole space.

Key Facts About the Counterexample
TopicResult
QuestionDoes a bounded nonscalar operator always have a nonzero proper closed hyperinvariant subspace?
Answer and ScopeNo. Corollary 1.2 gives a counterexample on every infinite-dimensional separable complex Hilbert space.
Forward OperatorA nonzero bilateral weighted shift acting fiber by fiber, with operator norm e−10e^{-10} and spectral radius zero.
Backward IntertwinerA commuting contraction that transports vectors to the next base point and moves their coordinate indices down by one.
CommutantA proper, strongly closed, unital transitive algebra. Its only common closed invariant subspaces are zero and the entire Hilbert space.
Ordinary InvarianceEach fixed coordinate tail remains a proper closed invariant subspace for the forward operator.

Corollary 1.2 transfers the example to every infinite-dimensional separable complex Hilbert space by unitary equivalence. The commutant is proper and strongly closed, so this example also answers negatively the question of whether every unital transitive operator algebra is strongly dense in the algebra of all bounded operators. The paper is part of OpenAI’s mathematics release. The family-293 Lean scope note documents the existence theorem with the stated commutant properties.

2. Invariant vs. Hyperinvariant Subspaces: What Was the Original Problem?

For a bounded operator SS on a complex Hilbert space HH, a closed subspace MM is invariant if SM⊆MSM\subseteq M. Hyperinvariance requires AM⊆MAM\subseteq M for every AA in the commutant of an operator. Writing ℬ(H)\mathcal{B}(H) for the bounded linear operators on HH, the commutant is

{𝑺}′={𝑨∈ℬ(𝑯):𝑨𝑺=𝑺𝑨}.\boldsymbol{\{S\}’=\{A\in\mathcal{B}(H):AS=SA\}.}

The problem asks whether every bounded nonscalar operator on an infinite-dimensional separable complex Hilbert space has a nonzero proper closed hyperinvariant subspace. Since SS belongs to its own commutant, hyperinvariance implies ordinary invariance. The ordinary invariant subspace problem asks only for a subspace preserved by SS. Here the fixed tails supply such subspaces explicitly. The counterexample addresses what can be guaranteed about the whole commutant.

The distinction also explains why adjoints matter. In a self-adjoint algebra, common invariant subspaces are reducing, meaning their orthogonal complements are invariant too. The associated projections then commute with the algebra. This is the setting of irreducible representations of C*-algebras. Here the stronger test for a candidate subspace is invariance under an algebra that can contain operators without containing their adjoints, so the projection argument applies first to the self-adjoint multiplier family.

3. Why Earlier Weighted-Shift and Operator-Algebra Methods Were Insufficient

Domar’s weighted-translation theory supplies the support-separation method. Grabiner’s work on unicellular shifts gives the setting in which the invariant subspaces form a chain. Classifying a shift’s invariant subspaces narrows the possibilities for hyperinvariance, but the classification alone leaves open how other commuting operators act on the chain. The missing step in this construction is a bounded commuting map that changes the cutoff between fibers.

Arveson’s density theorem gives a strong positive result under an extra hypothesis. A transitive algebra on a separable Hilbert space is strongly dense if it contains a maximal abelian self-adjoint subalgebra of all bounded operators. The scalar multipliers here leave an entire infinite-dimensional coordinate space at each base point. For example, the projection onto coordinate zero commutes with them, although a scalar multiplier acts identically on all coordinates at that point. Thus the multiplier algebra used in the proof falls outside Arveson’s hypothesis.

The manuscript contrasts its construction with Atzmon and Sodin’s completely indecomposable bilateral weighted shifts, whose spectrum is the unit circle. Their invariant-subspace analysis left open the hyperinvariant question for those shifts. OpenAI’s added base variable gives it room to design a different commuting action. It also describes a companion construction with an irrational rotation on the circle. The 2-adic proof supplies the concrete coordinate lattice and explicit power estimate developed below.

4. Constructing the Weighted-Shift Operator on a Hilbert Space

Let K=ℓ2(ℤ)K=\ell^2(\mathbb{Z}), the square-summable sequences indexed by all integers, and let eje_j be the unit vector at coordinate jj. For a finite integer hh, the tail K≥hK_{\geq h} is the closed span of the eje_j with j≥hj\geq h. The label −∞-\infty gives the whole fiber and +∞+\infty gives the zero space. On the base XX with probability measure μ\mu, the global Hilbert space is H0=L2(X,μ;K)H_0=L^2(X,\mu;K), the space of square-integrable vector fields.

𝑺𝒙𝒆𝒋=𝜷𝒋(𝒙)𝒆𝒋+1,(𝑺𝒇)(𝒙)=𝑺𝒙𝒇(𝒙).\boldsymbol{S_xe_j=\beta_j(x)e_{j+1},\qquad (Sf)(x)=S_xf(x).}

Here βj(x)>0\beta_j(x)>0 is the weight attached to the step from jj to j+1j+1. For instance, e0+2e2e_0+2e_2 in K≥0K_{\geq0} becomes β0(x)e1+2β2(x)e3\beta_0(x)e_1+2\beta_2(x)e_3. The indices advance, so the image lies in K≥1K_{\geq1} and hence in K≥0K_{\geq0}. This explains why tails are invariant. Determining whether other subspaces can survive the shift requires controlling vectors with contributions at arbitrarily distant coordinates.

5. The Quadratic Weighted-Translation Criterion: Why Only Coordinate Tails Survive

Lemma 2.1 rewrites a fiber as a right translation RR on sequences with norm given by the square sum of wjzjw_jz_j, where wjw_j is positive and decreasing. The map z↦(wjzj)jz\mapsto(w_jz_j)_j is unitary onto KK and carries RR to the shift with weights βj=wj+1/wj\beta_j=w_{j+1}/w_j. Convexity of logwj\log w_j on the nonpositive indices means successive differences increase. They decrease on the nonnegative indices, where the logarithm is concave. The lemma also needs the signed-root sum ∑j≠0wj1/j\sum_{j\ne0}w_j^{1/j} to be finite, and the following growth for some a>log3a>\log3.

lim𝒏→∞⁡log⁡𝒘𝒏𝒏2=−𝒂,lim𝒏→∞⁡log⁡𝒘−𝒏𝒏2=𝒂.\boldsymbol{\lim_{n\to\infty}\frac{\log w_n}{n^2}=-a,\qquad \lim_{n\to\infty}\frac{\log w_{-n}}{n^2}=a.}

Take a vector cc in a proper closed invariant subspace and a nonzero continuous linear functional represented by coefficients bb that annihilates the subspace. Their correlations ∑jbjcj−r\sum_j b_jc_{j-r} vanish at every nonnegative shift rr. For example, if cc starts at hh and the last nonzero entry of bb is at t>ht>h, the shift r=t−hr=t-h aligns those endpoints. The sum has just one possible nonzero term, the product btchb_tc_h. That makes vanishing impossible. The proof bounds coefficients at the relevant end by a convex envelope of negative log magnitudes whenever its support extends indefinitely in the relevant direction. Where the envelope bends strictly it touches an actual coefficient. If both problematic ends are unbounded, the proof matches slopes at two such contact points. Their product dominates the other middle terms by geometric decay, with relative sum at most 2/(eλ−1)<12/(e^\lambda-1)<1 for log3<λ<a\log3<\lambda<a. Normalized estimates make the two outer tails negligible. That correlation therefore cannot vanish.

The resulting support separation places each annihilator’s rightmost index at or before the vector’s first supported index. The zero-shift pairing rules out equality. Once the earliest index attained by any vector in the subspace is identified, every vector vanishes below it and every annihilator vanishes on its tail. Closedness and separation by continuous functionals give exactly that tail.

6. How 2-Adic Weights Produce the Required Opposite Drifts

The base is X=ℤ2X=\mathbb{Z}_2, the 2-adic integers, whose points record compatible residues modulo every power of two. Put Nm=2mN_m=2^m and let Tm(x)T_m(x) be the residue between zero and Nm−1N_m-1. Under Haar probability every residue at level mm has mass 1/Nm1/N_m. The map θx=x+1\theta x=x+1 permutes these residues cyclically. An invariant set has the same conditional proportion in each residue class. Since those classes generate the measurable sets up to completion, the invariant set is independent of itself, giving ergodicity.

Write βj(x)=e−aj(x)\beta_j(x)=e^{-a_j(x)} and uses blocks Im={k:ℓm−1≤k<ℓm}I_m=\{k:\ell_{m-1}\leq k<\ell_m\} with ℓ0=1\ell_0=1 and ℓm=8m\ell_m=8^m. For k∈Imk\in I_m, the formula below uses c=10c=10, D=100D=100, and Jm=D(Nm−2)J_m=D(N_m-2). The positive side uses aj=aj+a_j=a_j^+ for j≥0j\geq0 and the negative side uses aj=a−j−1−a_j=a_{-j-1}^- for j<0j<0. Both central exponents equal cc.

𝒂𝒌+(𝒙)=𝒄(𝒌+1)+𝑱𝒎+𝑫𝑻𝒎(𝒙),\boldsymbol{a_k^+(x)=c(k+1)+J_m+DT_m(x),}
𝒂𝒌−(𝒙)=𝒄(𝒌+1)+𝑱𝒎+𝑫(𝑵𝒎−1−𝑻𝒎(𝒙)).\boldsymbol{a_k^-(x)=c(k+1)+J_m+D\bigl(N_m-1-T_m(x)\bigr).}

Advancing the base usually raises the positive exponent by DD and lowers the negative exponent by DD. A residue wrap reverses those increments in that block. For example, at level two a move from residue 1 to 2 gives changes of +100+100 and −100-100. A move from 3 to 0 gives −300-300 and +300+300. Across block boundaries the offsets keep both side sequences increasing outward. Their corrections are uniformly O(k1/3)O(k^{1/3}), so cumulative log weights have quadratic limits with a=5a=5. The bound ak±≥c(k+1)a_k^\pm\geq c(k+1) also gives the signed-root summability, completing the hypotheses in every fiber.

7. The Explicit Power Bound That Proves Quasinilpotence

Four-step cost chart for the hyperinvariant subspace problem, showing why the shift has spectral radius zero.
Four-step cost chart for the hyperinvariant subspace problem, showing why the shift has spectral radius zero.

A power SxnS_x^n multiplies a basis vector by the product of nn consecutive weights before shifting it by nn indices. The logarithmic cost per step is at least 10d(j)10d(j), with d(j)=j+1d(j)=j+1 for j≥0j\geq0 and d(j)=−jd(j)=-j for j<0j<0. The cheapest interval centers as closely as possible on indices −1 and 0, where both costs have d=1d=1. For n=2mn=2m the minimum sum is m(m+1)m(m+1). For n=2m+1n=2m+1 it is (m+1)2(m+1)^2. Each fiber power sends different basis vectors to orthogonal outputs, so the bound on products is a norm bound. Its uniformity in xx gives the same estimate on H0H_0.

‖𝑺𝒏‖≤exp⁡(−10⌊(𝒏+1)24⌋).\boldsymbol{\|S^n\|\leq\exp\!\left(-10\left\lfloor\frac{(n+1)^2}{4}\right\rfloor\right).}

For four steps the lower costs are 2,1,1,22,1,1,2, yielding the upper bound e−60e^{-60}. These are lower bounds on the logarithmic costs, since the residue terms can increase the actual costs. For two steps, the interval at −1 and 0 uses the two exact central weights, so ∥S2∥=e−20\|S^2\|=e^{-20}. This gives a case of equality. Taking nnth roots makes the bound tend to zero because the floor term grows quadratically. The spectral radius formula therefore gives radius zero. The two central weights are exactly e−10e^{-10}, so this is the exact operator norm.

8. The Backward Intertwiner: Building an Operator That Commutes but Moves Tails

	Two commuting paths in the hyperinvariant subspace problem show a forward shift and a backward move between fibers.
Two commuting paths in the hyperinvariant subspace problem show a forward shift and a backward move between fibers.

Define (Vf)(θx)=Vxf(x)(Vf)(\theta x)=V_xf(x) with Vxej=bj(x)ej−1V_xe_j=b_j(x)e_{j-1}. Both orders of composition starting at eje_j in fiber xx end at coordinate jj in fiber θx\theta x. Equating the coefficients gives the recurrence. Since every shift weight is positive, it can be solved in both directions starting from b̃0(x)=1\tilde b_0(x)=1. For the central input index j=0j=0, the two weights in the recurrence are both e−10e^{-10}, so b1(x)=b0(x)b_1(x)=b_0(x). At other indices the ratio compensates for both the coordinate change and the change of base point.

𝒃𝒋(𝒙)𝜷𝒋−1(𝜽𝒙)=𝜷𝒋(𝒙)𝒃𝒋+1(𝒙).\boldsymbol{b_j(x)\beta_{j-1}(\theta x)=\beta_j(x)b_{j+1}(x).}

A wrap at level mm has probability 2−m2^{-m}. The sum over levels is finite, so the first Borel–Cantelli lemma leaves only finitely many exceptional blocks almost everywhere, even with dependent levels. In the logarithm of the unnormalized coefficient, the forward sum takes the negative of the cumulative positive-side drift, and the backward sum takes the cumulative negative-side drift. Beyond the finitely many exceptional blocks, each contributes −D-D times the distance from zero plus a finite fiber-dependent additive constant. Telescoping introduces a boundary term with linear slope cc and a sublinear correction. Since D−c=90D-c=90, the unnormalized coefficients tend to zero at both ends.

A countable supremum of these coefficients is measurable and finite on the invariant set of full measure obtained by taking all translates of the good set. Dividing by that supremum leaves the recurrence intact and makes the fiber maps contractions. Setting the coefficients to zero off that set and using probability preservation yields a global contraction with SV=VSSV=VS. Every basis direction in K≥h−1K_{\geq h-1} is attained from K≥hK_{\geq h} where the coefficients are positive. Taking closures gives VxK≥h¯=K≥h−1\overline{V_xK_{\geq h}}=K_{\geq h-1}. The coefficients decay to zero, so the argument uses density, with no surjectivity assumption.

9. From Scalar Multipliers to Measurable Fields of Invariant Subspaces

Suppose a closed subspace MM is invariant under SS, VV, and every scalar multiplier MgM_g, defined by (Mgf)(x)=g(x)f(x)(M_gf)(x)=g(x)f(x) for essentially bounded gg. The family contains adjoints, since Mg*=Mg‾M_g^*=M_{\bar g}. Therefore MM reduces each multiplier and its orthogonal projection PP commutes with them. The field lemma gives a measurable projection field PxP_x with (Pf)(x)=Pxf(x)(Pf)(x)=P_xf(x). Membership in MM becomes a pointwise condition almost everywhere.

Testing the invariance identities on constant basis fields yields fiber invariance under SxS_x and transport of ranPx\operatorname{ran}P_x into ranPθx\operatorname{ran}P_{\theta x} under VxV_x. Countably many basis tests suffice. The first condition makes ranPx=K≥s(x)\operatorname{ran}P_x=K_{\geq s(x)}. The label is measurable because s(x)≤js(x)\leq j holds precisely when ⟨Pxej,ej⟩=1\langle P_xe_j,e_j\rangle=1.

10. The Decisive Contradiction: Why Strict Descent Forces a Transitive Commutant

	Tail cutoff descent in the hyperinvariant subspace problem leaves only the zero space and the full space.
Tail cutoff descent in the hyperinvariant subspace problem leaves only the zero space and the full space.

A finite input label forces the next fiber to contain the closed tail starting one coordinate earlier. Tail inclusion reverses the order of labels, giving

𝒔(𝜽𝒙)≤𝒔(𝒙)−1when 𝒔(𝒙)∈ℤ.\boldsymbol{s(\theta x)\leq s(x)-1\qquad\text{when }s(x)\in\mathbb{Z}.}

A full input fiber also forces a full output fiber by density. In Lemma 5.1 let E={s=−∞}E=\{s=-\infty\} and Fj={s≤j}F_j=\{s\leq j\}. Up to null sets, θE⊆E\theta E\subseteq E, so probability preservation makes EE invariant modulo null sets. Ergodicity gives it measure zero or one. Also θFj⊆Fj−1⊆Fj\theta F_j\subseteq F_{j-1}\subseteq F_j, so all the FjF_j have equal measure. Their intersection over all integer cutoffs is EE. If EE is null, each FjF_j is null, and their countable union excludes finite labels almost everywhere. The argument compares the measures of nested sublevel sets, so it works even if the labels have no finite expectation. Only the all-zero and all-full fields remain. The generated algebra is transitive, and the full commutant inherits that property because it contains the generators.

11. What the Counterexample Changes and What It Leaves Open

Commutation is preserved under strong operator limits, which converge in norm on each vector. The coordinate-zero projection fails to commute with SS on the constant field e0e_0, so the commutant is a proper strongly closed transitive operator algebra. Each ordinary fixed tail survives SS and loses invariance under VV. Each fiber shift is compact by truncating its decaying weights. Globally, an orthonormal sequence of scalar functions at coordinate zero moves to an orthogonal sequence at coordinate one with common positive norm e−10e^{-10}. Hence the global operator is noncompact.

There is a second structural distinction in Section 6.3. Let UU be the unweighted forward shift and |S|=(S*S)1/2|S|=(S^*S)^{1/2}. Each polar conjugate U−n|S|UnU^{-n}|S|U^n reads one shifted weight. A finite family gives a countable partition into joint value sets, producing an atomic abelian von Neumann algebra. The whole family recovers both the coordinate, from the two central weights, and every 2-adic residue. It generates the diffuse multiplication algebra on X×ℤX\times\mathbb{Z}. The modulus has pure-point spectral resolution, while the combined family distinguishes arbitrarily fine residue classes of a nonatomic base. This diffuse algebra sits outside the commutant: the modulus scales coordinates zero and one differently, while the shift sends the former to the latter. The spectral calculation is independent of the transitivity argument.

The answer to the hyperinvariant subspace problem rests on the interaction between the fiber classification and the commuting transport. Even an explicit chain of ordinary invariant tails can leave the commutant with no common nontrivial closed subspace. Follow Binary Verse AI for more research explainers in operator theory, from the Crouzeix conjecture on the numerical range to the Phillips–Toms formula, and for coverage of machine-checked mathematics.

What is the hyperinvariant subspace problem?

It asks whether every bounded nonscalar operator on an infinite-dimensional separable complex Hilbert space has a nonzero proper closed subspace invariant under every bounded operator commuting with it. OpenAI’s construction gives a negative answer.

Did OpenAI’s proof solve the ordinary invariant subspace problem?

No. The constructed operator preserves every fixed coordinate tail. The proof shows that its commutant has no nonzero proper closed common invariant subspace.

Why is the constructed operator quasinilpotent?

The paper bounds the norm of the nth power by an exponential whose exponent is negative and has quadratic growth in magnitude. The nth root of that bound tends to zero, so the operator has spectral radius zero.

What is a backward intertwiner, and why is it essential?

It is a bounded operator that commutes with the constructed shift and moves coordinate tails backward between fibers. Its image of a tail is dense in the preceding tail, so common invariance forces the next fiber to contain that larger tail.

How does the proof establish a transitive commutant?

The only closed subspaces simultaneously invariant under the shift, the backward intertwiner, and every scalar multiplier are zero and the full Hilbert space. Each of these generators belongs to the shift’s commutant, so the same conclusion holds for the full commutant.

Why does the result not contradict Arveson’s density theorem?

Arveson’s theorem requires the transitive algebra to contain a maximal abelian self-adjoint subalgebra. The scalar multiplier algebra used here has infinite multiplicity and fails maximality. The projection onto a single coordinate commutes with every scalar multiplier and lies outside their algebra.

What part of OpenAI’s proof is documented in Lean?

OpenAI’s family-293 Lean documentation records the existence, on every infinite-dimensional separable complex Hilbert space, of a nonzero quasinilpotent operator with a proper, strongly closed, transitive commutant. The scope note states the existence theorem without itemizing every intermediate step of the manuscript.

Leave a Comment