A system cannot fix what it cannot represent
Give a network of coupled oscillators a damaged image and something strange happens: it scrubs away scattered noise effortlessly, then fails to fill a single clean-edged hole. The split is a fault line between two kinds of work, cleaning and creating, and the same line runs straight through the visual cortex, concert-hall acoustics, and modern machine learning.
Mechanically it’s simple: little clocks, each nudging its neighbors toward agreement, relaxing in step until the field settles onto the nearest clean shape it has learned. That trick meets the two kinds of damage very differently.
Case one: scattered static. Speckle the image with noise, everywhere. The field eats it effortlessly — every corrupted point has intact information sitting right next to it, so a few settling steps pull each site back into line. This is what these systems are for, and they’re beautiful at it.
Case two: one big hole. Cut out a single contiguous region and ask the field to fill it. It fails — not slightly, but fundamentally. Why is the whole point.
The failure is not speed. It’s the wrong answer.
The intuition most people carry — including a lot of machine-learning people — is: if I iterate a local rule long enough, information will eventually propagate everywhere. Given enough steps, the fix reaches the center.
This is wrong in two separate ways.
First, the finite-budget reality. Local coupling moves information at a finite speed: each step, influence spreads by about one coupling radius. It’s a light cone, identical to how a wave front or a diffusion front travels. The interior of a large hole is many radii from the nearest intact pixel, so within any real settling budget, the boundary information physically cannot arrive at the center. There aren’t enough steps in the universe you’ve allotted.
Second, even with infinite time, you get the wrong result. A local operator filling an empty interior is solving Laplace’s equation with the hole’s edge as its boundary condition. The answer a clean linear field converges to is the harmonic extension: the smooth, minimal-energy membrane stretched across the gap. A soap film. And that’s the best case — a trained nonlinear field doesn’t settle to anything so tidy; it lands on some low-frequency-dominated attractor that in practice is often worse, smeared and streaked with boundary artifacts. Either way the verdict is identical: stable, correct by the system’s own rules, and smooth nonsense — every bit of interior detail erased, because that detail was never determined by local information in the first place.
So the failure isn’t that the system is too slow to reach the right fixed point. The failure is that the right fixed point isn’t in the system at all.
Say it in frequencies
You might think the field simply lacks the low frequencies — that it speaks only in short wavelengths, and a big hole is a long word it can’t pronounce. Not quite. A local coupling operator has eigenmodes at every spatial frequency, the lowest included; diffusion is in fact dominated by those long-wavelength modes, precisely because they decay slowest.
What’s missing is the value. Nothing local can tell the field what that low-frequency mode should be set to. The content that belongs in the hole isn’t present in the neighborhood to be read off, and whatever could in principle be inferred from the distant boundary can’t reach the center within any finite settling budget. The mode is representable; its value is both undetermined by local data and unreachable in time.
Which, in the loose and useful sense, is still the one-liner:
A system cannot fix what it cannot represent.
The real axis: contraction vs. transport
Strip away the specifics and a much deeper distinction appears:
Local dynamical systems are contraction machines, not transport machines.
- Contraction — remove inconsistency, denoise, pull the state back toward the learned manifold.
- Transport — move information across space, coordinate globally, fill regions that are missing entirely.
Scattered noise is almost entirely off the manifold, so contraction annihilates it. A contiguous hole is different in kind: the content that belongs there lies along the manifold but somewhere else in space, and getting it there is transport. Contraction cannot do transport. That single sentence explains why the exact same mechanism triumphs at one task and collapses at the other — and it’s not an oscillator quirk. A plain local convolutional-recurrent system fails in precisely the same way. It’s a property of the whole class of local operators.
Which yields the tightest statement of all:
Local dynamics solve consistency, not completeness.
Nature already knows this
Nature runs into this wall constantly, and it has never once solved it by making a bigger, better field of purely local oscillators. Every time it bolts on a non-local mechanism.
- The brain. Perceptual filling-in is real — the blind spot, completion across a scotoma. But it isn’t bare lateral spreading. It’s driven by feedback from higher visual areas with huge receptive fields and learned priors, riding long-range cortico-cortical and thalamo-cortical loops. The global carrier is literally wired in as anatomy.
- Synchronizing physics. Kuramoto oscillators, laser arrays, Josephson-junction arrays achieve global coordination through mean-field or long-range coupling or a common drive — never strict nearest-neighbor. Global order is a long-wavelength phenomenon, and you have to pay for long wavelengths with long-range connections.
- Acoustics. A room’s low-frequency behavior is owned by standing-wave modes that span the entire cavity. A “hole” in a pressure field is a non-issue because the representation is already global. It works because the basis is non-local.
- Holography. The inverse case, and a telling one. On a hologram, each image region is smeared across the whole plate, so occluding a contiguous patch costs you resolution, not a chunk of the picture. Distributed encoding makes big holes benign — a direct hint that the lever, if you want to shrug off holes, is a distributed representation, not a cleverer local rule.
- Morphogenesis. Planaria and hydra regrow enormous missing pieces — via stored positional information, morphogen gradients, long-range signaling. Evolved priors, not local relaxation.
And modern machine learning agrees: Diffusion models own the scattered-noise, iterative-denoise regime — the exact oscillator sweet spot. But when they inpaint a large hole, the transport is done by the learned global prior: the U-Net’s wide receptive field and attention supplying the long-range structure. The dynamics denoise; a global prior does the transporting. Same division of labor, every single time.
What all these non-local tricks share is one function: they collapse distance in representation space. Instead of needing k steps to move information across k units of space, they make it roughly O(1). Attention, global standing-wave modes, mean-field coupling, distributed holographic encoding, long-range cortical feedback — these are all the same move wearing different clothes.
Three regimes, not two
Put it together and the landscape has three floors, not two:
- Contraction. Distributed noise, small perturbations. Task: project back onto the manifold. Tool: local dynamics — oscillators, diffusion steps, local recurrence.
- Transport. Large missing regions, long-range dependencies. Task: move information across space. Tool: attention, global coupling, hierarchy.
- Generative prior. Genuinely underdetermined interiors, where the missing content isn’t recoverable from anywhere — it has to be invented plausibly. Task: hallucinate the right thing. Tool: a learned world model.
Local dynamics give you floor one. A contiguous hole demands floors two and three. Confuse the floors and you will build something that hums along beautifully and then quietly converges to a soap film in the middle of the exact region you cared about most.
The design law
A local operator’s reach grows with depth: stack enough settling steps, or enough layers, and the effective receptive field expands one light-cone at a time until it eventually spans the hole. So it isn’t that local dynamics can’t transport. It’s that the price is brutal, and fixed:
Local coupling fails whenever its effective receptive field is smaller than the defect. To fill a hole of size L you need reach ≥ L — and for a purely local field, reach costs O(L) depth.
The scarce resource is reach, and there are exactly two ways to buy it. The expensive way is depth: settling steps paid out linearly in the size of every defect you might ever meet. The cheap way is non-locality — attention, mean-field coupling, a standing-wave basis, holographic encoding — which delivers reach in O(1), independent of distance. That is the single move under all of them: they refuse to pay the depth tax.
It sits alongside Nyquist, the speed of light, the bias–variance tradeoff — not something you out-engineer, but something you architect around, the way biology and physics always have: a fast local field for coherence, and non-local structure layered over it for reach. Or in short:
Local dynamics can clean signals, but only global structure can create them.