LLM basin and how to drain them
Default basins, niche basins, and why LLMs can sometimes be genuinely useful for discovery.
When people say that large language models only produce "averaged" answers, they are often describing what happens when we ask them average questions in average ways.
A language model has something like a learned map of human concepts. If we were to project it as a 3d plane, it wouldn't be flat. There's a region that it is very easy to fall into. This is the default basin. When you ask a broad, normal question, the model usually slides into one of these basins and gives you the familiar "averaged" answer.
But the same map also contains smaller, less obvious regions. These are the niche basins. They may correspond to rare expert perspectives, minority traditions or highly specific professional intuitions. The model does not automatically go there. You often need stronger constraints, a different persona (of course), more context, or another deliberate attempt to pull the model away from the default basin (YMMV, there are multiple ways to do that).
Basins form because human knowledge is not a smooth territory. It is more like empty space filled sparsely with small clusters of facts. Even if some fields are well mapped internally, the spaces between them are often almost empty. Biology talks to statistics in some places, psychology talks to computer science in others, but many interfaces between domains are barely explored.
So, if you say "LLMs only average the past", I think it needs a caveat. Because you only describe the default basin. But when we deliberately navigate toward niche basins and force interpolation across underexplored interfaces, we may get candidate hypotheses or conceptual bridges. This is how OpenAI's model made a progress on the 80-year-old unit distance problem - it connected remote subfields of mathematics
Most of these outputs should not be trusted immediately. But I think there's a value in generation of structured possibilities in places where human knowledge has holes.
In that sense, the future of LLM-based discovery is about learning how to navigate the latent landscape.
(it's not "LLMs are creative" discussion - it is a very specific description on hypothesis generation in language space captured in LLMs)