On Creativity
I've spent most of my free time in the last year trying to make creative thinking legible to (and usable by) LLMs through natural collaboration. It started small, as an exercise in making aesthetic guidance generalizable, and gradually led to deeper questions about what knowledge is, where it originates from, and how it's transmitted.
Through experimentation and reflection I've begun to develop a framework for thinking about knowledge and its relationship with physical reality. It's changed the way I approach my professional work, my feelings about the emergence of machine intelligence, and my interpretation of the creative process.
I was thinking about that last point this morning, and jotted down the following.
Healthy humans possess the ability to simulate reality internally. We replay what we've experienced and extrapolate what we might experience: storing memories, recalling them, and predicting their implications. Accuracy varies, but we unavoidably do some version of this in order to survive the physical world.
Creativity is guided simulation, combining the essential ability with a degree of individual direction (the will). That guided simulation is seeded by experience and grows from what's been seen; this is where I disagree with Deutsch and Popper's belief creativity is fundamentally non-derivative.
The apparent non-derivativeness of some creativity is an illusion of compressive abstraction: a guided internal simulation can grow quite far from what's been seen before ever being made real. In other words, if we've all seen point A and someone presents point L, it's easy to miss the implicit stream of points B–K between the widely seen and the newly created.
Large leaps are definitionally present in people we think of as highly creative. Often these people are not conscious of or able to articulate the internal structure of the leaps they're making. This is because they're operating on a level of abstraction with respect to the form that is second-nature to them and imperceptible to us, trapped in our relatively “lower” level of abstraction.
The large leaps we come to respect (in art, in science, in belief) are not an unbounded generation of disconnected ideas: they're accurate simulated navigation of a reality-constrained possibility space powered by good explanations (conscious or intuitive). These leaps are, even before manifestation or verification, tracing a continuous topology of explanation that exists separate from the person making the leap or the physical reality the leap will eventually be performed in.
The language used by people working at the edges of any pursuit reflects this clearly: channeling, finding, sensing, touching, receiving, discovering. The feeling that the knowledge is coming “from” somewhere is not mystical. It's a sense signal in our simulative equipment. Highly creative individuals are very sensitive to this signal, and it allows them to simulate long chains of explanatory structure internally without veering out of explanation space (describable and achievable) and into pure conceptual space (describable, but not achievable).
In this sense, creativity is significant because it enables the physical materialization of constraint space at a greater speed than unguided reality is able to perform it. This is Deutsch's explanatory reach or the compressive effect of Hayakawa's ladder of abstraction: the distillation of complex systems into composable units crystallizes the structure of reality on one “level” and unlocks the malleability of reality at the next “level,” with constraint space's immaterial structure constraining the shape of material reality over time and space.
Turning our eye back on creativity itself, an implication emerges: while human creativity navigates constraint space intuitively, that navigability signals regularity, and regularity can be modeled and predicted with dramatically greater reach and accuracy by machines.
LLMs' ability to both navigate the constraint space implicitly encoded in their training data and extrapolate its implications in semi-novel contexts after training hints at where their latent potential could lead.