Skip to Content

How Image Generation Works

The model does not draw. It removes noise from a patch of noise, step by step, until a picture is left. Get that one sentence and everything else — the seed, the step count, the way prompts behave — falls into place on its own.
September 12, 2026 by
How Image Generation Works
Krisztian Eyssen, Krisztian Eyssen

It does not draw, it removes noise

Start with the training, because generating is that same thing run backwards. You take a real photograph and add noise to it in many small steps until nothing is left but random speckle. In the original description this was a thousand steps. At every step you know exactly how much noise you added — so you have a perfect worksheet: here is the noisy image, and this much noise is in it.

What you train the model to do is solve that worksheet: look at the noisy image and say how much noise is in it. It does not learn "this is what a cat looks like", it learns "in this grainy thing, that part is noise and that part is not".

To generate, you run the whole thing in reverse. You start from a clean patch of noise, ask the model how much noise is in it, subtract that, and repeat. Twenty to fifty rounds later the noise has turned into a picture. At no point does the machine "draw" — it only ever estimates what should be taken away. That is the one sentence that really matters here, and almost every oddity you run into with image generators follows from it.

How text turns into a picture

The prompt is still missing from the description so far. Your text goes through a text encoder model that turns the sentence into a sequence of numbers, and those numbers sit alongside the image at every single denoising step. An attention mechanism ties the two together: every part of the image can "look at" what the text says.

This is where the most important shift of recent years sits, and it explains why prompting today is not what it was three years ago. Early on the text encoder was a short, keyword-trained model with a capacity of seventy-odd words — which is why piling up "cat, oil painting, golden light, 8k" worked and why flowing prose did not. In today's image models, by contrast, a full language model does the text encoding, and in the open-weight families this is documented: language models with billions of parameters sit in front of the image model.

That is why a proper sentence now produces a better picture than a keyword list: the language model understands relations — who is doing what to whom, what is behind what. The practical consequences are worked through in the post on writing image prompts, layer by layer.

Why it works in latent space

A 1024×1024 image is more than a million pixels, each with three numbers. Running the denoising over that fifty times would be an enormous amount of arithmetic. So most models do not work on pixels at all but on a compressed representation: an encoder first squeezes the image down onto a much smaller grid of numbers, the denoising happens there, and a decoder unpacks the result back into pixels at the end.

This has a visible consequence. The model "thinks" in the compressed space, and small, fine details — the pupil of an eye, a hairline stroke, a tiny letter — simply get very little room there. It is no accident that those are exactly the things that go wrong most often.

The seed, and why you do not always get one

The whole process starts from a random patch of noise. Which one exactly is decided by a number: the seed. Same seed, same prompt, same model and same settings give you the same picture — which is how you can return to an image that worked and ask for one small change to it.

The catch is that not every interface hands you the seed. Open-weight models and several APIs expose it; the two largest consumer chat interfaces have no such setting, so "run that exact thing again" is simply not available there. If your work needs repeatability — series, product shots, brand assets — that alone can decide which tool you pick. There is more on this in the post about choosing an image tool, where it is one of the main criteria.

What changed by 2026

Four things moved meaningfully, and all four show up in daily use.

  • The denoising network was rebuilt. Early models used a convolutional network; today's use a transformer. That is what made scaling possible, and better prompt-following came with it.
  • The text encoder became a language model. This is why long, natural-language prompts work — and also why the negative prompt is fading out: some recent models no longer have such a control at all, because the sentence itself carries the negation.
  • Editing moved inside. Generating an image and altering an existing one are now the same model with the same call; you do not need a separate tool for it.
  • The step count dropped. The research description used a thousand steps; in today's production systems the default sits around fifty, and at four in the accelerated variants.

What you cannot know: the two largest closed providers do not publish how their image models are actually built. Anyone confidently claiming "that is not even diffusion any more" is not speaking from a source. What has been published — the open-weight families — all belongs to the diffusion family, on a transformer base.

The numbers worth knowing

Four numbers worth being clear about, because all four show up either in your wallet or in your patience.

  • Step count. Typically 20-50 in production. Fewer steps is faster and cheaper, but fine detail smears; beyond a certain point more steps do not give a better picture.
  • Resolution. Today's models deliver between 1K and 4K, and higher resolution is not only slower but more expensive — most providers price it separately.
  • Aspect ratio. Not a crop but a generation parameter: the model builds a different composition for 16:9 than for a square. That is why cropping afterwards is not the same thing.
  • Images per request. Most tools hand you several variants at once. Choosing from four is almost always quicker than nursing one image until it comes good.

Common misconceptions

  • "It looks up similar pictures on the internet." No. There are no images in the model's weights, only patterns formed during training. That is also why you cannot "find the original inside it".
  • "If I describe it more precisely, I get exactly that." The description influences strongly but does not determine. A different starting noise gives a different picture for the same sentence — which is why there is a seed at all.
  • "More steps give a better picture." Up to a limit yes, after that it is only more expensive. Visible improvement comes from the model and the prompt, not from the step count.
  • "The negative prompt is mandatory." It was, once. Some recent models have dropped it, and the vendors themselves advise describing what you want rather than what you do not.

What to take away

Image generation is denoising, not drawing — the model estimates fifty times over how much noise to take away, and a picture is what is left. The prompt influences that process through a language model, which is why a proper sentence now works better than a keyword list. The starting noise is fixed by the seed, and if you need repeatability, check whether your chosen tool even hands it to you. Fine detail — small type, eyes, thin lines — goes wrong because the model is working on a compressed representation.

How Image Generation Works
Krisztian Eyssen, Krisztian Eyssen September 12, 2026
Share this post
Archive