Then your GPU fans spin up, your computer consumes enough electricity to concern your spouse when the power bill arrives, and eventually something appears on the screen.
Text. An image. Maybe a video.
That makes the process look simple. Input goes in. AI performs some apparent black magic. Output comes out. Meanwhile, inside the computer, the software loads several billion numbers into memory and pushes them through an increasingly aggressive series of mathematical operations.
So what is actually happening?
The specific details depend on whether you are running a language model, an image generator, or a video model. But the basic process is similar: your computer loads the model, converts your input into numbers, performs inference, decodes the result, and sends it back to whatever interface you are using.
At the end of the day, it is just a pipeline.
A very complicated, expensive, and occasionally temperamental pipeline.
First, the Model Has to Go Somewhere
An AI model is, at its core, a large collection of numbers called weights.
These weights contain the numerical relationships the model learned during training. They are why the model can associate words with other words, connect prompts with visual concepts, or determine what the next frame of a video might look like.
When the model is sitting on your SSD, it is not doing anything. It is just a large file consuming storage and making you question whether buying the smaller drive was really worth saving forty dollars.
Before the model can run, the application has to load those weights into memory.
Ideally, that means VRAM: the high-speed memory attached directly to your GPU. Depending on the software and the size of the model, some of it may instead remain in system RAM or move between RAM and VRAM as needed.
This is why launching a model can take time even before it generates anything. Your computer is reading gigabytes of data from storage, arranging the model, loading supporting components, and reserving space for the actual workload.
Once the model is loaded and ready, later generations may start much faster because the computer does not need to repeat that entire process.
Assuming, of course, that another application has not already wandered into your VRAM and eaten the available space.
Ahem. Chrome knows what it did.
Next, Your Prompt Stops Being a Prompt
The model does not read your prompt the way you do.
It cannot directly process words, images, or audio as meaningful human concepts. The software first converts them into numbers.
A language model breaks your text into pieces called tokens. A token might be a whole word, part of a word, punctuation, or something else the tokenizer recognizes. Those tokens are associated with numerical representations the model can process.
Image generation works a little differently. The text prompt is converted into embeddings that represent relationships between words and concepts. If you supply an image, that image may also be encoded into a compressed internal form called a latent.
Video models do something similar, except now the system also has to represent change over time.
Prompts, negative prompts, reference images, masks, control signals, and other inputs all become mathematical conditioning for the model. By this point, the sentence you typed is gone. It has been translated into math.
Probably for the best. The model would only judge your spelling.
Then Inference Begins
Inference is the process of using an already-trained model to produce an output.
The model is not relearning its entire training dataset every time you ask it a question. Its weights remain unchanged. It is using what it already learned to calculate a result from the input you provided.
For a language model, inference usually means predicting which token should come next.
The model examines the tokens already in the context, calculates probabilities for possible next tokens, selects one, adds it to the sequence, and repeats. That happens over and over until the response is complete.
This is why language models can stream text one piece at a time. You are watching the sequence being constructed as it runs.
Diffusion image models take a different path. They generally begin with noise and repeatedly predict what should be removed or changed to move that noise toward an image matching the prompt. Each sampling step refines the result.
Video generation extends the same general idea across both the image and the passage of time. The model has to create individual visual details while also maintaining motion and some degree of consistency between frames.
This is where most of the heavy computation happens. The GPU performs an enormous number of matrix operations as data moves through the model’s layers. Model size, precision, resolution, context length, sampling steps, and frame count can all change how much work is required.
This is also why two workloads using the same model can behave completely differently.
A short LLM conversation is not the same workload as feeding it 100,000 tokens of context.
A 512-pixel image is not the same workload as a 4K image.
A 17-frame video is absolutely not the same workload as an 81-frame video, no matter how politely you ask the GPU.
The Result Still Has to Be Decoded
When inference ends, the result may still not be something you can use directly.
A language model has generated token IDs. The software converts those tokens back into readable text.
An image model may have produced a latent representation rather than ordinary pixels. A component such as a VAE decoder converts that latent into the image you finally see.
A video workflow may have to decode many latent frames, combine them, and encode them into an actual video file.
This explains why a workflow can appear to finish the main generation step and then continue working.
The model may now be done, but the rest of the pipeline is not. Your computer still has to turn the answer into something recognizable, package it correctly, and save it without collapsing into an out-of-memory error three feet from the finish line.
What Happens When It Does Not Fit?
If the model and its working data fit inside VRAM, the GPU can access everything relatively quickly.
When they do not fit, the software has choices.
It can move part of the model into system RAM. It can load and unload different sections as needed. It can use smaller chunks, reduce precision, fall back to the CPU, or simply crash and present an error message written as though you personally offended Python.
Offloading can make an impossible workload possible.
It can also make it painfully slow.
System RAM is useful, but moving data back and forth between it and the GPU creates a bottleneck. The model may technically run while producing results slowly enough that you begin reconsidering your hobbies.
This is why “Can I run this model?” is not quite the right question.
The better question is, “Can I run this model at a speed and quality that makes it useful?”
Those are not the same thing.
Finally, You See the Result
Once the output has been decoded, the backend sends it to the interface.
Text appears in a chat window. An image appears in ComfyUI. Video frames become an MP4. The software may also attach metadata and generation settings. Temporary memory is released, or the model stays loaded for the next job.
This last separation is worth remembering: the interface is not necessarily the thing doing the inference.
A browser can freeze while the backend continues working. A preview can fail even though the output was successfully saved. The front end can behave like an uncooperative pile of garbage while the actual generation process underneath it remains perfectly healthy.
Ask me how I learned that one.
Current Thoughts
Pressing Generate feels like one action. It is not.
The model has to be read from storage, loaded into memory, given numerically encoded input, run through inference, decoded, packaged, and displayed.
Every part of that chain uses different resources and can fail in different ways.
Understanding those stages does not make local AI simple. It does, however, make it less mysterious.
When a workflow slows down, crashes, or produces something strange, you can start asking which part of the pipeline is responsible.
- Did the model fail to load?
- Did the workload exceed VRAM?
- Is inference still running?
- Is it decoding?
- Did the backend finish while the interface lost its mind?
Once you start seeing those separate systems, local AI stops looking like black magic.
It starts looking like engineering.
Engineering-adjacent chaos, perhaps.
But engineering.