Block Generation: How We Made Endless Visual Novels Affordable

MDL started as something nobody had built before: an endless visual novel engine. Not a branching script with a finite number of endings, but a story that keeps writing itself for as long as you keep playing. You make a choice, the engine writes the next moment of your world, you respond, and it writes again. There is no last page.

It worked better than we expected. The immersion was the thing people kept coming back to. We watched players stay inside a single universe for days at a stretch, the way you fall into a good book and lose the afternoon, except the book was writing itself around their decisions. That was the proof we needed that the core idea was real.

There was one problem, and it was a big one. MDL was a premium engine. To experience a story for any meaningful length of time, you had to pay, and you had to pay a lot.

Why it was so expensive

The first version of the engine ran on a lower-cost generation path. It had real creative range, and for our task that combination was hard to beat. Every step of the story was a fresh call to it. We even built a prefetching system on top so the next beat would already be on its way before you asked for it, which kept the experience feeling smooth instead of turn-based.

Smooth, but costly. One step, one call, every time. And the cost got worse in a way we did not anticipate. The engine would sometimes fall into a rhythm of producing illustrated scenes back to back, and each of those scenes cost 15 credits to generate. A few of those in a row and a player could watch their balance evaporate in minutes. Some people blew through their credits before they had really gotten anywhere.

For an experience whose whole promise is endless, that is close to fatal. An endless story you can only afford in short bursts is not the product we set out to build. We needed the economics to make sense, not just the storytelling.

The two months we spent on the wrong answer

The plan we committed to was training our own generation system. We had tens of thousands of real generations from actual play, which is exactly the kind of data you want, and the idea was to use it to train an open-source system we could host ourselves and run far more cheaply than calling out to a flagship for every step.

It did not work, and it is worth being honest about why.

The candidates we could realistically train were smaller. Smaller is what makes them cheap to run, but smaller is also where the creativity goes to die. None of the candidates were inherently creative in the way the task demands. And they were not small enough to be cheap on the training side either. We did not have the capital to fully fine-tune them, only to nudge them.

This is the uncomfortable shape of the problem. The reason the baseline was good at writing a living story is that it came from a much larger system, and that scale is also why it adapted to our task so readily. The system we would actually want to fine-tune was that baseline itself, and the moment you fine-tune something that large you have given back the cost savings that were the entire point.

So we stopped. Every candidate we trained came out meaningfully worse than the baseline, and the thing that gave it away every time was the writing. You can forgive a candidate a lot, but you cannot forgive flat prose in a product whose only job is to make you feel like you are inside a story.

The accident

There was one experiment from the training work that did not look like a dead end.

While we had the training pipeline open, we tried to teach a candidate to do something the baseline had never reliably done for us: produce several story steps in a single generation instead of just one. If we could get multiple beats per call, the cost math changed completely. The trained candidate managed it, sort of. It could output multiple steps, but not consistently enough to ship, and the writing was still the weak link.

So we went back to the old baseline and moved on.

Except we left something behind. When we reverted, we forgot to turn structured output enforcement back on, and the multi-step prompt from the experiment was still in place. Structured output enforcement is the mechanism that forces generated data into a strict, schema-shaped form. It is normally how we guarantee the engine gets something it can parse. With it switched off and the multi-step instructions still present, the baseline did the thing the trained candidate had only fumbled at: it produced clean multi-step output, consistently.

That was the part we did not see coming. With strict structured output turned on, the same baseline had always struggled to produce more than one good step at a time. Take the constraint away and it could write a whole block of beats at once, and write them well. We had spent two months trying to train this behavior into a smaller candidate, and the behavior was sitting in the baseline the whole time, locked behind a setting.

We tested it properly before we let ourselves believe it. The writing was not worse. If anything it was slightly better, because the engine was composing a continuous run of scene rather than restarting cold each step. And it was steady: across real playthroughs, the old setup settled at an average of about 6.7 story steps per generation.

What that bought us

We call the new approach block generation: one call produces a block of playable beats, and the player walks through them while the next block is already being prepared.

And here is the part we did not expect when we started. Because each call now carries a whole block instead of a single beat, the per-beat cost dropped far enough that we could afford to stop running the lower-cost path and put our strongest generation path on the task. The writing got better, as you would hope — but the blocks also got longer. Where the old setup settled around 6.7 beats per call, the production path composes more story in a single pass. Across the first real sessions on it the blocks ran 6, 8, and 13 beats, an average of about 9 story steps per generation. The better path is also the cheaper one to run, because it does more per call.

Put that rate against a single long session — call it a 150-beat story — and the block-versus-one-step math comes out like this:

Same story. Same number of beats reaching the player. Close to 90% fewer generation calls, and well over 100 credits saved on a single universe.

The reason this saves money is worth being precise about, because it is not where people assume. The system still writes every beat. The output, the actual story, is the same length. What changes is that all the work around the writing — assembling the prompt, the round trip, the per-call billing — now gets paid once per block instead of once per beat.

The images, and the wait

The illustrated scenes are the other place block generation quietly fixed things, on two fronts.

The first is how many illustration requests we make. Under the old one-step system, an illustrated scene could be triggered on any step, and the engine would sometimes chain them, firing several heavy scene renders back to back. That was both the latency problem and the 15-credits-a-scene problem stacked on top of each other. With block generation the structure naturally caps it: a block carries at most one illustrated scene, and it tends to land in the middle of the block rather than at the front. Instead of a run of synchronous image waits and a run of 15-credit charges, you get one, placed where there are beats on either side of it.

The second is where the wait happens, and this is where the longer blocks really pay off. The old engine only ever had one step in flight plus a single prefetched one, so when a scene came up there was at most a beat of reading ahead of it — almost nothing to hide a twenty-second image behind. A longer block gives the engine more room, and the prompt now steers each scene away from the front of the block. So the text block goes out first, the player reads through several beats, and the image finishes in the background before they ever reach it.

The playthrough we measured generated three illustrated scenes:

The two scenes that landed with beats ahead of them generated in the background. Across the run, about 65% of total scene-render time (~41s of ~63s) happened after the response had already reached the player, hidden behind beats they were still reading. The background renders averaged around 20.6s each, and with three beats of reading in front of them at roughly 6 to 8 seconds per beat, that is 18 to 24 seconds of cover, enough to swallow most of the wait.

The one exception proves where to push next. The sketch reveal sat at the front of its block with nothing to read ahead of it, so it was still an inline wait, about 37 seconds of total response time with roughly 23 of those being the image. That is the case the placement is meant to avoid, and it tells us the goal is not just “make images faster” but “make sure there are beats to read before the expensive one arrives.”

There is a quieter consequence of all this that we are only starting to spend. Because illustrated scenes are now sparse — roughly one per block instead of the old engine firing one off at nearly every step — each image is a rarer, more deliberate event. That changes what we can afford per image. The scenes that do appear can now be treated as rarer, more deliberate moments: more detail, stronger composition, and better timing without the cost or the latency running away from us, because there are so many fewer of them.

Where this leaves us

Block generation is the gain we needed to make an endless visual novel actually worth playing at length instead of in expensive bursts. Lower cost per beat is the headline, but the more interesting consequence is what it frees up. Because each call now carries a block of beats instead of one, we can afford to use our strongest generation path for the task. The production path raises the writing quality and the immersion another step while producing even longer blocks, on stories that are cheaper to tell than they ever were on the old per-beat setup.

The honest lesson is that the win did not come from the plan. It came from a reverted experiment and a setting nobody re-enabled. We will take it. But the reason we caught it at all is that we were reading the logs of real playthroughs closely enough to notice when the numbers changed. That habit is the actual asset, and it is the one we are keeping.