Qwen training tools
Fine-tuning Qwen for Story Copilot
Story Copilot needs to suggest the next reply in a tabletop game. A useful answer responds to the players, respects what their characters know, and leaves unresolved choices open. I built the data, training and evaluation tools to investigate whether a local model could do this better after fine-tuning.
Qwen3.8-27B · QLoRA / FSDP2 · Two RTX 3090s · llama.cpp
What changed in the training task
Earlier versions asked the storyteller to produce a structured object containing narration and other fields. I separated the jobs: the writer now returns prose, while another model extracts state changes from what the participants actually said. A suggested reply never becomes part of the recorded conversation on its own.
The data work then focused on complete responses to other speakers. An isolated narrator line is often a poor training target: the model needs the question, the relevant facts and any restriction on what a character knows. Reviews keep those connections and exclude uncertain boundaries or overlapping targets.
An example of the context and training target
In this invented scene, one player has a private clue and another has explicitly chosen to stay outside a room. The next reply must address both players without losing either condition.
Keep both players’ requests and the private fact
Training on Ada’s answer alone would omit the reason for it. This example keeps the scene, the private note and both player turns, including Bram’s decision to stay outside.
Input supplied to the storyteller
Established scene: At a harbor signal station, keeper Ada says the east ferry left at nine. The bell has been disconnected since yesterday. The brass tide dial is missing from its wall mount. Only Inez has seen a note saying Ada moved the dial to the workshop. She has not told Bram. Nobody knows why Ada moved it. Inez: I don’t mention the note. I ask Ada how the ferryman knew it was time to leave if the bell was disconnected. Bram: While they talk, I stay by the doorway and check whether I can see the workshop from here. I’m not going inside.
Authored example of a suitable continuation
Ada rubs a thumb over the pale circle where the dial used to hang. “He didn’t need the bell. I was down at the pier.” She glances at the dark windows before adding, “He usually waits for me.” Bram, the workshop door is visible across the courtyard. A lamp burns behind its frosted glass, but from the doorway you cannot make out anyone inside. Ada has left your way clear. What do you ask her next?
Ada’s claim about the pier is NPC dialogue, not independently established truth. The reply answers both players without revealing Inez’s note to Bram.
Fixed, self-authored fictional illustration. These passages explain the data contract; they are not private training excerpts, benchmark cases or generated model answers.
Results of the 1,250-example run
The latest run trained a fresh prose adapter for 4 hours 39 minutes on two 24 GB GPUs. I compared it with the untuned base on 40 validation cases, then a separate 40-case test. Each model produced one answer per case.
The adapter was preferred in validation: 24 comparisons to 13, with three ties. On the separate test, preference was tied at 19 each, with two ties. The result did not establish an overall storytelling improvement.
| Split / model | Pass | Fail | Uncertain |
|---|---|---|---|
| Validation · Untuned base | 25 | 12 | 3 |
| Validation · 1,250-example LoRA | 33 | 6 | 1 |
| Separate test · Untuned base | 27 | 8 | 5 |
| Separate test · 1,250-example LoRA | 30 | 9 | 1 |
The replies became much shorter: a median of 18.5 words on the test, compared with 159.5 from the base. Some improved by dropping unnecessary narration. Others lost part of the player’s request. Definite failures rose from eight to nine, even as the number of passes increased.
Median generation time fell from 7.47 to 1.90 seconds, largely because there was less text to generate. Validation loss also fell, from 2.608 to 2.017. Neither measurement resolves whether a person would find the next reply useful. The adapter remains experimental and is available in the private application for hands-on testing.
Review method and limits
Review checked whether the answer addressed the current request, preserved established facts and uncertainty, respected player choices and handled mechanics appropriately. A consistent new fictional detail was allowed; reproducing the reference continuation was not required.
All 80 scenarios were audited and independently cross-reviewed using Astra High before the run. The final answer reviewer was an assistant, with model identities hidden until each split’s grades were saved. It had previously seen reference continuations, so this was not a fully independent assessment. No human calibration was performed.
The cases come from four source families and include related characters and situations. There was one generation per case. This measures the writer with prepared context, not transcription accuracy or the complete live copilot.
Aggregate measurements ↗Training and serving on two GPUs
Fit the training job
QLoRA learns small weight updates while the quantized base stays frozen. FSDP2, CPU offload and activation checkpointing let the 27B training job run on two 24 GB cards with 128 GB of host RAM. Recomputing activations and moving data to RAM trade time for memory.
Check the exact training input
The dataset formatter checks source groups, repeated targets and the model’s own chat-template boundaries. Only the response and its end token contribute to loss. Required context is not silently cut to fit a token limit.
Verify the saved adapter
After training, the loader checks tensor names, shapes and values against the exported adapter. Conversion to GGUF checks that attention-head rearrangement preserves the low-rank update. A file existing on disk is not enough to show the model is using the intended weights.
Share the base at inference
The classifier, rules, storyteller and player adapters use the same quantized base. Each request enables its adapter and disables the others. This saves model memory, but requests can still queue and longer context still needs more memory.
Earlier SFT, DPO and agent-RL experiments
I also compared supervised recipes and direct preference optimization (DPO) for the earlier structured writer. Preference training improved some editorial judgments but did not fix the reliability problems. Those scores use a different response contract and should not be combined with the prose experiment into one learning curve.
A separate 4B model learned evidence decisions through supervised warm-up and GRPO reinforcement learning. On 36 authored scenarios at two seeds, it passed 66/72 attempts versus 56/72 after the same number of extra supervised updates. The checks concerned conclusions and citations, not storytelling. The RL extension took about twice as long; matching update counts did not match compute.
Run the process with your own data
The public repositories provide data preparation, training, reload checks, conversion, comparison reports and local serving. They include authored fixtures for exercising the tools. My training material, generated review traces and fine-tuned weights remain private.