GGUF Generator / Proprietary Format

Retrainable GGUF

Every GGUF exported by GGUF Generator is a Retrainable GGUF — reload it any time and keep training right where you left off. Where a standard quantized GGUF is inference-only, a Retrainable GGUF embeds the training corpus history and the Adam optimizer state alongside the model weights, enabling seamless, true continuation of learning.

        A world first

        No other tool in the llama.cpp ecosystem embeds training corpus history and full Adam state inside a GGUF file to enable complete continuation training on reload.

How It Differs from a Standard GGUF

Aspect	Standard quantized GGUF	Retrainable GGUF
Primary use	Inference (text generation)	Inference + continued training
Weight precision	Quantized (Q4, Q8, etc.)	F32 (full precision), F16, or Q8_0
Continued training	Not supported	Resume instantly via Load GGUF
Forgetting prevention	Not applicable	All corpus pairs replayed on every training step
True continuation	Not possible	Adam state carried over for smooth resumption
Architecture	LLaMA / Mistral / LFM2, etc.	llama.cpp-compatible small Transformer (proprietary design)
llama.cpp compatibility	Compatible	GGUF format compatible; custom metadata is ignored as unknown KV

Embedded Metadata Specification

A Retrainable GGUF stores three categories of data in fields that llama.cpp ignores as unknown KV entries. Inference is completely unaffected.

① Training corpus history (`training.pairs`)

Every instruction/output pair taught so far is stored as tab-separated, newline-separated text. On reload the corpus buffer is restored, so all previous pairs are available for the next training run.

training.pairs = "Good morning\tGood morning!\nHow are you?\tI'm fine, thanks\n…"

② Adam optimizer state (`training.adam_t / adam_m / adam_v`)

The first moment m, second moment v, and step count t are saved as raw float32 arrays. This is what makes true continuation possible: bias correction picks up at the right step, so learning resumes smoothly with no warm-up overhead.

training.adam_t = 1420　// step count
training.adam_m = float32[P]　// first moment (P = parameter count)
training.adam_v = float32[P]　// second moment

The Adam state is two float32 arrays each the same size as the model. The Retrainable GGUF is roughly 3× the size of the raw model weights (weights + m + v). For a tiny model of ~1M parameters at F32, expect a few MB total.

③ Training metadata

training.samples　training.epochs　training.learning_rate　training.final_loss

Retraining Flow in Interactive Mode

The most direct way to use a Retrainable GGUF is Interactive (New GGUF) mode.

① Start

Tap "New" to create a fresh model, or "Load GGUF" to reload an existing Retrainable GGUF.

② Train

"New Learn" (two-step: instruction → response) updates the weights immediately. Every step replays the full corpus to prevent forgetting.

③ Verify

"Try Inference" checks the current response. Wrong answers? Tap "Re-learn" to reinforce the last pair.

④ Export

"Export GGUF" saves a Retrainable GGUF. Reload it next session to pick up exactly where you stopped.

"Export GGUF" lets you choose the precision (F32 / F16 / Q8_0). F32 is best for continued training but produces larger files. Q8_0 is a good balance between precision and size.

Catastrophic Forgetting Prevention

Every time you add a new pair in interactive learning, the model is retrained on the full corpus. This prevents the model from forgetting earlier pairs when new ones are added.

Action	Corpus	What gets retrained
Teach pair 1	[A→B]	[A→B] — 1 pair
Teach pair 2	[A→B, C→D]	[A→B, C→D] — all 2 pairs
Teach pair 3	[A→B, C→D, E→F]	[A→B, C→D, E→F] — all 3 pairs

Each training step gets heavier as the corpus grows, but this approach works well at the scale of tens of pairs typical for a small interactive model. Use Batch Pair Import to handle larger datasets efficiently.

Four Continuation Patterns

① Resume interactive learning (Load GGUF → keep teaching)

Loading a Retrainable GGUF restores the corpus history and Adam state. You can immediately continue with "New Learn" — no warm-up, no re-teaching.

② Batch pair import

Import a JSONL file to bulk-add pairs, rebuild the BPE vocabulary, and retrain on the entire corpus. Set the target loss slider to stop automatically when the model converges.

③ Continued training with vocabulary expansion

BPE subword models keep the vocabulary fixed (unknown characters fall back to byte tokens). Character-level models extend the vocabulary for new characters and expand the embedding table, preserving all existing weight rows.

④ LoRA continuation (resume from adapter)

The same mechanism applies to LoRA interactive learning. "Load model" → "Load adapter" restores the adapter A/B matrices and their Adam state, letting you resume LoRA fine-tuning seamlessly.

Architecture Details

The small Transformer targeted by Retrainable GGUF is described with LLM_ARCH_LLAMA-compatible GGUF metadata:

pre-RMSNorm: RMS normalization before each block (ε = 1e-5, gain vector included)
RoPE (NORM/interleaved): rotary position encoding, freq_base = 10 000
SwiGLU: gating function in the feed-forward layer (gate × silu(gate) × up → down)
Tied output: embedding and output projection share token_embd.weight
Single-head attention: head_dim = dModel (current implementation)
Configurable dimensions: dModel / ffHidden / layers / ctx are set at creation. Verified to work at ~1M parameters.

The exported GGUF runs on llama.cpp and the llama.cpp Android server app without modification.