User Manual

GGUF Generator takes you from training-data generation to on-device training and GGUF export in a single app. The screen has a fixed status/log area at the top and collapsible sections below.

1. Common Settings

• Set the LLM AI server URL (default http://127.0.0.1:11434) and tap "Fetch" to load the model list (also fetched automatically at startup).
• Select the generation model (LLM) and embedding model (RAG). "Start Server" can launch an external local API server; the status tile shows connectivity.
• Choose the default save location (Downloads / Documents). Individual file references use the standard system dialog.

2. Pair Generation

1. Input source: Choose Brave Search / Wikipedia / a URL / an LLM theme prompt, and enter a keyword or URL. Only the fields relevant to the source are shown.
2. Run: Run "Generate (FG)" (in-app) or "Generate (BG)" (background). Results are saved even if you stop. You can also append to an existing JSONL.
3. Generation options (optional): Tune sample count and acceptance threshold, and edit the prompt sent to the LLM.
4. Additional processing: augmentation (optional): Expand the current pairs via translation expansion (multilingual) or paraphrase amplification.
5. Post-processing: Edit pairs, copy output, or save JSONL. The edited result becomes the "latest" and feeds subsequent augmentation and training.

3. Early Stopping & Evaluation (optional, shared across all training modes)

• When "Enable early stopping / evaluation" is on, training stops early once the pass rate reaches the threshold (default 60%).
• Choose how expected values are specified: manual keywords ("input => keywordA|keywordB", one per line) or automatic token extraction (top-surprisal).
• You can set the pass threshold, evaluation interval, max generated tokens, and number of evaluation cases.

4. Training & GGUF Export

Specify the training data (uses the latest generated/edited result if none selected) and the save location, then pick a training method from the two subsections.

GGUF new creation / refine

• Select "New creation" or "Refine (continued training)" at the top.
• New creation: Set model dimensions, staged target losses, and template, then train a small model from scratch and export F32 / F16 / Q8_0 GGUF.
• Refine: Select a base GGUF and continue training (dimensions inherited from the base).

LoRA (LFM2, on-device, pure-C native)

• Select a base LFM2 GGUF and set rank / alpha / lr / epochs / targets to train a LoRA.
• Exports a llama.cpp-compatible adapter GGUF and a merged full GGUF (loadable standalone). You can also set the merged model card (name / description).

Tips

• For staged target losses, larger values mean weaker training (generalization) and smaller values mean stronger training (memorization).
• On-device training is CPU- and memory-intensive. For long runs, grant notification permission and battery-optimization exemption.
• Check progress in the log area at the top.