Android / On-device Training / GGUF

GGUF Generator

GGUF Generator is an Android app that covers everything from real-time chat-based training to automated data generation and quantization — all on-device. Interactive mode lets you grow a model through conversation, and every exported GGUF can be reloaded and trained further as a Retrainable GGUF.

Docs: User Manual | About Retrainable GGUF | Privacy Policy

        Interactive mode (New GGUF / LoRA) — teach the model in real time through a chat interface.
Pair editing & generation — add, edit, and derive Q/A pairs by prompt instruction (input source: a URL or one Wikipedia result). Saved pairs feed straight into interactive learning.
Quantize — convert any GGUF to a different quantization type (F32 / F16 / Q8_0 / Q6_K / Q5_K_M / Q4_K_M / Q2_K, etc.). Strip retraining metadata to produce a clean, distribution-ready GGUF.
All exported GGUFs are Retrainable — reload and keep training at any time.

      

Screenshots

Home screen. A card-based UI with five modes: Interactive (New GGUF), Interactive (LoRA), Quantize, Pair Edit & Gen, and Settings.

Interactive training — New GGUF — Interactive (New GGUF). Train the model through chat with NEW LEARN / RESPONSE LEARN / TRY INFERENCE. Inference speed (108.2 tok/s) is shown in the log.

Pair Edit & Gen. Generating 30 Harry Potter Q&A pairs from Wikipedia. Use ADD NEW PAIRS / EDIT PAIRS / DERIVE to build and refine the list.

Quantize. Check "Strip retraining metadata" to remove pair history and Adam optimizer state, producing a clean inference-only GGUF ready for distribution.

Register Profile dialog. Save the loaded GGUF with a name and base profile to manage and resume training sessions.

Key Features

Interactive training — New GGUF: Three chat actions — NEW LEARN (new instruction + response pair), RESPONSE LEARN (learn from last model output as a reply), TRY INFERENCE (check what the model says now) — let you grow a model in real time. Bad answers are automatically re-injected for correction. LOAD GGUF resumes a previous session; EXPORT GGUF writes the result.
Interactive LoRA: Fine-tune a LoRA on an existing GGUF (llama / gemma3 / qwen3 / LFM2) through conversation. EXPORT ADAPTER writes a llama.cpp-compatible adapter GGUF that embeds the training state (pairs + optimizer); resume via Load model then Load adapter. Temperature, max new tokens, and context length are configurable, and Think (reasoning output) can be disabled for Qwen3 during evaluation.
Register Profile: During interactive training, the "Register Profile" dialog lets you save the loaded GGUF with a name and base profile. Manage multiple training sessions by name and resume with the same settings.
Quantize: Convert any GGUF to F32 / F16 / Q8_0 / Q6_K / Q5_K_M / Q5_1 / Q5_0 / Q4_K_M / Q4_1 / Q4_0 / Q2_K and more. 1-D tensors (norm gains) stay F32 for accuracy; matrices whose row length is not block-aligned fall back to F16 automatically. Enable "Strip retraining metadata" to remove pair history and Adam state, producing an inference-only GGUF suitable for distribution.
Pair editing & generation: Work on the displayed pair list and, by prompt instruction to the LLM, "add new pairs", "edit selected pairs", or "derive" new pairs from a selection. Attach a URL or one Wikipedia result as source material; multi-select via checkboxes, clear and save are supported, and saved pairs feed interactive continuous learning with no file picker. (RAG evaluation and Brave search are currently disabled.)
Keyword evaluation & early stopping: Stop training early via manual keywords or automatic top-surprisal token extraction once the pass rate hits the threshold. Shared across all training modes.
Ollama-compatible server integration: Connects to the LLM server URL in settings (default http://127.0.0.1:11434). Also serves a built-in Web AI Chat for browser-based chat with any loaded model.
Retrainable GGUF: Every GGUF the app exports can be reloaded and trained further. Learn more about the format and how to use it.
Fully on-device: All outputs (JSONL / GGUF) are saved to the device's Downloads / Documents / app storage. No tracking or analytics collected.

Operational Notes

Pair generation and editing involve communication with the user-configured LLM server (on-device by default), Wikipedia, or a specified URL. Content sent and destinations depend on your input and actions.
On-device training is CPU- and memory-intensive. For long runs, granting notification permission and battery-optimization exemption is recommended.
Quantization has dimension constraints (Q8_0 requires multiples of 32). Model dimensions for new creation may be automatically rounded or reduced.