Android / On-device Training / GGUF

GGUF Generator

GGUF Generator is an Android on-device training tool that takes you from generating Q/A training data all the way to training a small model, exporting GGUF, and LoRA fine-tuning — all in one app. Generate training pairs from keywords, URLs, Wikipedia, or an LLM prompt, then create or refine GGUF models (F32/F16/Q8_0) directly on the device.

Docs: User Manual | Privacy Policy

        Generates Q/A training pairs with RAG evaluation from input sources: Brave Search / Wikipedia / a specified URL / an LLM theme prompt.
Supports data augmentation (translation expansion, paraphrase amplification), pair editing, and JSONL export.
Runs three on-device training modes — new GGUF creation, refine (continued training), and LFM2 LoRA (pure-C native) — and exports GGUF.

      

Screenshots

GGUF Generator training screen — Status header and processing log with pair generation, training (new/refine/LoRA), and GGUF export in a single screen.

GGUF Generator direct-run screen — Connect to the common LLM server (Ollama-compatible), pick generation and embedding models, and run.

Key Features

On-device training-data generation: From an input source (Brave Search / Wikipedia / URL / LLM theme prompt), it performs full-text chunking, RAG evaluation, and multi-pair generation to build Q/A training data (JSONL). Supports background execution with stop and resume.
Augmentation & editing: Amplify existing pairs via translation expansion (multilingual) and paraphrasing. Edit pairs directly; the edited result becomes the "latest" and feeds subsequent augmentation and training.
New GGUF creation: Train a small Transformer (pre-RMSNorm, RoPE, SwiGLU, tied output) from scratch on-device and export F32 / F16 / Q8_0 GGUF. Staged target losses let you tune training strength (generalization ↔ memorization).
Refine (continued training): Select an existing base GGUF, continue training, and export the updated GGUF. Model dimensions are inherited from the base.
LFM2 LoRA (pure-C native): Train a LoRA on an existing LFM2 GGUF in pure C, and export both a llama.cpp-compatible adapter GGUF and a merged full GGUF (loadable standalone).
Keyword evaluation & early stopping: Evaluate via manual keywords or automatic token extraction (top-surprisal) and stop training early once the pass rate reaches the threshold (shared across new / refine / LoRA).
Ollama-compatible server integration: Connects to the LLM server URL in common settings (default http://127.0.0.1:11434), using /api/tags to fetch models, /api/generate to generate, and /api/tokenize to tokenize. Can also start an external app's local API server.
Fully on-device: Outputs (JSONL / GGUF) are saved to the device's Downloads / Documents / app storage. No tracking or analytics are collected.

Operational Notes

Pair generation and augmentation involve communication with the user-configured LLM server (on-device by default), the Brave Search API, Wikipedia, or a specified URL. Content sent and destinations depend on your input and actions.
On-device training is CPU- and memory-intensive. For long runs, granting notification permission and battery-optimization exemption is recommended.
Quantization has dimension constraints (Q8_0 requires multiples of 32). Model dimensions for new creation may be automatically rounded or reduced.