QLoRA Fine-Tuning for Plain-English Rewriting
QLoRA fine-tuning of a small open-source model to rewrite complex text in plain English, trained on a free Colab GPU. It tests whether a 1.5B-parameter model, tuned for under an hour, can do the job more reliably than the same model with prompting alone.
View the code on GitHubFull results
- reading grade on ASSET, against 8.38 for the base model
- 7.53
- of numbers kept, against 0.89 for the base model
- 0.92
- of the model's parameters trained
- 1.2%
Problem
Technical writing is often hard to read for the people who need it. Large hosted models can simplify it, but they cost money per call and send your text to a third party.
Approach
- The base model is Qwen2.5-1.5B-Instruct, trained with 4-bit QLoRA using PEFT and TRL.
- Training data is 20,000 cleaned sentence pairs, filtered for length, encoding, near-copies, readability, number changes and meaning.
- No LLM API was used to create data, and the project makes no paid API calls.
- Base and fine-tuned models run in the same 4-bit setup on 359 unseen ASSET sentences and 20 hand-written technical passages.
Results
- Run 1 lost too many numbers, so run 2 rejected training pairs that drop a number. Numbers kept rose from 0.68 to 0.92.
- Output is 81% of the source length, while the base model writes 116%.
- There was no readability gain on technical text: the difference is within noise on only 20 sentences.
- It still invents facts. Six ASSET outputs contain an invented number, and the README shows examples.
Limitations
- Training data is encyclopedia text, not technical documentation.
- The model can drop details, change facts and invent figures.
- Each result comes from one training run.
- Do not use it for medical, legal or safety text without expert review. Parts of the data have non-commercial terms.
Built with
- Python
- PEFT
- TRL
- QLoRA
- Qwen2.5
- Hugging Face
- Gradio
- Google Colab