Best Base Model for LoRA Training in 2026: I Trained the Same Two Faces on Six Models
I trained the same two faces on six base models (Ideogram, Flux.1 Dev, Flux.2, Klein, Krea, Z-Image) to find the best base model for character LoRA training in 2026.
A few days ago I got curious about which base model is actually the best at training LoRAs. I run a site that trains custom LoRAs (think AI headshots), it has been on Flux.1 Dev for a long time, and I wanted to know if the newer models were worth switching to. So I let my basement GPUs run for four days straight and trained the same two faces on six different models.
If you train character or headshot LoRAs and you have been living on Flux.1 Dev, Ideogram 4 is worth a serious look. It held the likeness on both an easy face and a hard one better than anything else I tested, and it is the model I am moving my own pipeline to. Flux.1 Dev is still a perfectly good default. Flux.2 Dev is the one to skip unless you have a big-VRAM card to burn. You can also try trained-model headshots on bestphotoAI, which is the site this whole experiment was for.
Why I bothered
Most of these models have been out for a while now, so it felt like a good time to compare them. The thing that pushed me over the edge was CivitAI. If you go looking for character LoRAs on the newer models, people just are not sharing them the way they did during the Flux.1 Dev days. Nobody had really done the boring, apples-to-apples version of this test.
I have a couple of RTX 4070 Ti SUPER cards (16GB each) sitting in my basement. With AI doing the driving now, the whole hassle of training is mostly gone: babysitting runs, fixing CUDA errors, tuning rank and learning rate and text-encoder learning rate. So I handed the entire pipeline to Claude, start to finish, and told it to fix its own problems when they came up. One card kept serving a production image model the whole time. The other one did the training.
The two subjects
First subject is a generic white woman, because most models have no trouble with that. The second is a South Asian man, and this is where it gets interesting. In my experience South Asian and Black faces get much worse resemblance, and the models love to overtrain and slide into racist caricature. You will see exactly that in a few of the runs below, usually at the higher step counts. Same reference photos went into every single model.
The two subjects
Same reference sets fed to every model. One easy face, one the models historically struggle with.
Woman
Man
How to read this (v1, v2, v3)
Fair warning on the versions. I only noticed after a lot of trainings were already done that my v1 and v2 sample prompts were too basic. A basic prompt has a nasty failure mode: if a model is overtrained, it just hands you back your input photos and everything looks great. That is a lie detector you want pointed the other way.
So v3 is the comparison that matters. The v3 prompts are more complex and put the person in novel scenarios (riding a bike through a vineyard, a neon Tokyo street, hiking an alpine ridge). That is how you find out if a model actually learned the face and can recreate it somewhere new, versus just memorizing a handful of pictures. I left v1 and v2 in so you can see the difference yourself.
Each model gets its own section below. Toggle the subject (woman or man) and the version, then use the step tabs to walk the checkpoints, newest first. The prompts are seed-locked, so every step renders the same scenes and you can watch the face lock in (and then, often, overcook). Hit "Config & prompts" on any model to see the exact rank, learning rate, and captions it trained on. Almost everything here fit in 16GB, which honestly surprised me. Do not ask me how, Claude figured out the quantization. Flux.2 Dev was the one exception, and it has its own sad story further down.
Ideogram 4
ideogram4 · fp8
Once I fed it its native JSON caption format, Ideogram 4 held the likeness on both people and stayed convincing on the harder, novel prompts. This is the one I'm moving my pipeline to.
Step 3,000 checkpoint · seed-locked prompts, so every step renders the same scenes.
Flux.1 Dev
flux · fp8
Still what my production runs on, still solid. But at fp8 on a 16GB card I can't push rank or resolution, and the woman run tops out at good-not-great. It's held up, it's just no longer the top.
Step 2,500 checkpoint · seed-locked prompts, so every step renders the same scenes.
Z-Image
zimage · fp8
Tiny and quick, about 90 minutes a run. It overfit at 3000 steps and looked fine only on training-like prompts, but dropping to 1500 steps fixed it. Impressive for how small it is; the hard face stayed a touch soft.
Step 1,500 checkpoint · seed-locked prompts, so every step renders the same scenes.
Flux.2 Klein
flux2-klein-9B · fp8
The 9B Klein fits in about 4GB of VRAM and trains fine. Early runs literally lightened the man's skin; caption fixes plus rank 64 pulled it back to a believable likeness.
Step 2,000 checkpoint · seed-locked prompts, so every step renders the same scenes.
Krea
krea2 · fp8
No drama, it just trains. But it's the slowest sampler in the group, around two minutes per preview image, and it never gave me a result that beat the others. Solidly mid.
Step 2,000 checkpoint · seed-locked prompts, so every step renders the same scenes.
Flux.2 Dev
flux2 · 24B + Mistral TE
The 24B model plus a 24B Mistral text encoder makes this a big-VRAM, real-money problem. Two RunPod attempts looked bad, so I quit partway. The only model here I don't have a finished LoRA for.
Step 2,000 checkpoint · seed-locked prompts, so every step renders the same scenes.
Things I did not expect to learn
The single biggest quality lever was not rank or steps. It was captions. When the captions described the person's face, the identity stopped binding to the trigger word and resemblance fell apart. Captions that only described the scene, and let the trigger word own the face, were the biggest jump in the whole test.
The stuff that actually moved the needle
- Ideogram wants its own JSON caption schema. Feeding it plain-text captions gave artifact-ridden results. Its native structured format took it from mediocre to the best model in the test on both faces.
- Novel prompts are the only honest overfit test. Z-Image at 3000 steps looked perfect on training-like prompts, then collapsed on new scenes. Dropping it to 1500 steps fixed it. That is the only 1500-step run in the whole bench.
- Caching text embeddings is the 16GB trick. Encode the captions once, then unload the text encoder before training. On Flux.1 and Ideogram that was the single biggest speedup and the reason a 16GB card can do this at all.
- There is a real 6x speed spread on identical hardware. Z-Image trains at about 2 seconds an iteration with 19-second previews. Krea is nearly 4 seconds an iteration with previews over a minute and a half each.
- Flux.2 Klein literally lightened the man's skin on an early run before the caption and rank fixes. Worth staring at if you care about who these models work well for out of the box.
The Flux.2 Dev saga
Flux.2 Dev was impossible on my hardware, and it is worth explaining why, because it is not a normal VRAM problem. The model is a 24B transformer paired with a 24B Mistral text encoder, and to quantize it you need to hold both in system RAM at once. That is roughly 90GB of RAM, not VRAM. The 9B Klein sibling fits in about 4GB of VRAM; the 24B Dev would not load a single layer even at the smallest resolution with everything offloaded to CPU.
So I rented GPUs on RunPod. The short version: a 4090 was too small on RAM, a couple of 5090s either failed to init or quantized too slowly to be worth it, an A100 worked but I killed it as too costly, and I finally landed on a 96GB RTX PRO 6000 at about two dollars an hour. I ran two trainings, the results looked bad, and I quit partway rather than keep paying to confirm what I already suspected. It is the only model in this test I do not have a finished LoRA for. You can see the partial samples in its section above.
The scoreboard
| Model | Architecture | Fits 16GB? | v3 run time* | Where it landed |
|---|---|---|---|---|
| Ideogram 4 | ideogram4 + native JSON captions | Yes (fp8) | ~4.5 h | #1 winner |
| Flux.1 Dev | Flux, T5 text encoder | Yes (fp8) | ~1.8 h | #2, hit its ceiling |
| Z-Image | small, fast arch | Yes (fp8) | ~1.5 h | #3, fastest by far |
| Flux.2 Klein | 9B, fits in ~4GB VRAM | Yes (fp8) | ~4.3 h | #4 |
| Krea | krea2 | Yes (fp8) | ~4 h | #5, slow and mid |
| Flux.2 Dev | 24B + 24B Mistral TE | No (~90GB RAM) | cloud only | #6, DNF |
*Wall-clock for a v3 run on one RTX 4070 Ti SUPER 16GB, read from the training log's last checkpoint. Not a controlled speed benchmark, just what these actually took me.
My verdict
I am probably switching my main pipeline from Flux.1 Dev to Ideogram. Outside of a few wonky results, it is a huge improvement over what Flux.1 gives me, on both the easy face and the hard one. The only open question is speed on production hardware. I run my Flux.1 Dev LoRAs on an H100, and I genuinely do not know yet how long an Ideogram run takes there. That is the next thing I need to measure before I flip the switch.
Final ranking
After four days of GPU time and about thirty training runs, this is how I would order them for training a character or headshot LoRA today.
If you just want the trained-headshot result without owning a GPU, that is what bestphotoAI does, and it is the reason I ran this test in the first place. Everything above is the unglamorous work behind picking which base model sits under a product like that.
FAQ
Frequently Asked Questions
Which base model is best for training a character LoRA right now?+
Can you really train these on a 16GB GPU?+
Why did the South Asian man's LoRA turn into a caricature?+
How do I tell if a LoRA is overtrained?+
So is Flux.2 Dev actually bad?+
What captioning approach worked best?+
More from the Blog
Best Free AI Tools for Marketing in 2026
The best free AI tools for marketing in 2026, sorted by job: copywriting, research, design, ad creative, social scheduling, motion, and AI-answer visibility.
Best Spotify Alternatives in 2026
Spotify keeps raising prices while lossless lags. We compared the best Spotify alternatives on price, sound quality, catalog, and free tiers to find your fit.