ML · Generative · 2026

Handwriting Diffusion

A conditional diffusion model that generates handwriting from the text you give it and a reference writing style, using a frozen BERT text encoder, a CNN style encoder, and a DDPM-style UNet.

Generated handwriting sample from the conditional diffusion model. — A generated sample from the conditional diffusion model.

Problem

Good handwriting generation has to match two things at once: the exact words you ask for and the look of a particular writer.

Approach

I combined text encoding, style conditioning, caching, sampling tools, and the diffusion model into one training-and-generation pipeline.

Outcome

A model you can actually control, with concrete targets for both quality and speed: FID under 30 on IAM, 55% faster training throughput, and 30+ samples per second.