Adaptive high-resolution image synthesis based on diffusion transformers

Adaptive high-resolution image synthesis based on diffusion transformers

Ananev V.V. (ISP RAS, Moscow, Russia)
Efimov E.V. (ISP RAS, Moscow, Russia)
Zemnukhov E.S. (ISP RAS, Moscow, Russia)
Timakova A.F. (Sechenov University, Moscow, Russia)
Karpulevich E.A. (ISP RAS, Moscow, Russia)

Abstract

The paper addresses adaptive high-resolution image synthesis for digital pathology. The problem is caused by the shortage of annotated histology data, expensive expert labeling, covariate shift between whole-slide imaging devices and severe class imbalance in tasks involving minor morphological structures. We propose a generative pipeline based on a latent diffusion transformer. The pipeline combines image encoding into a latent space, visual conditioning using features extracted by the UNI2-h pathology foundation encoder, attention-layer optimization through key-value compression and domain specialization with low-rank LoRA adapters. We also present a context-oriented object insertion algorithm that performs spatial and chromatic alignment, semantically conditioned diffusion from an intermediate latent state and boundary-aware morphological blending. The software implementation is organized as a modular system with independent input-output, latent-feature encoding, generative and downstream data preparation components. Experiments are designed for blood vessel segmentation and lymphovascular invasion classification on whole-slide image fragments. The results confirm that targeted synthetic data generation can improve downstream model quality under data scarcity and class imbalance. The experiments were carried out using a database of 449 WSIs consisting of the DHMC, LCNO and NLST datasets, where 216 WSIs are annotated; vessel segmentation used 8212 masks converted into 5764 images, while invasion classification used 1875 images. The synthetic extension included 5172 images for segmentation and 4520 images for classification. The applied hardware and software optimizations reduced the epoch time from 1459.4 to 161.2 min and peak VRAM consumption from 48.20 to 32.65 GB; FID was 15.6866, KID×10³ was 4.9407, and the addition of synthetic data increased the UNet++ IoU to 0.7376±0.0014 and the EfficientNet-b1 F1-score to 0.9577±0.0090.

Keywords

latent diffusion model; diffusion transformer; image synthesis; visual conditioning; LoRA adapter; inpainting; gigapixel images; digital pathology.

Edition

Proceedings of the Institute for System Programming, vol. 38, issue 4, part 1, 2026, pp. 225-238

ISSN 2220-6426 (Online), ISSN 2079-8156 (Print).

DOI: 10.15514/ISPRAS-2026-38(4)-12

For citation

Ananev V.V., Efimov E.V., Zemnukhov E.S., Timakova A.F., Karpulevich E.A. Adaptive high-resolution image synthesis based on diffusion transformers. Proceedings of the Institute for System Programming, vol. 38, issue 4, part 1, 2026, pp. 225-238 DOI: 10.15514/ISPRAS-2026-38(4)-12.

Full text of the paper in pdf (in Russian) Back to the contents of the volume