Investigation of diffusion-based editing applied to data augmentation in image classification tasks

Investigation of diffusion-based editing applied to data augmentation in image classification tasks

Matiushin D.A. (ISP RAS, Moscow, Russia; MIPT, Dolgoprudny, Moscow Region, Russia)
Bykov O.I. (ISP RAS, Moscow, Russia)
Bubenkov I.I. (ISP RAS, Moscow, Russia)

Abstract

Modern image classification tasks are often constrained by class imbalance, limited data volume, and insufficient sample representativeness, all of which significantly reduce the generalization ability of models. Traditional augmentation methods (such as geometric transformations and color modifications) introduce only local, semantically limited variations. This work presents a systematic study of image editing based on vision-language models (VLMs) and diffusion architectures as an effective approach to semantic augmentation. A comparative analysis is conducted between two contemporary VLM-based image editing pipelines (Instruct-Pix2Pix and Qwen-Image-Edit) and both classical and modern diffusion-based augmentation methods on the CUB-200-2011 dataset (with artificially induced imbalance) and a 13-class subset of iWildCam (with natural imbalance). Five strategies for selective image sampling with varying degrees of adaptability to imbalance structure are developed, and the impact of different types of textual instructions as well as key system hyperparameters is evaluated. The results demonstrate the high effectiveness of semantically rich VLM-based editing in improving the robustness of classifiers under conditions of pronounced class imbalance.

Keywords

data augmentation; diffusion models; visual-language models; image editing; class imbalance; image classification.

Edition

Proceedings of the Institute for System Programming, vol. 38, issue 6, part 2, 2026, pp. 249-274

ISSN 2220-6426 (Online), ISSN 2079-8156 (Print).

DOI: 10.15514/ISPRAS-2026-38(6)-36

For citation

Matiushin D.A., Bykov O.I., Bubenkov I.I. Investigation of diffusion-based editing applied to data augmentation in image classification tasks. Proceedings of the Institute for System Programming, vol. 38, issue 6, part 2, 2026, pp. 249-274 DOI: 10.15514/ISPRAS-2026-38(6)-36.

Full text of the paper in pdf (in Russian) Back to the contents of the volume