Vishal Asnani

Vishal Asnani

Research Scientist at Adobe

Trustworthy AI · Watermarking · Media provenance

Research / 2023

Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Images

Vishal Asnani, Xi Yin, Tal Hassner, Xiaoming Liu

Published in IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023

Read paper Code repository All publications
Model parsing goes beyond real-or-fake detection and source attribution to infer model hyperparameters using fingerprint estimation and parsing networks.
Model parsing estimates properties of an unseen generator from its output, rather than only classifying an image as real or fake. Figure 1 from the authors' July 2023 manuscript accompanying the TPAMI article. Select the figure to enlarge it.

Overview

What can a generated image reveal about the model behind it? This work defines model parsing: estimating generator architecture and training-loss attributes from generated images, including images from models unseen during training.

How it works

  • A Fingerprint Estimation Network learns traces associated with the image’s generator.
  • A Parsing Network uses those fingerprints to predict architecture and loss-function attributes.
  • Evaluation holds out entire generative models, rather than only holding out images from models already used in training.

A reported result

0.813 ± 0.019 F1 for loss-function prediction in Table 5 of the authors’ July 2023 manuscript, versus 0.751 ± 0.027 for the mean/mode baseline.

The result is averaged over four model-held-out splits, each training on 104 models and testing on 12. Section 4.1 describes a benchmark of 116,000 images from 116 generative models.

Scope and source version

Model parsing estimates selected attributes; it does not reconstruct a generator’s exact weights or complete source code. The figure and quantitative details here come from the authors’ July 2023 manuscript, version 3. The primary reading link remains the official TPAMI record.

Sources and resources

Published paper Supplementary GitHub Dataset

Bibliographic reference

Asnani, V., Yin, X., Hassner, T., & Liu, X. (2023). Reverse engineering of generative models: Inferring model hyperparameters from generated images. Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 12, pp. 15477-15493.